With the rapid development of AI technology and advancements in natural language processing, building powerful large language models has become increasingly important for enterprises. Graph database software, as a robust tool for handling complex data structures, provides strong support for constructing domain-specific large language models.
 
NebulaGraph Database, an [open-source distributed graph database](https://www.nebula-graph.io/) built for super large-scale graphs with milliseconds of latency, released its Graph + AI big models solution. It pioneered the Text2Cypher technology for building and querying knowledge graphs based on natural language, offering users powerful capabilities in data management, semantic understanding, and information extraction. This solution enables enterprises to achieve higher-performance domain-specific large language model applications at lower costs, resulting in more intelligent and precise human-computer interactive question-answering results.
 

# NebulaGraph Database: Introducing the Graph + AI Solution
 
Graph database is a novel type of database that represents entities as nodes and relationships between entities as edges, allowing efficient storage, retrieval, and analysis of complex multidimensional data. Practice has shown that constructing knowledge graphs using graph technology enhances In-Context Learning, providing users with more contextual information and enabling large language models to better comprehend relationships between entities and enhance their expressive and reasoning abilities. As a graph database vendor to adopt LangChain, NebulaGraph Database has successfully implemented the Graph In-Context Learning solution based on a Knowledge Graph + Vector DB. Moreover, NebulaGraph Database is dedicated to introducing GraphStore storage context into the Llama Index, incorporating external storage of knowledge graphs and creating a more efficient and user-friendly Graph + LLM solution.
 
In terms of interaction, NebulaGraph Database has already implemented Text2Cypher based on the Graph + LLM solution. Users can easily build and query knowledge graphs through natural language in the dialogue interface. With out-of-the-box enterprise-level services, enterprise users can import massive data into NebulaGraph Database and quickly construct domain-specific knowledge graphs. Leveraging the powerful querying capabilities and performance of NebulaGraph Database, users can accomplish highly accurate searches and intuitive visualizations at a lower cost. Moreover, they can directly interact with the system through natural language for interactive questioning and querying, significantly reducing the barriers to enterprise adoption.
 
# NebulaGraph AI Large Models Solution: Industry Use Cases
 
In the era of information explosion, enterprises need to handle vast amounts of natural language text from different channels and types to obtain valuable information and insights. Large language models (LLMs) tailored to specific industry domains are capable of understanding, analyzing, and generating text information related to the industry.
 
However, traditional training methods suffer from high costs, low efficiency, and insufficient context information, making it challenging to effectively implement large language models in production environments. NebulaGraph Database, with its ability to handle massive, diverse, and complex data scenarios, provides a solution that addresses these issues.
 
## Healthcare Industry

In the healthcare industry, effective management and analysis of vast amounts of medical literature, clinical data, and patient records are essential. With the assistance of NebulaGraph Database's graph technology, enterprises can construct large language models specific to the healthcare domain. By constructing a medical knowledge graph that models medical entities (such as diseases, drugs, treatment methods) and their relationships, large language models can acquire extensive medical knowledge.
 
Using NebulaGraph Database helps accurately link entities mentioned in medical texts to the knowledge graph, eliminating ambiguities and improving model accuracy. Large language models in the healthcare industry can be applied to intelligent diagnosis, disease prediction, and personalized healthcare recommendations, providing more precise and intelligent solutions.
 
## Financial Industry

In the financial sector, in-depth analysis and prediction of complex financial data are required. NebulaGraph Database can assist in building a knowledge graph specific to the financial domain, modeling financial entities (such as stocks, transactions, financial indicators) and their relationships. Language models built on top of NebulaGraph Database can acquire professional knowledge in the financial domain and play a vital role in financial data analysis, investment decision-making, and more.
 
Furthermore, using NebulaGraph's graph technology for relationship extraction and semantic understanding can help extract key information from financial news and research reports, aiding large language models in better understanding the dynamics and trends of the financial market. Large language models in the financial domain can be applied to investment analysis, risk management and intelligent customer service, and users can receive precise answers through natural language, providing more intelligent and efficient services for financial users.
 
## E-commerce retail industry

In the retail industry, businesses often need to deal with a large amount of product information, user reviews, sales data, and more. NebulaGraph Database can help build a knowledge graph for the retail industry, modeling entities such as products, brands, users, and their relationships. Domain-specific large language models in the retail sector can be applied to intelligent product recommendations, customer segmentation, market trend analysis, and more, providing a more intelligent and personalized shopping experience for retail companies.
 
At the same time, NebulaGraph Database's Graph + AI Large Models solution can also play a role in areas such as product recommendations and user-personalized services by leveraging professional knowledge in the retail industry. Entity linking and relationship extraction using graph technology can extract useful information from user reviews, helping the large language models better understand user demands and offer personalized shopping preferences. Users can receive targeted services using natural language, greatly enhancing the user experience.
 
# Future Outlook: Building more intelligent AI large model applications
 
With the continuous development of big data and artificial intelligence technology, the deep integration of graph technology and language models will become a future trend. NebulaGraph Database not only provides a richer knowledge base and semantic understanding capability to language models, helping them better understand industry knowledge and semantics, but also enables more efficient, flexible, and intelligent processing of massive complex relational data.
 
In the future, AI large models will be further developed in various industry domains, and NebulaGraph Database will continue to advance and improve its technological advantages, helping businesses better understand industry data, gain insights into market trends, optimize business decisions, and bring more opportunities and momentum for innovation and development.


This article demonstrates how the NebulaGraph database offers robust support for building domain-specific large language models.

NebulaGraph x LLM：Pioneering the New Paradigm of "Graph + AI" Applications

# LLM App Paradigms

As a big improvement in Cognitive intelligence, LLM had changed many industries, in a way that we didn’t expect to automate, accelerate or enable. Seeing new LLM-enabled applications being created every day, we are all still exploring new methods and use cases for leveraging this magic.

One of the most typical patterns to bring LLM into the loop is to ask LLM to understand things based on proprietory/ certain domain knowledge. For now, there are two paradigms we could add that knowledge to LLM: **fine-tuning and in-context learning**.

Fine-tuning refers to performing add-on training on LLM models with extra knowledge, whereas in-context learning is to adding some extra piece of knowledge to the query prompt. What we observe now is that in-context learning has gained popularity over Fine-tuning due to its simplicity.

And in this blog, I’ll share what we had been doing around the in-context learning approach.

# Llama Index: Interface between data and LLM

## In-context learning

The basic idea of in-context learning is to use existing LLM(not updated) to handle special tasks toward specific knowledge datasets.

For instance, to build an application to answer any questions about one person, or even act as one’s digital avatar, we can apply in-context learning to an autobiography book with LLM. In practice, the application will construct a prompt with the question from the user and some information “searched” from the book, then query the LLM for an answer.

```
┌───────┐ ┌─────────────────┐ ┌─────────┐
│ │ │ Docs/Knowledge │ │ │
│ │ └─────────────────┘ │ │
│ User │─────────────────────────────────────▶ LLM │
│ │ │ │
│ │ │ │
└───────┘ └─────────┘
```

One of the most performant ways to enable this searching approach to get the related info from the Docs/Knowledge(the book in the above example) for the special task, is to leverage Embeddings.

## Embedding

The embedding normally refers to a way to map real word things into a vector in a multidimensional space, for instance, we could map images into a space of (64 x 64) dimension, and if we are doing it well enough, the distance between the two images can reflect the similarity of them.

Another example of embedding is the word2vec algorithm, which literally maps every word into a vector, for instance, and if the embedding is good enough, we could have addition and subtraction on them, we may have:

* `vec(apple) + vec(pie) =~ vec("apple apie")`
Or the vector measure of `vec(apple) + vec(pie) - vec("apple apie") `tends to be 0:
*` |vec(apple) + vec(pie) - vec("apple apie")| =~ 0`
Similarly, we could have “pear” should be closer than “dinosaur” to “apple”:
* `|vec(apple) - vec(pear)| < |vec(apple) - vec(dinosaur)|`
With that, we could in theory search for pieces of the book which are more related to a given question. And the basic process is:
* Split the book into small pieces, create the embedding per each piece, and store them
* When a question comes, compute the embedding of the question
Find top-K similar embeddings of pieces of the book by calculating the distance
* Construct the prompt with both the question and the pieces of the book
* Query the LLM with the prompt

```
 ┌────┬────┬────┬────┐ 
 │ 1 │ 2 │ 3 │ 4 │ 
 ├────┴────┴────┴────┤ 
 │ Docs/Knowledge │ 
┌───────┐ │ ... │ ┌─────────┐
│ │ ├────┬────┬────┬────┤ │ │
│ │ │ 95 │ 96 │ │ │ │ │
│ │ └────┴────┴────┴────┘ │ │
│ User │─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─▶ LLM │
│ │ │ │
│ │ │ │
└───────┘ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ └─────────┘
 │ ┌──────────────────────────┐ ▲ 
 └────────┼▶│ Tell me ....., please │├───────┘ 
 └──────────────────────────┘ 
 │ ┌────┐ ┌────┐ │ 
 │ 3 │ │ 96 │ 
 │ └────┘ └────┘ │ 
 ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ 
```

## Llama Index

Llama Index is such an open-source toolkit to help in-context learning in best practice:

* It comes with [a bunch of data loaders](https://llamahub.ai/) to serialize docs/knowledge in a unified format, think of PDF, Wikipedia Page, notion, Twitter, etc, and we don’t have to deal with the preprocessing, split the data into pieces, etc, on our own.
* It helps create the Embedding(and some other form of the index) for us and stores the embeddings(in memory or [vector databases](https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/Using_vector_databases_for_embeddings_search.ipynb)), too, with one line of code.
* It comes out of the box with prompts and other engineering points, so that we don’t have to create and study from scratch, for instance, [create a chatbot on existing data with 4 lines of code](https://twitter.com/jerryjliu0/status/1663213212932902913).


# The problem of doc split and embeddings

The embedding and vector search worked well in many cases, while there are still challenges in some cases, and one of them is it could lose global context/cross-node context.

Think of we are asking “Please tell me things about the author and foo.”, and in this book, the piece with numbers: 1, 3, 6, 19~25, 30~44, and 96~99 are all about the topic of foo. In this case, the simple way of searching top-k embedding of the pieces of the book may not work well because we normally only take a few top-related pieces, which loses many contexts.

```
┌────┬────┬────┬────┐
│ 1 │ 2 │ 3 │ 4 │
├────┴────┴────┴────┤
│ Docs/Knowledge │
│ ... │
├────┬────┬────┬────┤
│ 95 │ 96 │ │ │
└────┴────┴────┴────┘

```

The mitigation for that, for instance with Llama Index is to create [composite indices](https://gpt-index.readthedocs.io/en/latest/how_to/index_structs/composability.html), where the VectorStore is only part of it, combining with that, we could define a summary index and/or a tree index, etc to [route different types of questions to different indices](https://gpt-index.readthedocs.io/en/latest/guides/tutorials/unified_query.html), thus to avoid risking the loss of global context when the question required it.

Or, with the help of Knowledge Graph, we could do something differently.

# Knowledge Graph

The term Knowledge Graph was initially coined by Google in May 2012 as part of its efforts to enhance search results and provide more contextual information to users. The Knowledge Graph was designed to understand the relationships between entities and provide direct answers to queries rather than just returning a list of relevant web pages.

A knowledge graph is a way of organizing and connecting information in a graph format, where nodes represent entities, and edges represent the relationships between those entities. The graph structure allows for efficient storage, retrieval, and analysis of data.

It looks like this:

![knowledge graph](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/llm/llm04.png)

But how could Knowledge Graph help?

# Combination of embeddings and Knowledge Graph

The general idea here is a knowledge graph, as the refined format of the information, can be queried/searched in way smaller granularity than the split we could do on raw data/docs. Thus, by not replacing the large pieces of the data, but combining the two, we can search queries that require global/cross-node context better.

See the following diagram, assume the question is about x, and 20 of all the pieces of the data are highly related to it. We could now still get the top 3 pieces of the doc(say, no. 1, 2, and 96) as the main context to be sent, apart from that, we ask for two hops of graph traversal around x from the knowledge graph, then the full context will be:

* The question “Tell me things about the author and `x`
* Raw doc from piece number 1, 2, and 96, in Llama Index, it’s called node 1, node 2, and node 96.
* Knowledge 10 triplets contain `x` in two-depths graph traversal:
* x -> y(from node 1)
* x -> a(from node 2)
* x -> m(from **node 4**)
* x <- b-> c(from **node 95**)
* x -> d(from node 96)
* n -> x(from **node 98**)
* x <- z <- i(from node **1 and node 3**)
* x <- z <- b(from node **1 and node 95**)

```
┌──────────────────┬──────────────────┬──────────────────┬──────────────────┐
│ .─. .─. │ .─. .─. │ .─. │ .─. .─. │
│( x )─────▶ y ) │ ( x )─────▶ a ) │ ( j ) │ ( m )◀────( x ) │
│ `▲' `─' │ `─' `─' │ `─' │ `─' `─' │
│ │ 1 │ 2 │ 3 │ │ 4 │
│ .─. │ │ .▼. │ │
│( z )─────────────┼──────────────────┼──────────▶( i )─┐│ │
│ `◀────┐ │ │ `─' ││ │
├───────┼──────────┴──────────────────┴─────────────────┼┴──────────────────┤
│ │ Docs/Knowledge │ │
│ │ ... │ │
│ │ │ │
├───────┼──────────┬──────────────────┬─────────────────┼┬──────────────────┤
│ .─. └──────. │ .─. │ ││ .─. │
│ ( x ◀─────( b ) │ ( x ) │ └┼▶( n ) │
│ `─' `─' │ `─' │ │ `─' │
│ 95 │ │ │ 96 │ │ │ 98 │
│ .▼. │ .▼. │ │ ▼ │
│ ( c ) │ ( d ) │ │ .─. │
│ `─' │ `─' │ │ ( x ) │
└──────────────────┴──────────────────┴──────────────────┴──`─'─────────────┘
```


And clearly, the refined information related to topic `x` that comes from both other nodes and across the nodes is included in the context to build the prompt of in-context learning.

# Progress of Knowledge Graph in Llama Index

The Knowledge Graph abstraction was initially introduced to Llama Index by [William F.H](https://github.com/jerryjliu/llama_index/pull/433). where the triplets in the knowledge graph were associated with the docs with keywords and stored in memory, then [Logan Markewich](https://github.com/jerryjliu/llama_index/pull/487) enhanced it by adding embedding per triplets, too.

Recently, in the last couple of weeks, I had been [working with the community](https://github.com/jerryjliu/llama_index/issues/1318) on bringing the “GraphStore” storage context to Llama Index and thus introducing external storage of Knowledge Graph, the first implementation is [NebulaGraph the Open-Source Distributed Graph Database](https://www.nebula-graph.io/) that I had been working on since 2021.

During the implementation of this, the option to traverse multiple hops of the graph, and the option to collect more key entities on top-k nodes(to search in the knowledge graph to enable more global context) was introduced, and we are still refining the changes.

With GraphStore introduced, it also makes it possible to perform in-context learning from an existing knowledge graph, combined with other indices, this is also quite promising due to the knowledge graph being considered with high Information density than other structured data.

# About NebulaGraph Database

As a high-performance native distributed graph database, NebulaGraph Database excels in handling mega data sets with hundreds of billions of nodes and trillions of edges, while maintaining millisecond query latency. Relying on its shared-nothing, distributed architecture and computation-and-storage separation, it is highly available, highly secure, and easy to scale out, largely meeting the security requirements of enterprises. Besides, it can be deployed on the public cloud and on-premises as needed.

# Talk to a Graph Database Expert

Get a customized solution for your organization and see how NebulaGraph can help you:

[Contact NebulaGraph](https://www.nebula-graph.io/contact)

NebulaGraph database helps industry-level large language model Applications by bringing the “GraphStore” storage context to Llama Index.

Graph + LLM｜How NebulaGraph database helps industry-level large language model Applications

The growth of e-commerce has led to an overwhelming number of choices for consumers, making it increasingly challenging for online retailers to showcase relevant products and services in order to maintain customer loyalty and retain customers.

One of the most effective solutions to this problem is the use of recommendation engines. 
It has emerged as a key technology to bridge the gap, providing personalized suggestions that improve customer satisfaction and increase sales by analyzing the user’s previous behavior. [Graph databases](https://www.nebula-graph.io/posts/what-is-a-graph-database) are particularly well-suited for recommendation engines as they can handle large amounts of interconnected data with ease. In this article, we will explore how recommendation engines work in e-commerce, their different types, and the advantages of using graph databases like NebulaGraph for building powerful recommendation systems.

# What is a Recommendation Engine?

A recommendation engine is an algorithmic system that analyzes data on user behavior, preferences, and historical interactions with a product or service, to make personalized recommendations on content, products, or services that a user is likely to enjoy or find relevant.
Recommendation engines are commonly used by companies that offer personalized content or services, such as online retailers, music or video streaming platforms, and social media sites. By using data on a user's past behavior and preferences, a recommendation engine can make accurate predictions on what the user might be interested in, and present them with personalized recommendations.
Recommendation engines play a crucial role in the customer journey, helping businesses improve user experience, increase sales, and retain customers.

# Types of Recommendation Engines for Ecommerce

There are several types of recommendation engines, each with their own approach to providing personalized recommendations. The three most common types of recommendation engines are:

1. **Collaborative filtering**: Collaborative filtering is a technique that recommends items based on the behavior of similar users. This approach analyzes user behavior data such as purchases, ratings, and viewing history, to find similarities between users and recommend items that users with similar behavior patterns have enjoyed. Collaborative filtering can be further divided into two categories: user-based and item-based filtering.
2. **Content-based filtering**: Content-based filtering recommends items based on the user's past behavior and preferences. This approach uses data about the user's past behavior, such as their viewing history or search queries, to recommend items that are similar in content or characteristics to the user's past interactions. For example, if a user has watched a lot of action movies, a content-based recommendation engine might recommend other action movies.
3. **Hybrid filtering**: Hybrid filtering combines both collaborative and content-based filtering to make recommendations. This approach can provide more accurate recommendations by taking into account both the user's past behavior and preferences as well as the behavior of similar users.

Other types of recommendation engines include knowledge-based filtering, which recommends items based on a user's explicitly stated preferences, and context-aware filtering, which takes into account additional data such as the user's location or time of day to make recommendations.

And when it comes to recommendation engines used in e-commerce, common types are:

- **Personalized Recommendations**: These engines provide individualized suggestions based on a user's behavior, preferences, and history.
- **Popular Items**: These recommend the most popular items within a specific category, time frame, or location.
- **Similar Items**: These engines suggest items that are similar to a specific item, based on their features or user behavior.
- **Complementary Items**: Also known as cross-selling recommendations, these engines suggest items that complement a specific item, such as accessories or related products.

# Why Graph Databases Are Well-suited for Recommendation Engines

Graph databases excel in modeling complex relationships between data entities, making them an ideal choice for recommendation engines. Some key advantages of graph databases for recommendation engines include:

- Relationship-centric data modeling: Graph databases represent data as nodes and edges, making it easy to model relationships between users, products, and other relevant data points.
- High query performance: Graph databases provide fast and efficient query execution, enabling [real-time recommendations](https://www.nebula-graph.io/posts/use-cases-of-graph-databases-in-real-time-recommendation).
- Flexibility and schema evolution: Graph databases allow for schema changes without affecting existing data or queries, making it easy to incorporate new data sources and recommendation algorithms.
- Advanced analytics capabilities: Graph databases support advanced analytics, such as community detection and centrality measures, that can improve the accuracy and relevance of recommendations.

# NebulaGraph: A Recommended Solution for E-commerce Recommendation Engines

NebulaGraph is an [open-source, distributed graph database](https://www.nebula-graph.io/) that offers high performance, scalability, and ease of use. It is an excellent choice for building recommendation systems for e-commerce because of the following features:

- Efficient querying: NebulaGraph supports a flexible, SQL-like query language called nGQL, allowing for efficient querying and traversing of complex relationships between data entities.
- High scalability: NebulaGraph can handle large-scale graphs and store billions of nodes and edges with low latency, making it ideal for e-commerce platforms with massive amounts of user and item data.
- Data consistency: NebulaGraph ensures strong data consistency through the Raft consensus algorithm, which guarantees that updates are consistently applied across all replicas.
- Easy integration: NebulaGraph provides comprehensive APIs and connectors for popular programming languages, making it simple to integrate with e-commerce platforms and other data processing tools.

# Designing a Recommendation Engine with NebulaGraph Database

Building a recommendation engine with [NebulaGraph](https://nebula-graph.io) involves several steps, including:

1. Data modeling and schema design: Model users, products, and other relevant data points as nodes and their relationships as edges.
2. Importing and processing e-commerce data: Collect data on user behavior, preferences, and product information, and store it in NebulaGraph.
3. Implementing recommendation algorithms using NebulaGraph's query language (nGQL): Develop queries to identify patterns and relationships between users and items and generate personalized recommendations.
4. Testing and validating recommendations: Ensure that the generated recommendations are relevant and accurate by comparing them with actual user behavior and feedback.
5. Integrating the recommendation engine with the e-commerce platform: Use NebulaGraph's APIs and connectors to seamlessly integrate the recommendation engine with the existing e-commerce system, enabling real-time recommendations for users.
6. Continuously updating and refining the recommendation engine: Regularly update the data in NebulaGraph and fine-tune the recommendation algorithms to keep the recommendations fresh and relevant.

# Conclusion

Recommendation engines are essential for e-commerce platforms to provide personalized experiences and improve customer satisfaction. Graph databases, with their ability to model complex relationships and offer high query performance, are well-suited for building such engines. NebulaGraph, an open-source, distributed graph database, is a recommended solution for e-commerce recommendation engines due to its scalability, efficiency, and ease of use. By implementing a recommendation engine with NebulaGraph, e-commerce businesses can significantly enhance their user experience, increase sales, and retain customers in the competitive online marketplace.

# How NebulaGraph Works
1. [NebulaGraph Architecture — A Bird’s Eye View](https://www.nebula-graph.io/posts/nebula-graph-architecture-overview)
2. [Benchmark: NebulaGraph vs Dgraph vs JanusGraph](https://www.nebula-graph.io/posts/benchmarking-mainstraim-graph-databases-dgraph-nebula-graph-janusgraph)
3. [Comparison: Neo4j vs NebulaGraph vs JanusGraph](https://www.nebula-graph.io/posts/performance-comparison-neo4j-janusgraph-nebula-graph)
4. [Deploy the Graph Database on Kubernetes](https://www.nebula-graph.io/posts/how-to-deploy-nebula-graph-in-kubernetes)
5. [Graph Database For Recommendation Systems](https://www.nebula-graph.io/posts/use-cases-of-graph-databases-in-real-time-recommendation)


Recommendation engines are essential for e-commerce platforms to provide personalized experiences and improve customer satisfaction. Graph databases, with their ability to model complex relationships and offer high query performance, are well-suited for building such engines. 

How Recommendation Engine for Ecommerce Works Using Graph Database

Graph databases have been the fastest growing database category since 2013.

Graph Database Market Overview

A graph database is a specialized type of NoSQL database that stores the connections between entities (nodes) and the relations associated with them (edges).

What is a Graph Database and What are the Benefits of Graph Databases

In the modern business environment where data inevitably rules, traditional relational databases are no longer enough to keep up with the increasing complexity and interconnectivity of information. Enter graph databases, the revolutionary technology that has taken the tech industry by storm - for a good reason. 

With the ability to store and manage vast amounts of complex and interconnected data, graph databases are rapidly becoming the go-to solution for businesses and organizations of all sizes. 

According to a [report](https://www.databridgemarketresearch.com/reports/global-graph-database-market) by Data Bridge Market Research, the global market for graph databases was around USD 1938.20 million in 2022. The same report projects that the industry will grow at a compounded annual rate of 18.20% and reach USD 7384.79 million by 2030.

But which is the best graph database for enterprise? There are a few good players, and this article is dedicated to looking at Neo4j vs TigerGraph vs Dgraph vs NebulaGraph. Whether your intention is to use [open source graph database](https://nebula-graph.io/) options or commercial ones, you'll find this useful. 

Meanwhile if you are just beginning out with graph databases, we recommend that you also make sure to go through this valuable [guide on how to choose a graph database](https://www.nebula-graph.io/posts/how-to-select-a-graph-database). 


# Benefits of graph databases for enterprises

These benefits represent the outstanding role that graph databases play in helping enterprises to get the most out of their big data.


## **Data Integrity**

Graph databases ensure data integrity by allowing developers to define constraints that enforce data relationships and eliminate anomalies or inconsistencies. This makes them ideal for use cases where data quality and accuracy are critical, such as financial transactions or healthcare records. 

For example, a financial institution could use a graph database to ensure the accuracy and completeness of customer data across multiple accounts and transactions.


## **Semantic Reasoning**

Semantic reasoning is a process by which a computer program or system is able to derive new facts or insights from existing data by analyzing the relationships between entities and inferring new information based on those relationships. This is done by applying logical rules and algorithms that take into account the meaning and context of the data being analyzed, as opposed to just processing the data as raw information.

In the context of graph databases, semantic reasoning can be used to gain deeper insights into the relationships between entities in the graph, even when those relationships are not explicitly stated. By analyzing the semantics of the data and applying logical rules, a graph database can infer new relationships and properties between entities, leading to a richer and more comprehensive understanding of the data. This can be particularly useful in fields such as life sciences, where researchers may be trying to identify new connections between genes and diseases, or in finance, where analysts may be trying to detect patterns in market data that are not immediately apparent.


## **Data Integration**

Graph databases can be used to integrate data from different sources, enabling users to analyze data from disparate sources in a single location. This enables enterprises to gain a more complete picture of their data, and also helps to identify relationships and patterns that might not be apparent when analyzing individual data sources in isolation. 

For example, an e-commerce company can use a graph database to integrate customer data from multiple sources, such as social media profiles, web browsing history, and purchase history, in order to better understand customer behavior and preferences.


## **Collaboration**

Collaboration is a crucial element for success in many enterprises, but it can often be challenging due to siloed data and disparate systems. Graph databases offer a solution to this problem by providing a unified and comprehensive view of data that can be accessed by multiple teams and departments within an organization.

For example, a pharmaceutical company may have separate teams working on drug development, clinical trials, and marketing. Each team may have its own systems and data sources, making it difficult to share information and collaborate effectively. However, by using a graph database to store all the data in a unified structure, each team can easily access and analyze the same information, regardless of where it originated. This can help to break down silos and improve collaboration across the organization, leading to more efficient drug development and faster time-to-market.

Furthermore, graph databases can enable real-time collaboration by providing a single source of truth that is always up-to-date. Changes made by one team or department are immediately visible to others, allowing for faster decision-making and more agile processes.


## **Regulatory Compliance**

One of the main advantages of graph databases in regulatory compliance is their ability to maintain a complete and auditable view of data. The ability to store information about data lineage and changes means that graph databases can track data as it moves through an enterprise's systems, allowing enterprises to demonstrate compliance with regulations such as GDPR and HIPAA.

For example, consider a healthcare provider that uses a graph database to manage patient records. By simply tracking changes to patient records over time, the provider can demonstrate compliance with regulations that require the maintenance of an audit trail. In the event of an audit or investigation, the provider can easily provide a complete record of all changes.

Another way that graph databases can help enterprises meet regulatory compliance requirements is by providing a granular level of access control. This is achieved through defining roles and permissions for each user, ensuring that only authorized personnel have access to sensitive data. This can help enterprises comply with regulations that require the protection of personal data.


# Overview of Neo4j, TigerGraph, Dgraph, NebulaGraph

Here is a brief view of each of the graph databases we are focussing on:


## **Neo4j**

Neo4j has become one of the most popular graph databases on the market. 

According to the parent  company Neo4j, Inc, the solution is currently being used by over 900 enterprises and enjoys a strong community of over 200,000 members. Over 30 million downloads and 150,000 instances are executed every year.


## **TigerGraph**

TigerGraph was first released in 2012 by TigerGraph Inc, though in stealth mode. It came out of stealth in 2017.

Information appearing on [Techtarget](https://www.techtarget.com/searchbusinessanalytics/news/252492072/TigerGraph-taking-advantage-of-graph-database-growth) indicates that TigerGraph may have had about 4000 users in TigerGraph Cloud by 2021. However, it's not clear if these are pure enterprise customers or a mix of enterprise and individuals.


## **Dgraph**

Written purely in Go, Dgraph came to the market in 2017 but was first introduced in 2015 by Dgraph Labs, Inc. According to Dgraph, their community is now at 21, 000 members.

Dgraph Enterprise, which is our focus, is actually built atop the open source offering.


## **NebulaGraph**

NebulaGraph is also an open source graph database whose first version, Nebula Graph v0.1.0-alpha, was first released in 2019. Check out the entire [timeline of NebulaGraph](https://docs.nebula-graph.io/3.0.2/20.appendix/history/).  

The [NebulaGraph Enterprise](https://www.nebula-graph.io/enterprise) version gives companies access to enhanced visual tools, Analytics, enterprise-level security and dedicated support. Big companies including Tencent, Oppo, Vivo, and WeBank are already using NebulaGraph Enterprise. Others like [360 DigiTech have migrated to NebulaGraph](https://www.nebula-graph.io/posts/why-migrate-janusgraph-nebula-graph) from their previous graph solutions.  

NebulaGraph has now passed 200,000 pull requests and is already at over 8000 Github stars (in 2023). This is remarkable considering NebulaGraph is in fact the youngest of all the rest here. 


#  Comparison across key graph database features and capabilities

The above graph databases can offer numerous benefits to your enterprise. But how do you choose between them? 

Below is a detailed comparison of the features and capabilities of these databases, which will help you make an informed decision when choosing the right one for your enterprise.

We would also like to clarify that the details here are mostly based on publicly available information and also from the graph databases own assertions, and not on practical tests that we have done ourselves. 


## **Performance**



* **Neo4j:** Offers several high performance features, such as optimized indexing and query planning. Also,[ according to InfoWorld,](https://www.infoworld.com/article/3679269/neo4j-50-improves-on-scalability-performance-and-agility.html) the latest database version includes autonomous clustering (for connecting related nodes).
* **TigerGraph:** According to TigerGraph's performance benchmark report, the database is up to 377 times faster than other graph databases in two-hop path queries.
* **Dgraph:** Dgraph offers several performance features, such as optimized read and write speeds and concurrent caching.  
* **NebulaGraph:** NebulaGraph is designed to be automatically highly performant for large-scale graph data.Here is the detailed [NebulaGraph performance report](https://www.nebula-graph.io/posts/nebula-graph-v2.6.0-performance-report).  The Tencent Cloud Security team tested NebulaGraph and Neo4j alongside HugeGraph. They found that while Neo4j outperformed NebulaGraph slightly with smaller data sets, NebulaGraph proved significantly faster than the others with larger data sets. Here are the [comprehensive results](https://www.nebula-graph.io/posts/performance-comparison-neo4j-janusgraph-nebula-graph) of the test.


## **Data modeling**



* **Neo4j:** Neo4j is a property graph database and thus represents data in nodes, edges, and properties. The relationships are directional – either unidirectional or bidirectional.
* **TigerGraph:** Like Neo4j, TigerGraph is a directional property graph representing data in nodes, edges, and properties. The edges are equally directed.
* **Dgraph:** Dgraph is also a property graph database, which models data into nodes and edges. The edges are also directional and contain properties that define the relationship of the data sets. It supports reverse traversals, allowing you to follow the data flow from the end to the start nodes.
* **NebulaGraph:** Users can create and modify their data models on-the-fly, without the need for any pre-defined schemas or constraints.This allows users to model their data in a way that best fits their specific use case, without being constrained by pre-existing schemas. It also supports property graphs. 


## **Query language**



* **Neo4j:** Neo4j uses its own query language, Cypher, which features simple and human-readable syntax and commands.
* **TigerGraph:** Uses the GSQL language, which has highly-expressive syntax, leading to improved completeness of the retrieved data. The language also supports parallelism, which improves query speeds.This language has some similarities to SQL.
* **Dgraph:** Uses the GraphQL language, which is highly expressive yet direct. GraphQL will enable you to define the shape of your data, leading to improved modeling.
* **NebulaGraph:** Provides a powerful query language known as [nQL](https://docs.nebula-graph.io/3.0.0/3.ngql-guide/1.nGQL-overview/1.overview/), which is similar in syntax to SQL and is designed for both developers as well as operations teams.


## **Ecosystem**



* **Neo4j:** Neo4j is equipped with a wide collection of libraries, tools, and drivers, enabling  seamless integration. For example, the connector tools make it easy to integrate Neo4j to popular data tools including data warehouses such as Google BigQuery.  
* **TigerGraph:** The TigerGraph ecosystem is powered by RESTful APIs and JSON output. Also has several integration connectors, allowing you to import your data from multiple sources.
* **Dgraph:** Uses proprietary software tools to offer data integration. According to a 2021 news report on DevOps, Dgraph Labs (the parent company) contracted software firm Capventis to help expand its data integration tools. One of the solutions acquired was Glu,  a software that allows the database to unite data from multiple sources during analysis.
* **NebulaGraph:** [NebulaGraph’s ecosystem](https://docs.nebula-graph.io/3.2.0/20.appendix/6.eco-tool-version/) allows users to seamlessly integrate NebulaGraph with other systems. For example, it integrates with Apache Spark for big data processing. This allows users to perform graph computations on NebulaGraph data using Spark's powerful parallel processing capabilities. Check out this comprehensive guide on [using NebulaGraph in Apache Spark](https://www.nebula-graph.io/posts/work-with-nebula-graph-apache-spark). 


## **Analytics**



* **Neo4j:** Neo4j has optimized graph algorithms, which help reveal hidden patterns and improve data visualization. The database also supports AI and machine learning integration, which allows for more optimal analytics.
* **TigerGraph:** Uses algorithms with parallel computation, allowing faster and improved data retrievals. It also supports AI and Machine Learning for cloud data.
* **Dgraph:** Also offers excellent analytic features, making it popular amongst businesses and enterprises that process big data. GraphQL is a potent analytical tool that offers fast and optimal data visualization with numerous analytical information.
* **NebulaGraph:** Offers an enterprise analytics platform known as [NebulaGraph Analytics](https://docs.nebula-graph.io/3.4.1/graph-computing/nebula-analytics/) which is purely dedicated to graph analytics. 


## **Scalability**



* **Neo4j:** Neo4j is famous for, among other things, offering impressive scalability capabilities. According to a press release published on PRNewswire in 2021, the database can support up to 200 billion nodes and over one trillion relationships.
* **TigerGraph:** TigerGraph is also becoming increasingly popular due to its scalability features. According to a news report on Silicon Angle, TigerGraph can support up to 70 billion nodes and 500 billion edges (relationships).
* **Dgraph:** Supports horizontal scaling. According to a news report published on GlobalNewsWire, the database was used by a company in China to store 48 billion triple datasets.
* **NebulaGraph:** Also offers horizontal scalability. You can  expand the cluster by adding more nodes or services without impacting performance. Moreover, scaling out NebulaGraph is a breeze as it doesn't require reconfiguring existing nodes - all you need is just sufficient bandwidth. 


## **Security**



* **Neo4j:** Offers a comprehensive access-control security mode, which makes it possible to restrict access to individual nodes and properties. The database also has control permission features for data reading, writing, and traversing. Additionally, it allows for integrating third-party data security measures such as data encryption, BOLT and HTTPS support, and SSL certificates.
* **TigerGraph:** Equally offers several security capabilities, including network-access control, authentication, and encryption. According to a news report on Datanami, TigerGraph has of late made several updates, including adding new security features to its cloud platform. The latest updates are expected to bolster the database's stability, performance, and security.
* **Dgraph:** Offers diverse security features, such as network access control and data encryption. The database also allows you to create access control to the GraphQL endpoint applications, thus limiting the people who can access and query data. 
* **NebulaGraph:** Supports role-based [access control](https://db-engines.com/en/system/NebulaGraph%3BNeo4j%3BTigerGraph) to ensure the confidentiality, integrity, and availability of user data. The access control mechanism is [implemented](https://docs.nebula-graph.io/3.2.0/7.data-security/1.authentication/1.authentication/) through local authentication or LDAP authentication. 


# Conclusion

The graph database you choose can enormously impact the kind of value your enterprise gets from data. This is why the selection process demands careful reflection to avoid the common pitfalls that businesses often encounter. 

One of the most prevalent missteps is choosing a database solution that has limited scalability, thereby restricting the ability to quickly adapt to future data management needs. Another mistake is settling for a database solution that underperforms or has inadequate security features, jeopardizing the integrity and confidentiality of the data.

With the above comparison and the resources provided, you should be able to make an informed decision and choose a graph database that is best suited for your enterprise's needs. 


The graph database you choose can enormously impact the kind of value your enterprise gets from data. With the comparison and the resources provided, you should be able to make an informed decision and choose a graph database that is best suited for your enterprise’s needs.

Best Graph Database for Enterprise: Neo4j vs TigerGraph vs Dgraph vs NebulaGraph Comparison



>This ariticle was written by Ajeet Singh Raina and Wey Gu, originally published on [Docker blog](https://www.docker.com/blog/distributed-cloud-native-graph-database-nebulagraph-docker-extension/).

Graph databases have become a popular solution for storing and querying complex relationships between data. As the amount of graph data grows and the need for high concurrency increases, a distributed graph database is essential to handle the scale.

Finding a distributed graph database that automatically shards the data, while allowing businesses to scale from small to trillion-edge-level without changing the underlying storage, architecture of the service, or application code, however, can be a challenge. 

In this article, we’ll look at NebulaGraph, a modern, open source database to help organizations meet these challenges.


# Meet NebulaGraph
[NebulaGraph](https://nebula-graph.io) is a modern, open source, cloud-native graph database, designed to address the limitations of traditional graph databases, such as poor scalability, high latency, and low throughput. NebulaGraph is also highly scalable and flexible, with the ability to handle large-scale graph data ranging from small to trillion-edge-level.

NebulaGraph has built a thriving community of more than [1000 enterprise users](https://nebula-graph.io/cases) since 2018, along with a rich ecosystem of tools and support. These benefits make it a cost-effective solution for organizations looking to build graph-based applications, as well as a great learning resource for developers and data scientists.

The NebulaGraph cloud-native database also offers [Kubernetes Operators](https://github.com/vesoft-inc/nebula-operator) for easy deployment and management in cloud environments. This feature makes it a great choice for organizations looking to take advantage of the scalability and flexibility of cloud infrastructure.

# Architecture of the NebulaGraph database
NebulaGraph consists of three services: the Graph Service, the Storage Service, and the Meta Service (Figure 1). The Graph Service, which consists of stateless processes (nebula-graphd), is responsible for graph queries. The Storage Service (nebula-storaged) is a distributed (Raft) storage layer that persistently stores the graph data. The Meta Service is responsible for managing user accounts, schema information, and Job management. With this design, NebulaGraph offers great scalability, high availability, cost-effectiveness, and extensibility.

![Nebulagraph database architecture infographic](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/docker-demo/nebulagraph-database-architecture-infographic.png)
Figure 1: Overview of NebulaGraph services.

# Why NebulaGraph?
NebulaGraph is ideal for graph database needs because of its architecture and design, which allow for high performance, scalability, and cost-effectiveness. The architecture follows a separation of storage and computing architecture, which provides the following benefits:

- **Automatic sharding**: NebulaGraph automatically shards graph data, allowing businesses to scale from small to trillion-edge-level data volumes without having to change the underlying storage, architecture, or application code.
- **High performance**: With its optimized architecture and design, NebulaGraph provides high performance for complex graph queries and traversal operations.
- **High availability**: If part of the Graph Service fails, the data stored by the Storage Service remains intact.
- **Flexibility**: NebulaGraph supports property graphs and provides a powerful query language, called Nebula Graph Query Language (nGQL), which supports complex graph queries and traversal operations. 
- **Support for APIs**: It provides a range of APIs and connectors that allow it to integrate with other tools and services in a distributed system.

# Why run NebulaGraph as a Docker Extension?
In production environments, NebulaGraph can be deployed on Kubernetes or in the cloud, hiding the complexity of cluster management and maintenance from the user. However, for development, testing, and learning purposes, setting up a NebulaGraph cluster on a desktop or local environment can still be a challenging and costly process, especially for users who are not familiar with containers or command-line tools.

This is where the NebulaGraph Docker Extension comes in. It provides an elegant and easy-to-use solution for setting up a fully functional NebulaGraph cluster in just a few clicks, making it the perfect choice for developers, data scientists, and anyone looking to learn and experiment with NebulaGraph.

## Getting started with NebulaGraph in Docker Desktop
### Setting up
Prerequisites: [Docker Desktop 4.10](https://docs.docker.com/desktop/) or later.

### Step 1: Enable Docker Extensions
You’ll need to enable Docker Extensions under the Settings tab in Docker Desktop. Within Docker Desktop, confirm that the **Docker Extensions** feature is enabled (Figure 2). Go to **Settings > Extensions** and select **Enable Docker Extensions**.

![Docker desktop extensions settings](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/docker-demo/docker-desktop-extensions-settings-980x557.png)
Figure 2: Enabling Docker Extensions within the 
Docker Desktop.

All Docker Extension resources are hidden by default, so, to ensure its visibility, go to **Settings > Extensions** and check the **Show Docker Extensions system containers**.

### Step 2: Install the NebulaGraph Docker Extension
The NebulaGraph extension is available from the Extensions Marketplace in Docker Desktop and on [Docker Hub](https://hub.docker.com/extensions/weygu/nebulagraph-dd-ext). To get started, search for NebulaGraph in the Extensions Marketplace, then select **Install** (Figure 3).

![Nebulagraph extensions marketplace](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/docker-demo/nebulagraph-extensions-marketplace-980x624.png)
Figure 3: Installing NebulaGraph from the Extensions Marketplace.

This step will download and install the latest version of the NebulaGraph Docker Extension from Docker Hub. You can see the installation process by clicking Details (Figure 4).


![Nebulagraph installation progress docker desktop](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/docker-demo/nebulagraph-installation-progress-docker-desktop-980x624.png)
Figure 4: Installation progress.

### Step 3: Waiting for the cluster to be up and running
After the extension is installed, for the first run, it normally takes fewer than 5 minutes for the cluster to be fully functional. While waiting, we can quickly go through the **Home** tab and **Get Started** tab to see details of NebulaGraph and NebulaGraph Studio, the WebGUI Utils.

We can also confirm whether it’s ready by observing the containers’ status from the **Resources** tab of the Extension as shown in Figure 5.

![Nebulagraph resources docker desktop](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/docker-demo/nebulagraph-resources-docker-desktop-980x624.png)
Figure 5: Checking the status of containers.

### Step 4: Get started with NebulaGraph
After the cluster is healthy, we can follow the Get Started steps to log in to the NebulaGraph Studio, then load the initial dataset, and query the graph (Figure 6).

![Nebulagraph get started docker desktop](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/docker-demo/nebulagraph-get-started-docker-desktop-980x624.png)
Figure 6: Logging in to NebulaGraph Studio.

## Step 5: Learn more from the starter datasets 
In a graph database, the focus is on the relationships between the data. With the starter datasets available in NebulaGraph Studio, you can get a better understanding of these relationships. All you need to do is click the Download button on each dataset card on the welcome page (Figure 7).

![starter datasets](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/docker-demo/nebulagraph-starter-datasets-980x937.png)
Figure 7: Starter datasets.

For example, in the ```demo_sns``` (social network) dataset, you can use the following query to find new friend recommendations by identifying second-degree friends with the most mutual friends:

```
1 Einstein
```

![Nebula console demo sns](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/docker-demo/nebula-console-demo-sns-980x782.png)
Figure 8: Query results shown in the Nebula console.

Instead of just displaying the query results, you can also return the entire pattern and easily gain insights. For example, in Figure 9, we can see LeBron James is on two mutual friend paths with Tim:

![Nebula console demo sns results graph](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/docker-demo/nebula-console-demo-sns-results-graph-980x853.png)
Figure 9: Graphing the query results.


Another example can be found in the ```demo_fraud_detection``` (graph of loan) dataset, where you can perform a 10-degree check for risky applicants, as shown in the following query:
```Cypher
MATCH 
p_=(p:`applicant`)-[*1..10]-(p2:`applicant`)
WHERE id(p)=="p_200" AND 
p2.`applicant`.is_risky == "True"
RETURN p_ LIMIT 100
```

The results shown in Figure 10 indicate that this applicant is suspected to be risky because of their connection to ```p_190```.

![Nebula console demo fraud detection graph](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/docker-demo/nebula-console-demo-fraud-detection-graph-980x783.png)
Figure 10: Results of query showing fraud detection risk.

By exploring the relationships between data points, we can gain deeper insights into our data and make more informed decisions. Whether you are interested in finding new friends, detecting fraudulent activity, or any other use case, the starter datasets provide a valuable starting point.

We encourage you to download the datasets, experiment with different queries, and see what new insights you can uncover, then share with us in the NebulaGraph community.

# Try NebulaGraph for yourself

To learn more about NebulaGraph, visit our [website](https://nebula-graph.io), [documentation site](https://docs.nebula-graph.io/), star our [GitHub repo](https://github.com/vesoft-inc/nebula), or join our [community chat](https://community-chat.nebula-graph.io/).

[![docker](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/docker-banner.png)](javascript:window.open('https://bit.ly/3XHS873'))



NebulaGraph Docker Extension provides an elegant and easy-to-use solution for setting up a fully functional NebulaGraph cluster in just a few clicks, making it the perfect choice for developers, data scientists, and anyone looking to learn and experiment with NebulaGraph.

Distributed Cloud-Native Graph Database with NebulaGraph Docker Extension

Organizations worldwide are facing a huge problem due to the exponential growth of data. According to the [Institute of Electrical and Electronics Engineers](https://cmte.ieee.org/futuredirections/2021/01/29/latest-data-on-data/), about 168 zettabytes of data will have been created by 2025. This is massive, and it’s not yet peak level. What levels of data are we looking at in a decade and beyond? 

These are mind boggling figures. But there is one more trend that is also worrying: Traditional databases are not very good at handling the overwhelming data volume .  A [study](https://www.researchgate.net/publication/360155336_Handling_Big_Data_in_Relational_Database_Management_Systems) published on ResearchGate reveals just how limited relational databases are, particularly when dealing with big data.

Because of this growing frustration with traditional databases in the midst of enormous data, organizations are turning to modern database solutions. And with this need, graph databases have now come of age. 

The unique thing, which also happens to be the greatest advantage of graph databases, is that they are designed to store and manage large sets of interconnected data points, such as social networks, supply chains, and financial transactions. As we have previously indicated when [breaking down the concept of a graph database](https://www.nebula-graph.io/posts/what-is-a-graph-database), these databases are particularly useful for applications that require real-time processing and analysis of data, such as [fraud detection](https://www.nebula-graph.io/posts/fraud-detection-with-graph-machine-learning)**,** [recommendation engines](https://www.nebula-graph.io/posts/use-cases-of-graph-databases-in-real-time-recommendation), and [social network analysis](https://www.nebula-graph.io/posts/social-networks-with-graph-database-1) tools.

Now, like any other database out there, graph databases too need a way (language) to communicate with their clients (users). For graph databases, these languages are known as graph query languages. Whether you are working with an [open source graph database](https://nebula-graph.io/) or a commercial one, you’ll need a super good graph query language by your side. 

Let’s break it all down. 


# **What is a graph query language?**

So what exactly is a graph query language?

A graph query language is a programming language designed to work with graph databases. It allows users to specify the structure of the graph they want to query, as well as the rules for traversing the graph and retrieving the relevant data. 

Graph query languages are typically declarative, meaning that users specify what they want to retrieve, rather than how to retrieve it. In other words the language instructs the database on what to do, which is what unlocks the numerous benefits that they offer.

Furthermore, graph query languages can recognize and relate the current data to other data sets in real-time.


# **How does a graph query language work?**

Graph query languages work by allowing users to specify patterns in the graph that they want to query. Users can specify the nodes and edges that they are interested in, as well as any conditions that must be met for the query to return results. The language then processes the query and returns the results in a format that can be easily consumed by the user.

For instance, let's say a financial institution wants to detect fraudulent activity related to credit card transactions. The institution can store all transactions in a graph database, where each transaction is a node and edges connect transactions to their associated accounts, customers, and merchants.

Using a graph query language, a developer can write queries to identify patterns that may indicate fraud, such as transactions that involve multiple cards, transactions from new or suspicious merchants, and transactions that occur outside of a customer's usual geographic region.

The output of this query could be further analyzed to identify potentially fraudulent transactions involving multiple cards and merchants, which could then be flagged for further investigation.

For example, a simple query might look for all nodes in the graph that are labeled "person" and return their names and ages. The query might also specify that the results should only include nodes that are connected to other nodes labeled "organization." The query language would then process the query and return a list of all the matching nodes and their associated information.


# **Popular graph query languages**

You’ll notice that most of the graph query languages are developed by players in the graph database market, which is only natural. A graph database provider would be best placed to develop a query language that works best for their solution. 

Below is a look at some of the top ones that your business can consider:


## **nGQL**

[nGQL](https://docs.nebula-graph.io/3.4.0/3.ngql-guide/1.nGQL-overview/1.overview/) is NebulaGraph’s query language. It is a declarative language that allows users to query the graph database using various patterns and criteria. It’s quite friendly for both developers and professionals on the operations side of things. 

nGQL provides a flexible and expressive syntax for querying graph data. It supports a wide range of operations, such as pattern match, traversal,  graph mutation, aggregation, access control, filtering, composite queries , and index.  

The beauty about this query language is that it’s constantly being improved to accommodate new features and support more environments. It’s compatible with openCypher 9, but not completely - does not support mutations and controls syntax.


## **Gremlin**

Gremlin is run by Apache TinkerPop. It allows users to specify complex graph traversals using a simple and intuitive syntax, enabling powerful queries that would be difficult or impossible to express using the traditional SQL. 

One of the key strengths of Gremlin is its ability to traverse graphs of any size, complexity, and shape, making it well-suited for analyzing highly connected data. Gremlin is also highly extensible, with a large and active community of developers contributing to its development and supporting a wide range of graph databases and tools.


## **SPARQL**

SPARQL is designed specifically for querying RDF (Resource Description Framework) data. As you may already know,  RDF is the model used to represent and link data on the web, and SPARQL allows users to query this data in a standardized and expressive way. It provides a flexible and powerful way to extract information from RDF datasets, allowing users to retrieve data based on complex patterns and relationships.

SPARQL was developed by the RDF Data Access Working Group of the World Wide Web Consortium (W3C), a global community that develops web standards and guidelines. The language was first introduced in 2008 and has since been updated to include additional features and functionality. 

The W3C continues to manage and maintain the SPARQL standard, and the language is widely used in academic, research, and industry applications for querying and analyzing RDF data. Several open-source and commercial SPARQL query engines are available, providing users with a range of options for working with RDF data.


## **Cypher**

Cypher was developed by Neo4j and is maintained as an open-source project under the openCypher initiative. openCypher is a community-driven project that aims to promote the adoption and standardization of the Cypher query language across different graph database platforms. 

The openCypher project is managed by a team of developers and contributors from various organizations, including Neo4j. 


## **AQL**

AQL (ArangoDB Query Language) is used for querying and manipulating data in the ArangoDB multi-model database. It is also declarative in nature, in which you express requests based on the data you want – as opposed to how to retrieve it. 

The language is designed to be simple and easy to learn, with a SQL-like syntax that makes it accessible to a wide range of users. AQL supports a variety of data manipulation operations, including filtering, sorting, grouping, and aggregating data, as well as complex graph traversals and joins between different collections and data models.


## **GraphQL**

GraphQL is specifically tailored for API-based data. With GraphQL, you can send a query to your API and retrieve only the data you need, making data retrieval simpler and more efficient.

One of the key benefits of using GraphQL is that it allows for highly predictable queries, which makes for faster and more reliable apps. 

Rather than relying on the server to control the data, GraphQL puts the control in your hands, allowing you to optimize data retrieval for your specific needs.


# **What makes graph query languages crucial for enterprises?**

How can the above graph query languages, and all the others we have not mentioned,  help your business? Of course the developers within the company have to select the right query language for the job, then implement it appropriately depending on the intended outcomes. 

Below is a look at some of the top benefits of graph query languages for businesses:


## **1. Simplified data queries**

Graph query languages allow you to retrieve data much more quickly and correlate it to other data sets in the database. 

The data is stored as nodes (data entities) and edges (data relationships) instead of tables, rows, and columns. The nodes and edges are easy to correlate, which helps simplify data queries, even when dealing with large data sets.


## **2. Better Data integrity**

Graph database query languages help guarantee data accuracy, completeness, and consistency. They allow you to retrieve data from multiple disparate sources and represent it in an easy-to-follow visual pattern. 

This means you are able to see how your data is connected and how changes to one set can affect other data sets.


## **3. Enhanced engagement efficiency**

With a graph query language, you can see how your requested data relates to other data in the database - eliminating the need for manual data exploration.

Additionally, these languages allow for precise data querying without over-fetching, which is a common issue with relational databases. According to an MDPI [study](https://www.mdpi.com/2076-3417/12/13/6490), graph databases outperformed relational databases when dealing with complex data queries.


## **4. Better data modeling**

Graph databases represent data much more naturally in easy-to-follow graphs when compared to a relational database. When using a graph query language, you can be able to model your data much better and identify relationships and patterns. It’s so easy to query and analyze data in a way that is intuitive and closer to the way humans think. For example, it's easier to model social networks, supply chain relationships, and other complex systems.

Graph query languages also make it easy to query data that is constantly changing, which is difficult to do with traditional query languages. As new nodes and edges are added or deleted, a graph database query language efficiently query the data and gibe returns that reflect these changes.

You can even automate the data queries and analysis process, leading to faster predictions.


## **5. Complex pathfinding**

Pathfinding queries, such as finding the shortest path between two nodes or all the paths between two nodes that meet certain criteria, are essential in applications such as transportation planning, logistics, and network optimization. 

Graph query languages make it possible to perform these complex queries pathfinding with ease.


# **Choosing the right graph database query language**

How do you choose between the many graph query languages we have covered here? Consider the factors below:


## **1. The type of graph database and data you are dealing with**

As we went about describing the various query languages, you may have noticed that some of them are tailored for specific functions or environments. 

Therefore, your choice of language may be heavily influenced by the choice of graph database you are using. Of course the choice of database will be influenced by what you want to achieve, which could be operational or user/customer centric. 

Consider these items:


## **2. Performance**

A key reason why graph query languages and databases have gained popularity is due to their ability to enhance data querying efficiency. 

As a result, it's crucial that you select a language that provides speedy returns on queries, particularly when dealing with large data sets requiring real-time data analysis.


## **3. Ease of use**

Maybe this is quite obvious but it's important that we mention it. A good graph query language should be easy to use. 

For example, declarative languages allow you to request data without indicating how the data will be retrieved – the language's engine does this for you. Such a feature makes it easy to use the language without writing complex algorithms.


## **4. Check out the security and privacy**

We are right in the middle of data security and privacy concerns - all over the world.  Thus, you need to be aware of any potential security risks that come with a specific language. 

For example, according to [DevOps](https://devops.com/does-graphql-introduce-new-security-risks/), GraphQL comes with potential security concerns, which you need to be aware of and prepare appropriate mitigation strategies. 


# **Concluding perspective**

A study published on [Science Direct](https://www.sciencedirect.com/science/article/pii/S2405844020311373) shows how graph queries were used to uncover hidden corruption of various individuals named in the Panama Papers. If you can take a moment to imagine the huge role that graph query language played in this groundbreaking investigation, then you immediately appreciate the power of these languages. 

Going forward, developers working with graph query languages should keep in mind the importance of data quality and consistency. As graphs become more complex, ensuring data accuracy and completeness will be critical to producing reliable outcomes.

One area where graph query languages may see significant growth is in the integration with AI and machine learning techniques, allowing for more sophisticated querying and analysis of graph data.

A graph query language is a programming language designed to work with graph databases. It allows users to specify the structure of the graph they want to query, as well as the rules for traversing the graph and retrieving the relevant data.

Graph Query Language: What You Should Know

As organizations collect and store more and more data, the need to understand and analyze complex relationships within that data has grown tremendously. Graph databases, which are designed specifically to store and query interconnected and complex data, are uniquely suited to this task. The [benefits of graph databases](https://www.nebula-graph.io/posts/why-use-graph-databases) are immense and this explains why they have become a popular choice for many businesses. For example, a [graph database](https://www.nebula-graph.io/posts/what-is-a-graph-database) offers fast performance on complex queries, making them useful for use cases such as fraud detection, [recommendation engines](https://www.nebula-graph.io/posts/use-cases-of-graph-databases-in-real-time-recommendation), and social network analysis.

One key aspect that users are increasingly seeking when working with complex data is [visualization](https://www.nebula-graph.io/posts/graph-database-visualization), as it makes it easier to quickly interpret what their data is telling them. 

Let’s explore graph database visualization in particular,  the benefits and use cases.  


# **What is graph database visualization?**

Graph database visualization is the representation of the data stored in a graph database in a visual format, for example a chart or diagram. This makes it easier to understand and analyze the data, because it's easier to see patterns and relationships when they're represented visually. 

For example, a social network company can use a bar chart to show how many connections each user has, or a line graph to show how the number of connections has changed over time. We might also use a scatter plot to show how different users are connected to each other, or a network diagram to show how different groups of users are connected. 

![graph-database-on-social-network](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/graph-database-on-social-network-02.jpeg)

When you visualize your data in a graph database, it makes your work much easier on many fronts. The structure becomes user-friendly and you gain a significant boost in efficiency when working with the data. We'll tackle the advantages shortly.


# **How graph database visualization works**

To best understand how graph database visualization works, it’s important to briefly go over how data is stored and used in a graph database. We are aware that a graph is basically a collection of nodes and edges, where each node represents an individual data point like a person or transaction, and each edge indicates how two nodes are related; for instance, a person makes a transaction. This data in the graph is stored in a graph database, such as NebulaGraph. 

Now, whenever a user wants to retrieve some data, they perform a query. By default, the results of the query normally come in non-visual format such as CSV, JSON, XML, Text Files, spreadsheet, etc. But with visualization, the results will come in a visual format specified by the user - as long as it’s supported by the Graph Database that is being used. Some of the formats can include network diagrams, interactive visualization, heat map, scatter plot, etc.

With this level of visualization, it becomes easy even for novices and non-technical users to understand complex data and make meaningful deductions from it. It means, for example, that a team of data experts can comfortably collaborate on a project with other team members who are not data experts themselves. A business development manager with little data knowledge can make use of such visualization to draw incredible insights that can be used to make valuable decisions. 

A graph visualization tool allows you to specify the layout of the display, the color and size of the nodes and edges, and other formatting options. These tools employ algorithms to work out node positions and showcase the data in either two or three dimensional formats. When you interact with the visualized graph, you can zoom in and out, pan around, and click on individual nodes and edges to see more information about them. It’s that simple. 


# **Advantages of graph database visualization**

If we had to give a concise description of the many benefits of graph database visualization, it would be "enlightening”. Check out this breakdown of the top advantages.


## **1. Straightforward view**

When data is presented in visual format, it becomes much more intuitive and easier to grasp. This is because the brain is naturally better at processing and understanding visual information. 

With graph database visualization, you can see the data in a way that is more similar to how you would see and understand the real-world objects or relationships that the data represents. 


## **2. You can easily discover more revelations**

Visual data formats allow users to easily find patterns and relationships that may not be immediately apparent in non-visual formats.. 

The visual appeal of the data also makes it easier for users to explore it more extensively. It encourages users to be more involved with the info, so they'll spend more time there and uncover other perspectives as a result. The fact that the data is visually appealing makes it even easier for them to explore it more extensively - leading to even further discoveries!


## **3. You can easily explain findings**

Graph database visualization allows users to easily share and explain their findings by highlighting important nodes and adding notes or comments. 

By adding annotations or highlights to specific nodes, you can can draw attention to trends or patterns in the data and help your colleagues to quickly recognize the insights that you have discovered


## **4. Friendly for non-experts**

Even users who have little or no technical knowledge in data analysis will find it so easy to make use of graph visualization.  

Many graph visualization tools are designed to be user-friendly and intuitive, with features such as interactive elements, hover-over text, and filtering options that allow users to explore the data and customize their views. 


## **5. You can zoom in and out**

Graph database visualization enables you to zoom in or out and explore the data at varying levels of detail. You can narrow in to pinpoint some areas of the graph or widen your view to observe what it looks like overall. 

This can be beneficial when you're attempting to recognize how dissimilar parts of the data interconnect or require finer details. Say, if you're analyzing a large network of people, you could zero in on particular clusters of people to see how they are interrelated or broaden your view to see the complete structure of the network.


# **Graph database visualization use cases**

Any context that requires understanding and analyzing complex data relationships is a good candidate for graph database visualization. Here are some common use cases:  


## **1. Fraud detection and investigation**

Fraud is a serious problem that is already having significant consequences for individuals, businesses, and society as a whole. It can take many forms, including identity theft, and[ financial fraud](https://www.nebula-graph.io/posts/financial-fraud-detection-one-of-the-best-practices-of-knowledge-graph) such as investment fraud, insurance fraud, bank fraud, mortgage fraud, and charity fraud, to name a few.

Graph database visualization can help to detect all these forms of fraud by identifying patterns and anomalies in data that may indicate fraudulent activity. It can be particularly useful for identifying suspicious patterns or behaviors, such as consistent trades on the same stocks or unusual money transfers between accounts.

![fraud-detection-using-graph-visulization](http://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/fraud-detection-using-knowledge-and-graph-database/figure3.png)

## **2. Supply chain management**

Supply chain management is a complex process with many moving parts. Graph visualization can help supply chain managers to coordinate and optimize many activities and entities in the production and distribution of goods and services.

You can so easily visualize data for example the flow of materials, information, and financial transactions as a graph, enabling you to quickly respond to bottlenecks and identify opportunities for optimization.


## **3. Cybersecurity**

[Cybersecurity](https://www.nebula-graph.io/posts/how-to-use-graphs-for-cybersecurity) has become a top priority for organizations in all industries, particularly due to these reasons:



* **Growing number of cyber threats:** There has been a dramatic increase in the number and sophistication of cyber threats in recent years, ranging from malware and phishing attacks, to ransomware and data breaches.
* **Increased regulation:** Many industries are now subject to various regulatory requirements that mandate certain cybersecurity practices and standards. For example, the healthcare industry is subject to HIPAA regulations, which require the protection of personal health information, and the financial industry is subject to various regulations, such as the [Gramm-Leach-Bliley Act](https://www.ftc.gov/business-guidance/privacy-security/gramm-leach-bliley-act), which require the protection of personal financial information.

Companies can use graph visualization to address the above cybersecurity challenges in the following ways:



* **Network security:** If you work in network security, try to represent the relationships between different entities (e.g., devices, users, and IP addresses) in your network as a graph and see how easy it is to identify anomalies that may indicate a security breach. For example, if a group of devices that normally communicate with each other suddenly start communicating with a new, unknown device, this could be a red flag and warrants further investigation.
* **Threat intelligence:** Visualize the relationships between different types of cyber threats (e.g., malware, phishing attacks, etc.) and the entities (e.g., IP addresses, domains, etc.) associated with them. Armed with this visual presentation, you will easily understand how different threats are related and how they may be used together in coordinated attacks.
* **Incident response:** In the event of a security breach, an organization can use graph visualization to understand the scope and impact of the incident. Simply represent the affected assets, the entities involved, and the timeline of events as a graph, and you could be on your way to quickly identifying the root cause of the breach. Next is to deploy the appropriate response plan.


## **4. Knowledge management**

Knowledge management is the process of creating, sharing, using and managing the knowledge and information of an organization. It involves capturing, distributing, and effectively using knowledge to support the goals and objectives of the organization. The goal of knowledge management is to improve efficiency and effectiveness by making the most of an organization's collective expertise and experience. 

A graph database can be used to represent the relationships between different concepts and ideas in a body of knowledge. Visualizing these relationships can help users understand the structure of the knowledge and how different concepts are related to one another.


## **5. IT operations management**

IT operations management refers to the process of managing the day-to-day operations of an organization's IT infrastructure and systems. This includes tasks such as monitoring and maintaining the availability and performance of IT systems, and managing the configuration and deployment of hardware or software.

One use case of graph database visualization in IT operations management is identifying and troubleshooting issues. IT operations teams can do this by creating a visual representation of the relationships between different components and events. This allows them to easily see how the different components may be impacting each other, enabling them to more effectively identify and resolve issues


## **6. Drug discovery**

Drug discovery is the process of identifying and developing new drugs to treat specific diseases or conditions. This process involves a number of activities, including identifying potential drug targets, synthesizing and testing potential drug candidates, and conducting clinical trials to evaluate the safety and effectiveness of the drug.

Researchers can use graph database visualization to identify potential drug targets by creating a visual representation of the relationships between different biological pathways.

For example, consider a researcher who is studying a particular disease, such as cancer. They may create a graph that represents the relationships between different proteins and genes that are known to be involved in the disease, as well as the pathways in which they are involved. By analyzing this graph visually, the researcher can identify potential targets for new drugs that may be able to disrupt the disease process.


## **7. Recommendation engines**

A recommendation engine is a system that makes personalized recommendations to users by identifying patterns in data. The recommendations are based on the user's past behavior, as well as the behavior of similar users. Recommendation engines are commonly used in e-commerce, media, and social media to help users discover new products, articles, or friends that they might be interested in.

Here are a couple of ways that organizations are putting graph database visualization to work in recommendation engines:



* **E-commerce:** Many e-commerce companies use graph database visualization to help recommend products to users based on their past purchases and the purchases of similar users..
* **Social media:** Social media platforms often use graph database visualization to make content or friend recommendations to users based on their connections and interactions with other users.
* **Media:** Media companies can use graph database visualization to recommend articles, videos, or other content to users based on their past consumption and the consumption of similar users. They use graph visualization to identify patterns in data and highlight groups with similar interests.
* **Gaming:** Gaming companies can use graph database visualization to recommend games or in-game items to users based on their playstyles and the playstyles of similar users. It helps identify patterns in data and highlight groups with similar interests.


# **Graph database visualization with NebulaGraph**

NebulaGraph offers an intuitive enterprise graph visualization tool known as [NebulaGraph Explorer](https://explorer.nebula-graph.io/welcome), which is browser-based. You can use this tool in conjunction with the NebulaGraph core to visualize complex interactions in your graph data. As we indicated earlier, you need not be a graph database expert in order to comfortably use a visualization tool, and so you’ll be glad to know that NebulaGraph Explorer is easy to use even for a non-technical user. 

The [NebuaGraph Manual](https://docs.nebula-graph.io/2.5.0/pdf/NebulaGraph-EN.pdf) has all the friendly help you need to get started with this amazing graph database visualization tool - please check it out. Simply go to the table of contents, look for Nebula Explorer and click. Whether you're working in e-commerce, media, social media, or any other industry, Nebula Explorer will help you get the most out of your data.


# **Conclusion**

We sincerely hope you now understand how beneficial graph database visualization can be for anyone working with data, regardless of the level of expertise or technical skill. From subject matter experts and data scientists to analysts, developers and non-technical users - anyone who wants to explore their data and discover hidden insights can benefit from graph visualization.

Finally, it's important to stay focused on the data itself, rather than just being captivated by the aesthetics of the visuals. We say this because it is easy to be mesmerized by the beauty or elegance of a visualization and overlook the underlying data and what it's actually trying to tell you. This can lead to a superficial understanding of the data and may prevent you from fully grasping the trends and patterns that are present in the data. The key is to move beyond just looking at the visualization, and instead really delve into the data points which make it up, while also taking into account any context or questions which it is meant to answer.


 Dealing with complex interconnected data? Learn how to effectively visualize your graph database to gain deeper insights and better understand your data.

Graph Database Visualization | Graph-Based Analytics and Visualization with NebulaGraph

Graph machine learning is used for fraud detection by analyzing the connections and relationships between entities in a network. It can be applied to a wide range of datasets, such as financial transactions, communication logs, and social media interactions, to uncover patterns and anomalies that indicate fraudulent activity.

Fraud Detection with Graph Machine Learning

Have you ever pondered why cyber attackers always appear to be one step ahead of security teams?

It’s because their work is much less extensive. 

While those who are tasked with defending networks and systems need to identify and block every potential vulnerability, attackers just have to search for a single flaw - no matter how small it is. From this single entry point, they can cause massive damage.

This discrepancy in focus gives attackers greater agility and malleability, allowing them to swiftly take advantage of newly found vulnerabilities.

On the other hand, security teams often find themselves spread thin as they struggle to keep up with an array of constantly evolving cyber threats. They have to make sure they can handle every attack or threat, continually searching all possible sources - which is an overwhelming and extremely time consuming undertaking.

But with the entry of the [graph database](https://nebula-graph.io/) era, we can change this and put the attackers behind. 

Let’s find out. 


# **Graphs for cybersecurity: what does it mean?**

It means that for the first time, security teams can get ahead of the cyber attackers.

Many of us might actually be surprised to learn that cyber attackers often view an organization's network from the perspective of a graph. So they mostly concentrate on getting access to just one entry point (node). Using this one entry point, they can construct an attack path to easily penetrate the entire infrastructure. 

What about security teams? They have historically relied on lists of alerts from their security systems. Since the lists are simply linear, they don’t offer a complete overview.

On the other hand if you were to use the graph approach, it means you'll essentially have a holistic view of the entire infrastructure and you can constantly add to the graph as more items are added to the network. In other words while the attackers are busy looking for an entry point, you have the entire view to yourself. You are now one step ahead as opposed to the traditional list approach where you lag behind. 

The concept is really simple: Make use of graph data structures and visualization techniques to represent then analyze it using [graph analytics](https://www.nebula-graph.io/posts/graph-analytics-and-graph-databases) to unearth cybersecurity issues. This includes data such as network traffic, log files, and malware samples. By representing this data as nodes and edges in a graph, analysts can quickly identify patterns and relationships that may not be immediately obvious in other forms of representation.

The nodes in the graph represent the data points, such as IP addresses, domains, files, or processes. The edges represent the relationships between the nodes, such as a connection between an IP address and a domain. 


# **How to use graphs for cybersecurity ( how it works)**

The data from various sources such as network traffic, log files, and malware samples is transformed into a format that can be used to create the nodes and edges of the graph. Once the data is in this format, it can be loaded into a **graph database** as **NebulaGraph**.

Once the data is in the graph, it can be queried, analyzed and visualized. For example, an attack analyst can use graph algorithms to identify patterns and relationships, such as the shortest path between two nodes, which can help them trace the origin of an attack or track the spread of malware. You can also use [graph visualization](https://www.nebula-graph.io/posts/graph-database-visualization) tools to create interactive diagrams that allow you to explore the data and identify anomalies or potential threats.

Additionally, graph databases can be integrated with other cybersecurity tools such as [intrusion detection](https://www.nebula-graph.io/posts/intrusion-detection-with-graph-technology) systems, network monitoring tools, and threat intelligence platforms. This allows analysts to correlate data from multiple sources, providing them with a more complete picture.

Very briefly, these are the general steps you will undertake when using graphs for cybersecurity: 



1. **Data collection:** As mentioned earlier, you’ll basically collect data from all the critical points in the infrastructure such as network logs, system events, and security devices. Network logs provide information about the traffic flowing through the network, including IP addresses, ports, and protocols. System events provide information about the activity on a particular system, including user logins, file access, and system updates. Security devices, such as firewalls and intrusion detection systems, provide information about threats and attacks that have been detected on the network.
2. **Data preprocessing:** The goal of data preprocessing is to prepare the collected data for further analysis. This typically involves a series of tasks such as data cleaning, data integration, data transformation, data reduction, and data discretization which is the conversion of continuous data into categorical data.
3. **Graph construction:** The preprocessed data is used to build a graph that can be used to analyze the data and identify patterns that could point to security issues. The process of graph construction typically involves several sub-steps, such as node creation, edge creation, attribute assignment, clustering, and visualization.
4. **Graph analysis:** Involves using graph-based algorithms to analyze the graph built in the previous step and identify patterns, anomalies, and potential threats. There are different types of graph-based algorithms that can be used for graph analysis, such as centrality measures,clustering algorithms,link analysis algorithms, and anomaly detection algorithms.
5. **Threat hunting:** Actively searching for potential threats in the graph, rather than passively waiting for alerts or alarms to be triggered. The process of threat hunting typically involves approaches including hypothesis generation, data exploration, verification,and mitigation: 
6. **Remediation:** Can involve anything from implementing a patch or software update to reconfiguring an entire system. 
7. **Continuous monitoring and improving:** This could involve adding or removing nodes, changing access control policies, implementing new algorithms, and more.


# **Why should your organizations consider switching to graphs for cybersecurity?**

Graphs offer an all-encompassing, comprehensive look at the network. This perspective has been absent, and as we've observed, this is precisely why security teams are constantly fighting fires.

It's telling that large companies like [Cisco have already started incorporating graphs in their security products](https://blogs.cisco.com/security/big-data-in-security-part-iii-graph-analytics) - clearly, there's something special about graphs that the major players have uncovered

Fortunately it’s not a secret. It’s all about these strengths: 


<table>
 <tr>
 <td>Holistic view
 </td>
 <td>The ability to see the interconnectedness of different elements within a network or system. This includes understanding how different nodes and edges relate to one another, and how they may impact the overall security. 
 </td>
 </tr>
 <tr>
 <td>Visualization
 </td>
 <td>The ability to represent data in a variety of ways, such as node-link diagrams, heat maps, and matrix plots. These different visualizations can be used to represent different aspects of the data, such as network topology, user activity, and threat detection.
 </td>
 </tr>
 <tr>
 <td>Real-time threat intelligence 
 </td>
 <td>Graphs allow for real-time analysis and correlation of data from multiple sources, such as network logs, system events, and threat intelligence feeds. With this, security teams can detect and respond to threats more rapidly.
 </td>
 </tr>
 <tr>
 <td>Prediction
 </td>
 <td>This can involve using machine learning algorithms.
 </td>
 </tr>
 <tr>
 <td>Investigation
 </td>
 <td>While it's not possible to see everything especially when you are dealing with huge data sets, the ability to narrow down to small points within the data and investigate, is a powerful element of a security solution that is powered by graphs. It means security professionals can pick a few nodes and engage them at a higher level.
 </td>
 </tr>
 <tr>
 <td>Compliance
 </td>
 <td>Graph-based security solutions can help organizations comply with industry and government regulations, such as HIPAA, PCI-DSS, and NIST. For example, you’ll be able to easily segment and access controls that are required by compliance bodies. 
 </td>
 </tr>
 <tr>
 <td>Concentration
 </td>
 <td>This appealing nature of graphs can help analysts stay focused and motivated as they work through large amounts of data. It can also make it easier to communicate their findings to other stakeholders.
 </td>
 </tr>
</table>


All these elements contribute to a cost-effective cybersecurity solution, meaning you can detect and block more threats or attacks at a much more affordable rate than the traditional method of solely relying on lists.


# **Conclusion**

Our brains mostly think best in terms of images, patterns, and relationships. This makes visual representation of data, such as graphs, more intuitive, easier to understand and analyze. Graphs in particular are an effective way to represent complex data and relationships, making it easier to identify patterns, anomalies, and potential threats. 

This is why using graphs for cybersecurity is proving disruptive. It’s all your security team needs to get ahead of the ever elusive cyber criminals. 

# How NebulaGraph Works
1. [NebulaGraph Architecture — A Bird’s Eye View](https://www.nebula-graph.io/posts/nebula-graph-architecture-overview)
2. [Benchmark: NebulaGraph vs Dgraph vs JanusGraph](https://www.nebula-graph.io/posts/benchmarking-mainstraim-graph-databases-dgraph-nebula-graph-janusgraph)
3. [Comparison: Neo4j vs NebulaGraph vs JanusGraph](https://www.nebula-graph.io/posts/performance-comparison-neo4j-janusgraph-nebula-graph)
4. [Deploy the Graph Database on Kubernetes](https://www.nebula-graph.io/posts/how-to-deploy-nebula-graph-in-kubernetes)

Make use of graph data structures and visualization techniques to represent then analyze it using graph analytics to unearth cybersecurity issues. This includes data such as network traffic, log files, and malware samples. By representing this data as nodes and edges in a graph, analysts can quickly identify patterns.

How to Use Graph Database for Cybersecurity


# What is a distributed system?

Generally, a distributed system is a set of computer programs that work together across multiple independent servers to achieve a common goal. Those servers refer to those commodity servers instead of mainframes. The hardware for cross-server collaboration here is mostly based on Ethernet devices or higher-end RMDA devices. 

# Why do we need a distributed system?

The main reason to build a distributed system is to replace the cost of expensive hardware devices with software technology and inexpensive hardware devices. Especially in most private server rooms, not public cloud or supercomputing conditions, procurement costs are an important basis for business decisions. 

In addition to reducing costs, another benefit of distributed technology is its scalability. By adding several servers to the original number of servers, and then combining the scheduling and distribution ability of the distributed system, the new servers can be used to provide additional services.  

Compared with purchasing more servers in equal numbers or purchasing higher configuration servers, the distributed technology allows you to purchase servers on demand, which reduces the risk of over-provisioning and improves the utilization rate of the hardware resources.

# Basic problems with distributed systems

In distributed technologies, since data storage and computation need to be implemented across multiple independent servers, a series of underlying technologies must be involved. In this article, we discuss only two problems: **One is the data copy or replica problem, and another is how to distribute the storage and computation of large data to independent servers.**

## The problem of data replicas 

The hardware reliability and maintenance of commodity servers are much lower than that of mainframes. Because loose network cables, damaged hard drives, and power failures occur almost every hour in large machine rooms. It is a basic problem for a distributed software system to solve or avoid these hardware problems. One common solution is to replicate data on multiple servers. Once some data replicas are lost, the system can still provide services by using the remaining data replicas. What's more, when the access load of the system is too large, the system can also provide more services by adding more replicas. In addition, some technologies are needed to ensure that the data replicas are consistent with each other, that is, the data of each replica on different servers is the same. For graph databases, the data replica problem also exists. The way to solve this problem is similar to the way to solve the data replica problem in relational databases or big data systems.


## The problem of data partitioning

The hardware, memory, and CPU of a single server are limited. If the data is too large, it is impossible to store all the data on a single server. **Therefore, the TB-level or even PB-level data must be distributed to multiple servers, and we call this process data partitioning**. When a request is to access multiple data partitions, the distributed system needs to distribute the request to each correct data partition and then combine the results.

# The problem of data partitioning in graph databases - graph partitioning 

In graph databases, the distribution process is imaginatively called graph partitioning. **A big graph is partitioned into multiple small graphs, and the storage and computation of each small graph are stored on different servers**. 
Compared with the partitioning problem in relational databases and big data systems, the graph partitioning problem deserves more special attention.

Let's take a look at a static graph structure, such as the CiteSeer dataset, which is a citation network of scientific papers, consisting of 3312 papers and their citations between them. It is a small-scale dataset, so it can be stored on a single server.

The Twitter 2010 dataset, is a social network of Twitter users, consisting of 12.71 million vertices and 0.23 billion edges. It is relatively easy to store this dataset on a single mainstream server produced in 2022. However, it may be necessary to purchase very expensive high-end servers produced 10 years ago to do so.

However, the WDC (Web Data Commons) dataset, consists of 1.7 billion vertices and 64 billion edges. It is difficult or impossible to store such a large-scale dataset on a current mainstream server.

On the other hand, since the data of human beings is growing faster than Moore's Law and the number of connections or relationships between data is exponentially higher than the speed of data production, the data partitioning problem seems to be an inevitable problem for the graph database system. But it sounds no different from the way data is partitioned or hashed in the mainstream distributed technologies. After all, data is partitioned into multiple big data systems.

Wait, is it that easy to partition a graph?

No, it is not. In the field of graph databases, the graph partitioning problem is a trade-off among technologies, products, and engineering.  

## Three problems faced by graph partitioning

**The first problem** - what should be partitioned? In big data or relational database systems, **row-based partitioning or column-based partitioning** is performed based on records or fields, or partitioning is performed based on data IDs, and these are intuitive in terms of semantics and technology. However, the strong connectivity of the graph data structure makes it difficult to partition the graph data. One vertex may be connected to many other vertices through multiple edges, and the other vertices may also be connected to many other vertices through their neighboring edges. It is just like the web pages that are almost linked to each other. So for a graph database, what should be partitioned that can make the semantics intuitive and natural? (In RDBMS, this is equivalent to how to partition the data when there are a large number of foreign keys in the table.) Of course, there also exist some natural semantic partitioning methods. For example, under the COVID-19 epidemic, the transmission chain of various strains in China and other countries are two different network structures.  

Then, a second problem is introduced.

**The second problem** - how to ensure that the data of each partition is roughly balanced after the data is partitioned. The naturally formed graphs conform to the power low that a minority of 20% vertices are connected to other 80% vertices, and these minority vertices are called super nodes or dense nodes. This means that the minority vertices are associated with most of the other vertices. Therefore it can be expected that the load and hotspots of the partition containing the super nodes are much higher than that of the other partitions containing the other vertices.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/distribution-and-partitioning-in-graph-database/2.png)

The above image shows the visual effect of the association network formed by hyperlinks to websites on the Internet, where the super websites (nodes) are visible.

**The third problem** - how to evaluate and perform repartitioning when the original partitioning methods are gradually outdated as the graph network grows and the graph distribution and connection patterns change? The following image shows the visual effect of the connections among 86 billion neurons in the human brain. With learning, exercise, sleep, and aging, neuronal connections are constantly changing at the weekly level. The original partitioning method may not be able to keep up with the changes at all.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/distribution-and-partitioning-in-graph-database/3.png)

Of course, many other details need to be considered. In this article, we try to avoid using too many technical terms. 

Unfortunately, there is no silver bullet for the graph partitioning problems from a technical point of view and each product has to make its trade-offs. 

Here are some examples of the trade-offs made by different products.

# Partitioning methods in different graph database products

## 1. Distributed but unpartitioned

Neo4j 3.5 adopts the unpartitioned distributed architecture.

**The reason to use a distributed system is to ensure the consistency and the read availability of the written data in multiple replicas.**

**That means the full amount of graph data is stored on each server, and the size of data cannot be larger than the capacity of a single server's memory and hard disk. We can ensure the failure of a single server in the process of data writing by adding multiple write replicas. We can improve the read performance by adding multiple read replicas (The write performance is not improved).**

This solution can avoid the above-mentioned three problems of graph data partitioning and in theory, there is nothing wrong with calling such a solution a distributed graph database.

To add to that, since the data is complete on each server, ACID transactions are relatively easy to implement.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/distribution-and-partitioning-in-graph-database/4.png)

For details about the Neo4j 3.5 causal cluster architecture, see https://medium.com/neo4j/querying-neo4j-clusters-7d6fde75b5b4.


## 2. Distributed and partitioned by users

The distributed and partitioned by users architecture is typically represented by Neo4j 4.x Fabric. Depending on the user's business case, users can specify that subgraphs can be placed on a (group of) server(s). For example, in one cluster, the subgraph of the product E is placed on the server E and the subgraph of the product N is placed on server N. (Of course, for the availability of the service itself, these servers can also be placed in the Causal Cluster mentioned in the above-mentioned image.) In this process, for both the write and read operations, the user needs to specify a server or a group of servers to operate.

This solution leaves the above-mentioned three problems to the user for decision at the product level. Such a solution is also called a distributed graph database.

What's more, this solution can guarantee the ACID transactions in server E. However, there are a certain amount of edges that connect the vertices in server E and the vertices in other servers, so the ACID transactions of these edges cannot be guaranteed technically.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/distribution-and-partitioning-in-graph-database/5.png)

For details about the Neo4j 4.x Fabric architecture, see https://neo4j.com/developer/neo4j-fabric-sharding/.


## 3. Non-equivalent distributed, partitioned, and coarse-grained replicated

This solution allows multiple replicas and graph data partitioning, and these two processes require a small number of users to be involved.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/distribution-and-partitioning-in-graph-database/6.png)

For details about TigerGraph's partitioning solution, see https://www.youtube.com/watch?v=pxtVJSpERgk.

In TigerGraph's solution, vertices and edges are scattered in multiple partitions after being encoded.

The first two problems among the above-mentioned problems can be partially solved by encoding vertices and edges. Users can make decisions on whether to read or compute the data in a distributed system or a single server. 

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/distribution-and-partitioning-in-graph-database/7.png)
![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/distribution-and-partitioning-in-graph-database/8.png)

The above image shows TigerGraph's single read mode and parallel computing mode.

However, such a set of partitions must be replicated in full and identical copies (so the granularity of the scale-out is the entire graph instead of a partition), which requires a larger amount of storage space.


## 4. Full-equivalent distributed, partitioned, and fine-grained replicated

There are also solutions whose architectural design purpose ranks graph scalability or resiliency relatively at the highest priority of the overall system design. The assumption is that data is being generated faster than Moore's Law, and the interactions and relationships between data are exponentially higher than the rate of data generation. Therefore it is necessary to be able to handle the such explosive growth of data and provide services quickly.

In this solution, the obvious characteristic is the separation design of the storage layer and the computing layer, each having the ability for fine-grained scalability.

Data is partitioned in the storage layer with a hash or consistent hash solution. Hashing is performed based on the IDs of vertices or primary keys. This solution just solves the first problem.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/distribution-and-partitioning-in-graph-database/9.png)

For details about the architecture of [NebulaGraph](https://nebula-graph.io), see the paper *Wu, Min, et al. "Nebula Graph: An open-source distributed graph database." arXiv preprint arXiv:2206.07278 (2022)*. 

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/distribution-and-partitioning-in-graph-database/10.png)

For details about the architecture of ByteGraph, see Li C, Chen H, Zhang S, et al. ByteGraph: A High-Performance Distributed Graph Database in ByteDance[J]. 

To deal with the super node and load balancing problem (the second problem), another layer of the B-tree data structure is introduced. It splits super nodes into multiple processing units, balances data among threads, and scales out the computing layer.

For the third problem, the solution is to use a fine-grained partitioning method so that the scale-out of some partitions can be performed.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/distribution-and-partitioning-in-graph-database/11.png)

For sure, this solution is also called [a distributed graph database](https://nebula-graph.io).

The above-mentioned four solutions make different trade-offs at the product and technical levels, with a focus on suitable business scenarios. These solutions can all be called distributed graph databases.

# Extended reading


- The problem of graph partitioning

  There has been a lot of research on how to partition graphs on a single machine. Here is the recommended tool METIS http://glaros.dtc.umn.edu/gkhome/metis/metis/overview.

  Graph partitioning is a very difficult problem in a distributed system. The following articles are recommended for further reading.

  - https://dgraph.io/blog/post/db-sharding/ 

  - https://hal.inria.fr/hal-01401338/document 

  - https://docs.janusgraph.org/advanced-topics/partitioning/ 


- About Pseudo-Distributed Mode

  Generally, this mode uses only a single node and simulates the distributed mode. It is recommended to use this mode for studying and debugging.

- An overview of the graph databases

    - Sahu, S., Mhedhbi, A., Salihoglu, S., Lin, J., ¨Ozsu, M.T.: The ubiquity of large graphs and surprising challenges of graph processing: an extended survey. VLDB J. 29(2-3), 595–618 (2020) 

    - Sakr, S., Bonifati, A., Voigt, H., Iosup, A., Ammar, K., Angles, R., Yoneki: The future is big graphs: a community view on graph processing systems. Communications of the ACM 64(9), 62–71 (2021) 

# Disclaimer

1. The above-mentioned solutions are the author's understanding and should not be used as a reference for making investment decisions. If there are any errors, please feel free to point them out.

2. To reduce the difficulty of reading, the author has simplified the description of the solutions, which does not mean that the original solutions are so simple.


This article summarizes the designs of the distribution and partitioning for several different types of graph databases to help you understand the differences between them.

Tag