This article attempts to explore the traces of these contributors using graph visualization, "peeping" into the relationship between contributors and projects in the Open Source NebulaGraph Community and the PR traces they leave behind.

Exploring Contributor Relationships in the Open Source NebulaGraph Community: A Graph Visualization Approach

We will now explore new techniques for pinpointing important content using common neighbors, pushing information flow based on friend relationships and geographic location, and using spatio-temporal relationship mapping to query the relationship between people. 

7 Fundamental Use Cases of Social Networks with NebulaGraph Database | EP 3

We have had a thorough introduction to graph data modeling and data import in the application of social network with graph technology. 

[**7 Fundamental Use Cases of Social Networks with NebulaGraph Database | Episode 1**](https://www.nebula-graph.io/posts/social-networks-with-graph-database-1)

Now, let me introduce more use cases in this episode.

# Identifying key people

Identifying influencers in social networks involves using a variety of metrics and methods to identify individuals who have a lot of influence in a given network. This is useful for many business scenarios, such as for marketing or researching the spread of information in a network.

There are many ways to identify them, and the specific methods and information, relationships, and perspectives considered also depend on the type of these key individuals, and the purpose of acquiring them.

Some common methods include looking at the number of followers a person has or the amount of content consumed, their reader engagement on their posts, videos, and the reach of their content (retweets, citations). These methods are also doable on the graph but are rather mundane, so I won't give examples. Here, we can try to derive these key people on the graph using graph algorithms that evaluate and calculate the importance of nodes.

## PageRank

PageRank is a very classic graph algorithm that iterates through the number of relationships between points on a graph to get a score (Rank) for each point. It was originally proposed by Google founders Larry Page and Sergey Brin and used in the early Google search engine to sort search results, where Page can be a pun on Larry Page's last name and Web Page.

PageRank has long been abandoned as too simple in modern, complex search engines, but it still shines in other graph-structured web scenarios, where we can roughly assume that all links are of similar importance and run the algorithm to find those key users in social networks.

In NebulaGraph, we can use NebulaGraph Algorithm, NebulaGraph Analytics to run PageRank on large full graphs, while in the daily analysis, validation, and design phases, we don't need to run results on full data, but on very small subgraphs (up to tens of thousands), we can easily run various graph algorithms in the browser to derive methods that can be used for production.

Today, we will use the built-in in-browser graph algorithm function of NebulaGraph Explorer to execute PageRank (the specific method is omitted here, you can refer to the [documentation](https://docs.nebula-graph.io/3.3.0/nebula-explorer/graph-explorer/graph-algorithm/ ), but it's really just a matter of mouse clicks).

![PageRank](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/PageRank.png)

We can see from the above that among all the green players (people) after PageRank calculation, "player.name: Tim Duncan" is the largest one, and the relationship associated with it does seem to be quite a lot, so we select him on the graph, then right-click to invert, select all the points except Tim Duncan, use the backspace key to delete all the other points, and then explore 1 to 5 steps in both directions with him as the starting point. In one step, we get Tim Duncan's subgraph.

![TimDuncan](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/TimDuncan.png)

As you can see from the subgraphs, Tim Duncan is associated with a very large number of other players, while some other very popular players have served with him in the very popular Spurs team, which confirms the way PageRank is evaluated.

Now let's see if the algorithm will come to the same conclusion for the other dimensions of determination.

## Betweenness Centrality

As you can see from the subgraphs, Tim Duncan is associated with a very large number of other players, while some other very popular players have served with him in the very popular Spurs team, which confirms the way PageRank is evaluated.

Now let's see if the algorithm will come to the same conclusion for the other dimensions of determination.

![Betweeness_centrality](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/Betweeness_centrality.png)

From its five-hop subgraph, it can be seen that unlike the star-shape of the key figure Tim Duncan obtained from PageRank before, Dejounte Murray's subgraph shows clusters, where it is sensory, intuitive to imagine that Dejounte Murray is really on the necessary path of the minimal path between many nodes, while Tim Duncan seems to be associated with more important connecters.

![DejounteMurray](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/DejounteMurray.png)

In practical application scenarios, we usually have to understand the definitions in different ways, experiment with different execution results, and analyze to find the structural features that affect the key people we care about, and use them to choose different algorithms for different needs.

# Detect communities and clusters

Community detection in social networks is a technique to discover community structure by analyzing social relationships. A community structure is a set of nodes that are closely connected to each other in a social network, graph, and these nodes usually have similar characteristics or interests. For example, a community structure may manifest itself as a group of users who are clustered together based on common topics or interests.

The purpose of community detection is to identify the boundaries of different communities and determine the nodes in each community by analyzing the social network. This process can be done by using various algorithms such as label propagation algorithm, weakly connected component algorithm and Louvain algorithm. By discovering the community structure, we can better understand the structure and characteristics of social networks, and help social network service providers to better infer and predict behaviors in social networks, and help in good social network governance, advertisement placement, marketing, etc.

Since our dataset is fake-generated, the results I get under different algorithms do not show the real meaning, so this chapter just shows the results after community identification using several graph algorithms, in real-world cases, we should also use domain knowledge or other technical means on top of that to collaboratively give the portraits and labels of different groups and communities.

Effect of label propagation algorithm.

![LPA](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/LPA.png)



Louvain algorithm:

![Louvain](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/Louvain.png)

WCC algorithm:

![WCC](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/Louvain.png)

In later sections, we could in better chance verify these algorithms again on smaller and simpler subgraphs, with somewhat more interpretable results.

# Friend Closeness

With the community detection algorithm, it is actually possible to obtain friends with similar interests and close associations to some extent, in a global calculation. So how do we get the other close friends of a given user? We can get this information by counting the number of friends this user has in common with him in order to get this information!

Let's take "Tim Duncan" for example, we know that his two-degree friends (friends of friends: `(:player{name: "Tim Duncan"})-[:follow]-(f:player)-[:follow]-(fof:player)`) are also his friends: Mutual Friend, then it is reasonable to believe that those who have more friends in common with Tim Duncan may have a higher closeness to him.

```cypher
MATCH (start:`player`{name: "Tim Duncan"})-[:`follow`]-(f:`player`)-[:`follow`]-(fof:`player`),
(start:`player`)-[:`follow`]-(fof:`player`)
RETURN fof.`player`.name, count(DISTINCT f) AS NrOfMutualF ORDER BY NrOfMutualF DESC;
```

This query shows that "Tony Parker" and Tim have 5 friends in common and are the closest.

| fof.player.name   | NrOfMutualF |
| :---------------- | :---------- |
| Tony Parker       | 5           |
| Dejounte Murray   | 4           |
| Manu Ginobili     | 3           |
| Marco Belinelli   | 3           |
| Danny Green       | 2           |
| Boris Diaw        | 1           |
| LaMarcus Aldridge | 1           |
| Tiago Splitter    | 1           |

Here, let's verify this result through visualization!

First, let's see who the common friends (f:) are for each of the friends.

```cypher
MATCH (start:player{name: "Tim Duncan"})-[:`follow`]-(f:player)-[:`follow`]-(fof:player),
(start:player)-[:`follow`]-(fof:player)
RETURN fof.player.name, collect(DISTINCT f.player.name);
```

The result:

| fof.player.name   | collect(distinct f.player.name)                              |
| :---------------- | :----------------------------------------------------------- |
| Boris Diaw        | ["Tony Parker"]                                              |
| Manu Ginobili     | ["Dejounte Murray", "Tiago Splitter", "Tony Parker"]         |
| LaMarcus Aldridge | ["Tony Parker"]                                              |
| Tiago Splitter    | ["Manu Ginobili"]                                            |
| Tony Parker       | ["Dejounte Murray", "Boris Diaw", "Manu Ginobili", "Marco Belinelli", "LaMarcus Aldridge"] |
| Dejounte Murray   | ["Danny Green", "Tony Parker", "Manu Ginobili", "Marco Belinelli"] |
| Danny Green       | ["Dejounte Murray", "Marco Belinelli"]                       |
| Marco Belinelli   | ["Dejounte Murray", "Danny Green", "Tony Parker"]            |

 Then we visualize the result on Explorer.

- First, let's find out all of Tim's 2-degree friend paths

```cypher
MATCH p=(start:player{name: "Tim Duncan"})-[:`follow`]-(f:player)-[:follow]-(fof:player)
RETURN p
```

- Then we render the node size by degree in which we select Tim and Tony and find all paths between them for follow type edge, bidirectional, up to 2 hops:

We can see that they are the closest of friends to each other and that their mutual friends are also in the paths.

`["Dejounte Murray", "Boris Diaw", "Manu Ginobili", "Marco Belinelli", "LaMarcus Aldridge"]`

![closest_friend](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/closest_friend.png)

## Small groups in your friends

At this point, as mentioned earlier, the non-authenticity of this dataset itself makes the results of the community discovery algorithm unable to get the insightful connotation of it. Now we can follow this small subgraph to see how groups and communities can be distinguished among Tim's friends.

- Weakly connected components can split Tim's friends into two or three parts that are not connected to each other, which is very much in line with the intuitive understanding and definition of connected components.

![Tim_wcc](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/Tim_wcc.png)

- Label propagation, we can control the number of iterations on-demand to delineate different degrees of division by random propagation, which results in a certain degree of differentiation.

  20 iterations

  ![Tim_LPA](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/Tim_LPA_1000.png)

  1000 iterations

  ![Tim_LPA_1000](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/Tim_LPA_1000.png)

- Louvain, a more efficient and stable algorithm, basically under this subgraph we can get a very intuitive division with a very small number of iterations.

![Tim_Louvain](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/Tim_Louvain.png)

# New friend recommendation

Following the previous idea of 2-degree friends (friends of friends), we can easily add those 2-degree friends who are not yet friends as recommended friends, and the sorting rule is the number of friends they have in common with each other:

```cypher
MATCH (start:player{name: "Tim Duncan"})-[:`follow`]-(f:player)-[:`follow`]-(fof:player)
WHERE NOT (start:player)-[:`follow`]-(fof:player) AND fof != start
RETURN fof.player.name, count(DISTINCT f) AS NrOfMutualF ORDER BY NrOfMutualF DESC;
```

| fof.player.name   | NrOfMutualF |
| :---------------- | :---------- |
| LeBron James      | 2           |
| James Harden      | 1           |
| Chris Paul        | 1           |
| Yao Ming          | 1           |
| Damian Lillard    | 1           |
| JaVale McGee      | 1           |
| Kevin Durant      | 1           |
| Kyle Anderson     | 1           |
| Rudy Gay          | 1           |
| Russell Westbrook | 1           |

Obviously, LeBron is the most recommended! And look at who these mutual friends are.

| fof.player.name   | collect(distinct f.player.name)    |
| :---------------- | :--------------------------------- |
| James Harden      | ["Dejounte Murray"]                |
| LeBron James      | ["Danny Green", "Dejounte Murray"] |
| Chris Paul        | ["Dejounte Murray"]                |
| Yao Ming          | ["Shaquille O'Neal"]               |
| Damian Lillard    | ["LaMarcus Aldridge"]              |
| JaVale McGee      | ["Shaquille O'Neal"]               |
| Kevin Durant      | ["Dejounte Murray"]                |
| Kyle Anderson     | ["Dejounte Murray"]                |
| Rudy Gay          | ["LaMarcus Aldridge"]              |
| Russell Westbrook | ["Dejounte Murray"]                |


Again, let's look for LeBron James in the subgraph we just created! And find the two-step, two-way path between them, and sure enough, it only goes through `["Danny Green", "Dejounte Murray"]` and, without a direct connection.

![Tim_newFriend](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/Tim_newFriend.png)

Now, the system could send reminders to both sides: "HEY, maybe you two should make new friends!"

# Continue Exploration:

- [Pinpointing important content using common neighbors](https://www.nebula-graph.io/posts/social-networks-with-graph-database-3#post-common-neighbor)
- [Push information flow based on friend relationship and geographic location](https://www.nebula-graph.io/posts/social-networks-with-graph-database-3#post-feed-generation)
- [Spatio-temporal relationship mapping](https://www.nebula-graph.io/posts/social-networks-with-graph-database-3#post-spatio-temporal-relationship-tracking)

[![docker](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/docker-banner.png)](javascript:window.open('https://bit.ly/3XHS873'))

Identifying influencers in social networks involves using a variety of metrics and methods to identify individuals who have a lot of influence in a given network. This is useful for many business scenarios, such as for marketing or researching the spread of information in a network.

7 Fundamental Use Cases of Social Networks with NebulaGraph Database | EP 2

Two months ago, my colleague brought in a graph of Twitter interactions  and asked me if I could write a post on graph technology to explore social network analysis and see how those connections are recognized through graph technology.

![graph-database-on-social-network](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/graph-database-on-social-network-02.jpeg)

![graph-database-on-social-network-3](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/graph-database-on-social-network-03.jpeg)

So, how are typical social relationship graphs like the one above, or even more complex social network-based graphs, generated? In the following episodes, I will use the NebulaGraph database to conduct the social network analysis, and you can find step-by-step guides to try all the methods in Playground.

# Social Network Analysis on Graph Database

We can use a graph database in a social network system to represent users and their connections or relationships. Graph databases allow efficient querying of relationships between users, making various business implementations on social networks based on connection findings, statistics, and analysis feasible and efficient.

For example, graph databases can identify "influential users" in a network, recommend new connections (friendships, favorite content ) based on commonalities between users, or find different groups of people and communities to profile users. Graph databases are ideal for social networking systems where user relationships constantly change because they can support complex multi-hop queries and real-time writes and updates.

There have been quite a few examples of social network graphs practical applications, including:

- Finding key people
- Identifying clusters of people and communities
- Determining the closeness between two users
- Recommending new friends
- Pinpointing important content using common neighbors
- Push information flow based on friend relationships and geographic location
- Use spatio-temporal relationship mapping to query the relationship between people, get the people who intersected in time and space, and the provinces visited

I will dive into each application with hands-on demonstrations to present graph databases' competence in a couple of episodes. Stay tuned!

# Social Network Graph Modeling

To showcase the SNS graph use cases, I'll build most of the examples on a typically small social network. I started by adding extra data on top of the NebulaGraph default dataset, basketballplayer:

Three new tags of vertices:

- address
- place
- post

Five new types of edges:

- created_post
- commented_at
- lived_in
- belong_to

It looks like this:

![schema_sketch](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/schema_sketch.png)

# Importing the data

## Load the default dataset

- In the Command Line Console, we could execute:play basketballplayer` to load the default dataset.

- Or, if we do so from NebulaGraph Studio/Explorer, click the Download from the Demos in the welcome page:

  ![baskertballplayer_studio_starter](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/baskertballplayer_studio_starter.png)

## Add the SNS Graph schema

First, the DDL for those new tags and edges:

```sql
CREATE TAG IF NOT EXISTS post(title string NOT NULL);
CREATE EDGE created_post(post_time timestamp);
CREATE EDGE commented_at(post_time timestamp);
CREATE TAG address(address string NOT NULL, `geo_point` geography(point));
CREATE TAG place(name string NOT NULL, `geo_point` geography(point));
CREATE EDGE belong_to();
CREATE EDGE lived_in();
```

## Load the data

Then we load the DML part to insert vertices and edges:

```sql
INSERT VERTEX post(title) values \
    "post1":("a beautify flower"), "post2":("my first bike"), "post3":("I can swim"), \
    "post4":("I love you, Dad"), "post5":("I hate coriander"), "post6":("my best friend, tom"), \
    "post7":("my best friend, jerry"), "post8":("Frank, the cat"), "post9":("sushi rocks"), \
    "post10":("I love you, Mom"), "post11":("Let's have a party!");

INSERT EDGE created_post(post_time) values \
    "player100"->"post1":(timestamp("2019-01-01 00:30:06")), \
    "player111"->"post2":(timestamp("2016-11-23 10:04:50")), \
    "player101"->"post3":(timestamp("2019-11-11 10:44:06")), \
    "player103"->"post4":(timestamp("2014-12-01 20:45:11")), \
    "player102"->"post5":(timestamp("2015-03-01 00:30:06")), \
    "player104"->"post6":(timestamp("2017-09-21 23:30:06")), \
    "player125"->"post7":(timestamp("2018-01-01 00:44:23")), \
    "player106"->"post8":(timestamp("2019-01-01 00:30:06")), \
    "player117"->"post9":(timestamp("2022-01-01 22:23:30")), \
    "player108"->"post10":(timestamp("2011-01-01 10:00:30")), \
    "player100"->"post11":(timestamp("2021-11-01 11:10:30"));

INSERT EDGE commented_at(post_time) values \
    "player105"->"post1":(timestamp("2019-01-02 00:30:06")), \
    "player109"->"post1":(timestamp("2016-11-24 10:04:50")), \
    "player113"->"post3":(timestamp("2019-11-13 10:44:06")), \
    "player101"->"post4":(timestamp("2014-12-04 20:45:11")), \
    "player102"->"post1":(timestamp("2015-03-03 00:30:06")), \
    "player103"->"post1":(timestamp("2017-09-23 23:30:06")), \
    "player102"->"post7":(timestamp("2018-01-04 00:44:23")), \
    "player101"->"post8":(timestamp("2019-01-04 00:30:06")), \
    "player106"->"post9":(timestamp("2022-01-02 22:23:30")), \
    "player105"->"post10":(timestamp("2011-01-11 10:00:30")), \
    "player130"->"post1":(timestamp("2019-01-02 00:30:06")), \
    "player131"->"post2":(timestamp("2016-11-24 10:04:50")), \
    "player131"->"post3":(timestamp("2019-11-13 10:44:06")), \
    "player133"->"post4":(timestamp("2014-12-04 20:45:11")), \
    "player132"->"post5":(timestamp("2015-03-03 00:30:06")), \
    "player134"->"post6":(timestamp("2017-09-23 23:30:06")), \
    "player135"->"post7":(timestamp("2018-01-04 00:44:23")), \
    "player136"->"post8":(timestamp("2019-01-04 00:30:06")), \
    "player137"->"post9":(timestamp("2022-01-02 22:23:30")), \
    "player138"->"post10":(timestamp("2011-01-11 10:00:30")), \
    "player141"->"post1":(timestamp("2019-01-03 00:30:06")), \
    "player142"->"post2":(timestamp("2016-11-25 10:04:50")), \
    "player143"->"post3":(timestamp("2019-11-14 10:44:06")), \
    "player144"->"post4":(timestamp("2014-12-05 20:45:11")), \
    "player145"->"post5":(timestamp("2015-03-04 00:30:06")), \
    "player146"->"post6":(timestamp("2017-09-24 23:30:06")), \
    "player147"->"post7":(timestamp("2018-01-05 00:44:23")), \
    "player148"->"post8":(timestamp("2019-01-05 00:30:06")), \
    "player139"->"post9":(timestamp("2022-01-03 22:23:30")), \
    "player140"->"post10":(timestamp("2011-01-12 10:01:30")), \
    "player141"->"post1":(timestamp("2019-01-04 00:34:06")), \
    "player102"->"post2":(timestamp("2016-11-26 10:06:50")), \
    "player103"->"post3":(timestamp("2019-11-15 10:45:06")), \
    "player104"->"post4":(timestamp("2014-12-06 20:47:11")), \
    "player105"->"post5":(timestamp("2015-03-05 00:32:06")), \
    "player106"->"post6":(timestamp("2017-09-25 23:31:06")), \
    "player107"->"post7":(timestamp("2018-01-06 00:46:23")), \
    "player118"->"post8":(timestamp("2019-01-06 00:35:06")), \
    "player119"->"post9":(timestamp("2022-01-04 22:26:30")), \
    "player110"->"post10":(timestamp("2011-01-15 10:00:30")), \
    "player111"->"post1":(timestamp("2019-01-06 00:30:06")), \
    "player104"->"post11":(timestamp("2022-01-15 10:00:30")), \
    "player125"->"post11":(timestamp("2022-02-15 10:00:30")), \
    "player113"->"post11":(timestamp("2022-03-15 10:00:30")), \
    "player102"->"post11":(timestamp("2022-04-15 10:00:30")), \
    "player108"->"post11":(timestamp("2022-05-15 10:00:30"));

INSERT VERTEX `address` (`address`, `geo_point`) VALUES \
    "addr_0":("Brittany Forge Apt. 718 East Eric  WV 97881", ST_Point(1,2)),\
    "addr_1":("Richard Curve Kingstad  AZ 05660", ST_Point(3,4)),\
    "addr_2":("Schmidt Key Lake Charles  AL 36174", ST_Point(13.13,-87.65)),\
    "addr_3":("5 Joanna Key Suite 704 Frankshire  OK 03035", ST_Point(5,6)),\
    "addr_4":("1 Payne Circle Mitchellfort  LA 73053", ST_Point(7,8)),\
    "addr_5":("2 Klein Mission New Annetteton  HI 05775", ST_Point(9,10)),\
    "addr_6":("1 Vanessa Stravenue Suite 184 Baileyville  NY 46381", ST_Point(11,12)),\
    "addr_7":("John Garden Port John  LA 54602", ST_Point(13,14)),\
    "addr_8":("11 Webb Groves Tiffanyside  MN 14566", ST_Point(15,16)),\
    "addr_9":("70 Robinson Locks Suite 113 East Veronica  ND 87845", ST_Point(17,18)),\
    "addr_10":("24 Mcknight Port Apt. 028 Sarahborough  MD 38195", ST_Point(19,20)),\
    "addr_11":("0337 Mason Corner Apt. 900 Toddmouth  FL 61464", ST_Point(21,22)),\
    "addr_12":("7 Davis Station Apt. 691 Pittmanfort  HI 29746", ST_Point(23,24)),\
    "addr_13":("1 Southport Street Apt. 098 Westport  KY 85907", ST_Point(120.12,30.16)),\
    "addr_14":("Weber Unions Eddieland  MT 64619", ST_Point(25,26)),\
    "addr_15":("1 Amanda Freeway Lisaland  NJ 94933", ST_Point(27,28)),\
    "addr_16":("2 Klein HI 05775", ST_Point(9,10)),\
    "addr_17":("Schmidt Key Lake Charles AL 13617", ST_Point(13.12, -87.60)),\
    "addr_18":("Rodriguez Track East Connorfort  NC 63144", ST_Point(29,30));

INSERT VERTEX `place` (`name`, `geo_point`) VALUES \
    "WV":("West Virginia", ST_Point(1,2.5)),\
    "AZ":("Arizona", ST_Point(3,4.5)),\
    "AL":("Alabama", ST_Point(13.13,-87)),\
    "OK":("Oklahoma", ST_Point(5,6.1)),\
    "LA":("Louisiana", ST_Point(7,8.1)),\
    "HI":("Hawaii", ST_Point(9,10.1)),\
    "NY":("New York", ST_Point(11,12.1)),\
    "MN":("Minnesota", ST_Point(15,16.1)),\
    "ND":("North Dakota", ST_Point(17,18.1)),\
    "FL":("Florida", ST_Point(21,22.1)),\
    "KY":("Kentucky", ST_Point(120.12,30)),\
    "MT":("Montana", ST_Point(25,26.1)),\
    "NJ":("New Jersey", ST_Point(27,28.1)),\
    "NC":("North Carolina", ST_Point(29,30.1));

INSERT EDGE `belong_to`() VALUES \
    "addr_0"->"WV":(),\
    "addr_1"->"AZ":(),\
    "addr_2"->"AL":(),\
    "addr_3"->"OK":(),\
    "addr_4"->"LA":(),\
    "addr_5"->"HI":(),\
    "addr_6"->"NY":(),\
    "addr_7"->"LA":(),\
    "addr_8"->"MN":(),\
    "addr_9"->"ND":(),\
    "addr_10"->"MD":(),\
    "addr_11"->"FL":(),\
    "addr_12"->"HI":(),\
    "addr_13"->"KY":(),\
    "addr_14"->"MT":(),\
    "addr_15"->"NJ":(),\
    "addr_16"->"HI":(),\
    "addr_17"->"AL":(),\
    "addr_18"->"NC":();

INSERT EDGE `lived_in`() VALUES \
    "player100"->"addr_4":(),\
    "player101"->"addr_7":(),\
    "player102"->"addr_2":(),\
    "player103"->"addr_3":(),\
    "player104"->"addr_0":(),\
    "player105"->"addr_5":(),\
    "player106"->"addr_6":(),\
    "player107"->"addr_1":(),\
    "player108"->"addr_8":(),\
    "player109"->"addr_9":(),\
    "player110"->"addr_10":(),\
    "player111"->"addr_11":(),\
    "player112"->"addr_12":(),\
    "player113"->"addr_13":(),\
    "player114"->"addr_14":(),\
    "player115"->"addr_15":(),\
    "player116"->"addr_16":(),\
    "player117"->"addr_17":(),\
    "player118"->"addr_18":();
```

## First glance at the data

Let's start with the stats of the data.

```sql
[basketballplayer]> SUBMIT JOB STATS;

+------------+
| New Job Id |
+------------+
| 10         |
+------------+
[basketballplayer]> SHOW STATS;
+---------+----------------+-------+
| Type    | Name           | Count |
+---------+----------------+-------+
| "Tag"   | "address"      | 19    |
| "Tag"   | "place"        | 14    |
| "Tag"   | "player"       | 51    |
| "Tag"   | "post"         | 10    |
| "Tag"   | "team"         | 30    |
| "Edge"  | "belong_to"    | 19    |
| "Edge"  | "commented_at" | 40    |
| "Edge"  | "created_post" | 10    |
| "Edge"  | "follow"       | 81    |
| "Edge"  | "lived_in"     | 19    |
| "Edge"  | "serve"        | 152   |
| "Space" | "vertices"     | 124   |
| "Space" | "edges"        | 321   |
+---------+----------------+-------+
Got 13 rows (time spent 1038/51372 us)
```

We could get all of the data:

```sql
MATCH ()-[e]->() RETURN e LIMIT 10000
```

As the data volume is quite small, we could render them all in the canvas of NebulaGraph Explorer:

![match_all](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-sns/match_all.png)

Above are the two fundamental steps to embark on the journey of social network exploration. I'll introduce other applications in the next episode:

- [Finding key people](https://www.nebula-graph.io/posts/social-networks-with-graph-database-2#post-identifying-key-people)
- [Identifying clusters of people, communities](https://www.nebula-graph.io/posts/social-networks-with-graph-database-2#post-detect-communities-and-clusters)
- [Determining the closeness between two users](https://www.nebula-graph.io/posts/social-networks-with-graph-database-2#post-friend-closeness)
- [Recommending new friends](https://www.nebula-graph.io/posts/social-networks-with-graph-database-2#post-new-friend-recommendation)
- [Pinpointing important content using common neighbors](https://www.nebula-graph.io/posts/social-networks-with-graph-database-3#post-common-neighbor)
- [Push information flow based on friend relationship and geographic location](https://www.nebula-graph.io/posts/social-networks-with-graph-database-3#post-feed-generation)
- [Spatio-temporal relationship mapping](https://www.nebula-graph.io/posts/social-networks-with-graph-database-3#post-spatio-temporal-relationship-tracking)

[![docker](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/docker-banner.png)](javascript:window.open('https://bit.ly/3XHS873'))


We can use a graph database in a social network system to represent users and their connections or relationships. Graph databases allow efficient querying of relationships between users, making various business implementations on social networks based on connection findings, statistics, and analysis feasible and efficient.

7 Fundamental Use Cases of Social Networks with NebulaGraph Database | EP 1

Qingteng Cloud Security is a host security unicorn company, which focuses on the security of cloud native applications. At present, its main business is the host product named Wanxiang and the container product named Honeycomb. It leads the industry and supports 8 million agents in total. The company's next-generation real-time intrusion detection system based on NebulaGraph and combined with graph technology has been initially put into the market and participated in the 2022 Network Protection, which has achieved good response.

In this topic, we will briefly introduce the intrusion detection system based on graph, hoping to have more people to join in the mining of graph and security application combination.

# The status and challenge of intrusion detection

## Mainstream intrusion detection system

Intrusion detection has always been a major direction of security research. The two sets of products of Qingteng, Wanxiang and Honeycomb, are host-based and container-based intrusion detection products, both of which have similar principles, as shown below.

![Intrusion detection system architecture diagram](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/intrusion-detection/intrusion-detection-1.png)

The agent is placed on the host and container side, receiving rules from the server side, combining the collected raw events (process/network connection/file read and write, etc.), and reporting them to the server side as alarms after hitting the rules written by security experts, such as the MD5 of the process file, execution command line, machine learning, and other features. The full amount of raw events are also reported to the server side log collection system and stored in the SIEM system.

Once the alarm is reported to the server, the security staff will take the information related to the alarm to SIEM system to check the events before and after the time of occurrence, log in to the relevant machine to check the relevant information if necessary, and make a comprehensive judgment on whether the current attack alarm is valid and do the relevant disposal.

This system is the current mainstream architecture of intrusion detection, but there are many problems:

1. For single-point detection based on single events and rules, it may result in false positives due to too loose rules or missed positives due to too strict rules.
2. Too many alarms may be triggered by the same attack, and security staff have a heavy workload in tracing the source.
3. The original events related to the same alarm need to be manually analyzed by the SIEM system, which lacks visual means.

## The research direction of graph and security

These questions essentially look at each alarm and event independently, when in fact, the alarms and events associated with an attack should be related to each other. We can use graphs to correlate these original events and alarms for overall analysis. In fact, this is also a popular direction of current security research -- traceability graph, with the help of traceability graph we do the following security analysis and detection.

### 1.Graph detection

Traditional security solutions solve similar IOC detection problems, that is, the judgment of single-point, for the process, network connections, files and other entities is not safe. The actual problem faced by the attack, may not be detected at all points, but the actual behavior is dangerous. A single rule is not fully written, even if the rule can be written, such as hitting a command line, but it will trigger many alarms, and eventually the rule cannot be used. We need to comprehensively judge various combination relationships, which is what graph is good at, and also the problem of graph detection in security. Put all related events into one graph to comprehensively judge whether they are valid attacks.

**The research of graph detection started early, but faced with the dual challenges of computational volume and algorithm, the current industry applications are few, and the combined judgments are all in the way of sequence detection**. But the sequence rules can not write too many rules in the rule list, otherwise it will lead to the inability to apply the rules. In addition, if the rules are too detailed, and can not be well fuzzy matching rules, it is easy to be bypassed. Hackers can easily bypass these rules with the help of automation tools.

### 2.Graph association and tracing

Just mentioned alarms, which is another problem all the security products facing today, either the rules do not hit, or too many rule hits, we call this situation **alarm flooding**. Security staff processing capacity is limited, maybe 100 alarms a day is fine, if a day or even an hour 10000 alarms can not be handled, it is no different from no alarms.

This is really a problem of correlation classification and traceability in security:

1. An attack will generate a lot of alarms, such as brute force login, used malicious files, executed malicious commands, etc.. The correlation classification to correlate the related alarms in an attack together, which is something the graph is good at. These alarms also allow a comprehensive assessment of whether the current attack is valid or not.
2. Each alarm will only tell you is a process, file, or network problems, but how this problem occurred, how the hacker came in, where the file was downloaded from, what was done first and then what was done. Security products need to help customers complete this analysis process, the industry is currently with the help of SIEM, THP, SOC and other security products, all the original events are uploaded and saved, and then find security experts, starting from the alarm to check the original event logs, to see what happened before the alarm, which has a relationship may lead to an attack, the process is as short as a few minutes, long as several hours. **Security is a confrontation process, early is better**, the sooner it is found the sooner it is confronted, blocked or isolated, otherwise it is too late even if it is found. The relevant original event entities (process/network, etc.) can be imported into the graph, with the help of the graph to visually explore and trace the entire attack process, which is the process of graph traceability, academically called causal graph, traceability graph.

### 3.Graph knowledge mapping and prediction

We know that **current security is fundamentally based on rules or a priori knowledge**, each vulnerability, Trojan horse, attack tools, attack process, attack organizations have its characteristics, the first few rules are still relatively good description, the attack process and attack organizations are difficult to complete the description.

The current mainstream approach is to build a knowledge base based on security frameworks, the current mainstream has Kill Chain, ATTCK and other frameworks, which is the U.S. Department of Defense-led cyber attack war related to the two companies proposed a security analysis framework, equivalent to the mapping of the tactics and specific attack techniques that delineate the attack. Security academia, such as the University of Illinois and Purdue University have been working on similar issues in the last two years, which is the construction of a security knowledge base (security knowledge mapping). With a complete knowledge base, the ultimate vision of security can be accomplished, for example, if I know your attack process and attack organization, is it possible to stop an attack before it actually starts.

## Domestic and international status

At present, the intrusion detection system based on graph that is really taking the lead in the real world is Crowd Strike, a star security company in the U.S. It builds its security system entirely on graphs, which now amounts to doing both graph detection and graph correlation traceability, and is currently valued at 67 billion. Cloud computing giants AWS and Azure are following its approach. Domestic intrusion detection system based on graph, currently there are public information is ThreatBook and Sangfor, they are equivalent to do part of the work of graph correlation and traceability.

Qingteng Cloud Security's Wanxiang and Honeycomb have been working on intrusion detection for many years and have achieved industry-recognized security detection capabilities, so we chose to start with graph association and traceability, and develop a next-generation real-time intrusion detection system based on NebulaGraph combined with graph technology, focusing first on solving the pain point problems of alarm flooding and association traceability.

# Qingteng cloud security next-generation intrusion detection system

## Detection principle architecture diagram

As shown below, the focus is the reported attack alerts and some of the original events are unified and correlated in real time in the graph.

![Detection principle architecture diagram](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/intrusion-detection/intrusion-detection-2.png)

## Product effect

The output after correlation processing by the correlation engine is the attack event, an attack event may be associated with multiple attack alerts and visualized, the current product effect is as follows.

- The same attack using malicious files and trojans

  ![The same attack using malicious files and trojans](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/intrusion-detection/intrusion-detection-architecture-03.png)

- Mounting of sensitive containers

  ![Mounting of sensitive containers](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/intrusion-detection/intrusion-detection-architecture-04.png)

- Custom script detection

  ![Custom script detection](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/intrusion-detection/intrusion-detection-5.png)

## Advantages of NebulaGraph

NebulaGraph was chosen based on the following considerations.

1. The immediate need for graph queries, especially for multi-level process relationships during association and traceability. It is much easier to query a graph database with cypher than to query multi-level relationships in a relational database with SQL.
2. Large-scale storage, involving the storage of a large number of events and alarms.
3. High-performance query scenarios, associations need to ensure near real-time, the current relevant queries are in ms level.

Thanks to the good performance of NebulaGraph, the association engine enters the graph and calculates the association in a near real-time manner. The tool for importing graph data is the real-time graph entry component created by Qingteng based on Flink, which only needs to change the configuration file to complete the map of points and edges.

# Next research direction and plan

At present, the intrusion detection system of Qingteng cloud security mainly supports the association and traceability between single machine and some multi-machine scenarios. The next step is to support more scenarios of multi-machine association with the help of the graph, especially some typical security attack scenarios (rebound/horizontal movement), so as to further enable security with the graph and provide better service for customers.

There are many problems in applying NebulaGraph to graph security applications at present. Some suggestions are given for reference.

## 1.Push down optimization

In our process of use, the most important problem is the optimization of push down. Such as a diffuse process tree, while specifying the root node and leaf nodes, the order of statement writing may lead to query from the leaf node or the root node, query efficiency varies widely, and finally can only specify the leaf node conditions to force the query from the leaf node.

The performance of MATCH is also essentially a push-down optimization problem, this problem is also encountered a lot, our current approach is to maintain a cache pool for some of the statements have been queried, also note that the official enterprise version actually provides a partial cache function, this point is good if you can consider adding to the community version, is a practical application just need.

## 2.Some technical requirements for real-time scenarios

Since we are doing real-time association based on NebulaGraph, one of the major problems is how to achieve a balance between consistency and speed, because currently NebulaGraph cannot actually determine whether the graph entry is complete after writing to it. In practice, the association engine needs to poll downstream to determine whether the current vertices and edges is actually in the graph. This problem has some impact on the actual use and performance, and it is worth considering whether there is a better way.

We can see that some competing products, such as TigerGraph and TuGraph, provide the ability to customize the matching and traversal algorithms. In our multi-host association scenario, we have encountered such a need. At present, we can only achieve it by splitting multiple paths and splicing them together. Efficiency and speed are definitely compromised.

## 3.ToB deployment

Another pain point is the ToB deployment of NebulaGraph. In the security industry, there are still more on-site deployments at home. Especially, state-owned enterprises and government affairs are relatively sensitive to SaaS, so they will face the problem of efficiency and cost.

**The first is single-machine deployment**. The default production version is a distributed deployment with separate storage and computing designed for the cloud, which is still too cumbersome for ToB deployments and consumes relatively more resources than other databases.

**The second is HDD deployment**. Considering the cost, many companies, especially small and medium-sized companies, have limited security budgets, which makes it difficult to provide SSD deployment, not to mention the large capacity and high configuration of SSD machines required by the official website. Therefore, more optimization considerations should be made for HDD and low configuration machines.

Above, for Qingteng cloud security team engineer Zhou Wen to share.

**Thank you for reading this article.**

Want to experience the graph database up close? Now you can use NebulaGraph Cloud on AWS to build your own graph data system, come save a lot of deployment and installation time to get your business done [click the link](https://go.aws/3VZay2I) to use the graph database!

If you want to see the source code, you can go to GitHub to read, use, star it -> [GitHub](https://bit.ly/3f7XHex). Exchange graph database technology and application skills with other NebulaGraph users, and join ["Slack"](https://bit.ly/3SmvFu2) to play together!

Intrusion detection has always been a major direction of security research. In this topic, we will briefly introduce the intrusion detection system based on graph, hoping to have more people to join in the mining of graph and security application combination.

Next-generation Intrusion Detection System Based on Graph

# Background

Traditional risk control mainly relies on manual review and empirical judgment. With the rise of digitalization in the past decade, leading companies began to introduce blacklisting systems to locate the risk points missed in the review stage through simple rules and post-facto audits, but this still cannot detect problems in advance. Since 2018, the anti-fraud system has gradually evolved to the stage of "intelligent decision engine" consisting of rule engines, real-time metric calculation, and machine learning technologies.

## Insurance Fraud Control Pain Points

The pain points of insurance fraud control are mainly reflected in the following aspects:

**Non-exchange information leads to poor risk control**: Data of insurance companies and the related industry is non-exchangeable. Even the data between various systems within the same insurance company is unshared. This can lead to asymmetry in business information and false claims, or the use of information asymmetry to increase claim amounts.

**Requirements of multi-dimensional computing**: Industry fraud diversity, specialization, and grouping will force the anti-fraud system to analyze data from multi-dimensional aspects, while the traditional analysis system based on relational models, dimensional models, or wide table models hardly analyzes data from dozens or even more dimensions.

**Accuracy requirements of algorithm recognition**: Business model tuning requires more feature data input during the training and serving stages. Generally, there are two methods to input data, one is writing more data, and the other is the supplementing of correlation data. However, if we go with the traditional method, it will be difficult to detect abnormal data with more than 4-hop queries, and the query latency is high.

## Underlying Storage Challenges


With the development of the anti-fraud system, the system faces challenges in the aspect of the underlying storage. 

The performance of traditional stand-alone databases has bottlenecks in capacity and query for linked data. In addition, business iterations and peaks and valleys have agility requirements for database scalability. Nowadays, most risk control systems use Hadoop ecosystem tools to co-accomplish system goals, and the corresponding development and operation and maintenance costs are high. In terms of real-time data analysis, it's difficult for traditional big data platforms to perform the analysis of multiple sources of data.


# Use Scenarios

NebulaGraph is used in many scenarios of insurance fraud detection. The following are 5 typical scenarios.


## Scenario 1: Relationship Detection


Customers may skip the family information in the application form. Such uncertain or missing family information are not well determined by the original method. While the graph data structure allows us to find relatives, which is not natively supported by other types of databases.

In addition, the regular form is filled with only immediate relatives, which does not reflect indirect relationships. In NebulaGraph's graph database, more hidden relationships can be discovered through the available data, including social and non-immediate family relationships. It allows for a comprehensive analysis of the family's insurance purchases.

## Scenario 2: Fraud Detection


Traditional fraud detection methods are mainly based on empirical rules and are unable to detect fraud in advance.

Taking car insurance as an example, there are relatively more entities involved in car insurance, including vehicles, owners, drivers, passengers, etc. It is very difficult to identify those entities among these data. However, in a graph database, it is possible to build a graph to see some outliers based on the number of claims or other business indicators, and it is also possible to classify groups with higher fraud rates through graph algorithms. In social relationship networks, the fraud possibility can be determined by the correlation with the blacklisted spots. In general, we can make full use of pattern matching and other graph database technologies to classify the major entities by using graph algorithms and then identify groups or individuals with high fraud rates based on business experience and existing fraud cases.


Fraud cases usually have certain characteristics. For example, a luxury used car with frequent accidents or multiple accident-related people driving a different vehicle frequently, or a high correlation between different vehicle groups and multiple accident-related owners. These are all suspicious characteristics that can be identified quickly and effectively by graph technology.

## Scenario 3: Precision Marketing 

In the scenarios of marketing, the data in a graph database can help identify target customers in depth. Based on the information left by customers, we can find social relationships, friends, and non-immediate family members other than immediate family members. Such relationship networks help us see a larger dimension of the customer's social circle. We can find more hidden relationships in the connections between customers and their friends. This can help us learn more about family insurance purchase behaviors.


In the private domain scenario, the marketing agents can recommend other insurance products based on the customer's purchase history. The groups with associated relationships may tend to prefer the same insurance products. From another perspective, such associated recommendations will be more convincing to customers. In the public domain scenario of getting target customers, graph databases can also help advertise to the target customers through the customer data system.

## Scenario 4: Insurance Underwriting and Claims Fraud Detection


At the underwriting and claim stages, we can find what insurance policies the insured has, the number of claims denied, and the association with known fraud cases by querying the in and out-degree of the insured. As for car insurance, we can detect fraud from "the case" to the "vehicle", to the "owner", to the "driver", and to the "vehicle", as shown in the above figure. In this way, we can find the fraud cases from the root cause. For example, in a DUI transfer case (the drunk driver A contacts his relative B to arrive at the scene and report to the insurance company in place of A to get insurance compensation after the DUI accident), we can analyze behaviors such as outgoing and incoming calls, and the location information of the involver. Through this information, multiple risk factors may be detected such as delayed reporting, non-frequently used cell phone numbers and distance from the scene.

## Scenario 5: Agents Management


A large number of domestic insurance agents and a high turnover rate may lead to difficulty in agent management. 

We can also build the agent's relationship network based on the graph model. The agent's professional habits and complaints received can be labeled in the graph to enhance the screening of agents from the source.

 For some underground insurance scenarios, such as forging the identity of a newcomer for commission. Fraud risks can be identified through cell phone numbers, electronic device numbers, IP associations, etc., all of which can effectively reduce customer complaints and insurance frauds caused by agents.



# Why NebulaGraph?

## Features of NebulaGraph

![NebulaGraph-Database](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/nebulagraph-ecosystem.png)


To address the above anti-fraud pain points and the need for the system-level construct, NebulaGraph has the following features that are suitable for the assurance risk control needs:

1. **Multi-source data fusion**: The sources of insurance data are diverse, including bank data, social network data, government data, medical data, marketing data, and so on. The data fusion of these data sources is a key to the success of the fraud detection system that can establish a comprehensive analysis of customers to better help insurance companies achieve automatic and accurate underwriting.

2. **Support of graph data model**: The construct of an underlying data model is the key to the multi-dimensional analysis. NebulaGraph data models fit the storage and query presentation requirements of linked data and provide quicker analysis compared to the relational model while its schema is flexible.

3. **Cloud-native architecture**: A good storage system should enable upper-tier applications to compute and scale horizontally based on business needs. NebulaGraph is a distributed cloud native graph database. It can be easily scaled out and scaled in to meet the needs of business growth and shrinkage.

4. **Enterprise-level feature support**: NebulaGraph, as a basic software, provides the support of peripheral tools for a better exploration and visual operation and maintenance.

5. **Risk factor detection**: NebulaGraph's association analysis and graph computing engines can help better identify the risk factors with algorithms.

6. **Accuracy of auxiliary algorithms**: Machine learning relies on input data constructed by tuples, which tend to ignore the relationships between data. So the input of graph data can be used to enrich the contextual information and thus improve the accuracy of relevant models.

In a word, NebulaGraph connects dimensional data. It can be used in different scenarios to help insurance companies effectively identify and avoid hidden risks at different stages. For details about how to deploy and use NebulaGraph, please refer to [live demos](https://www.nebula-graph.io/demo/).  

NebulaGraph can be used in different scenarios to help insurance companies effectively identify and avoid hidden risks at different stages. 

Insurance Fraud Detection with Graph Technology | A Practice of NebulaGraph

> An attempt to use ChatGPT to generate code for a data scraper to predict sports events with the help of the NebulaGraph graph database and graph algorithms.


#  The Hype of ChatGPT

In the hype for FIFA 2022 World Cup, I saw [a blog post from Cambridge Intelligence](https://cambridge-intelligence.com/fifa-world-cup-2022-prediction/), in which they leveraged limited information and correlations among players, teams, and clubs to predict the final winning team. As a graph technology enthusiast, I always would like to try similar things with NebulaGraph to share the ideas of graph algorithms to extract hidden information from the overall connections in a graph in the community.

The initial attempt was to complete it in about 2 hours, but I noticed the dataset needs to be parsed carefully from [Wikipedia](https://en.wikipedia.org/wiki/2022_FIFA_World_Cup_squads) and I happened to be not good at doing this job, so I put the idea on hold for a couple of days.

In the meantime, another craze, the OpenAI ChatGPT, was announced. As I had been a user of DALL-E 2 already(to generate feature images of my blog posts), I also gave it a try very quickly. And I witnessed how other guys(via Twitter, blogs, HackerNews) tried to convince ChatGPT to do so many things that are hard to believe they could do:

- Help to implement a piece of code at any time
- Simulate any prompt interface: shell, python, virtual machine, or even a language you create
- Act out almost any given persona, and chat with you
- Write poetry, rap, prose
- Find a bug in a piece of code
- Explain the meaning of a complex regular expression/Open Cypher Query

ChatGPT's ability to contextualize and understand has never been greater before, so much so that everyone is talking about a new way of working: how to master asking/convincing/triggering machines to help us do our jobs better and faster.

So, after trying to get ChatGPT to help me write complex graph database query statements, explain the meaning of complex graph query statements, and a large chunk of Bison code, which he/she had done them WELL, I realized: why not let ChatGPT write the code that extracts the data for me?

# Grabbing data

I really tried it and the result is... good enough.

The whole process was basically like a coding interviewer, or a product manager, presenting my requirements, then ChatGPT giving me the code implementation. I then try to run the code, find the things that don't make sense in the code, point them out, and give suggestions, and ChatGPT quickly understands the points I point out and makes the appropriate corrections, like:

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/chatGPT-correction-process.png)

I won't list this whole process, but I share the generated code and the whole discussion [here](https://gist.github.com/wey-gu/78cb28bee130966e7d6e9d573b51deff).

The final generated data is a CSV file.

- Raw version [world_cup_squads.csv](https://github.com/siwei-io/talks/files/10152775/world_cup_squads.csv)

- Manually modified, separated columns for birthday and age [world_cup_squads_v0.csv](https://github.com/siwei-io/talks/files/10152923/world_cup_squads.csv)

  It contains information/columns of a team, group, number, position, player name, birthday, age, number of international matches played, number of goals scored, and club served.

```CSV
Team,Group,No.,Pos.,Player,DOB,Age,Caps,Goals,Club
Ecuador,A,1,1GK,Hernán Galíndez,(1987-03-30)30 March 1987,35,12,0,Aucas
Ecuador,A,2,2DF,Félix Torres,(1997-01-11)11 January 1997,25,17,2,Santos Laguna
Ecuador,A,3,2DF,Piero Hincapié,(2002-01-09)9 January 2002,20,21,1,Bayer Leverkusen
Ecuador,A,4,2DF,Robert Arboleda,(1991-10-22)22 October 1991,31,33,2,São Paulo
Ecuador,A,5,3MF,José Cifuentes,(1999-03-12)12 March 1999,23,11,0,Los Angeles FC
```

- Final version with header removed [world_cup_squads_no_headers.csv](https://github.com/siwei-io/talks/files/10152974/world_cup_squads_no_headers.csv)



# Graph algorithm to predict the FIFA World Cup 2022

With the help of ChatGPT, I could finally try to predict the winner of the game with Graph Magic, before that, I need to map the data into the graph view.

If you don't care about the process, go to the [predicted result](https://www.nebula-graph.io/posts/predict-fifa-world-cup-with-chatgpt-and-nebulagraph#post-result) directly.



## Graph modeling

> Prerequisites: This article uses [NebulaGraph](https://github.com/vesoft-inc/nebula)(Open-Source) and [NebulaGraph Explorer](https://docs.nebula-graph.io/3.3.0/nebula-explorer/about-explorer/ex-ug-what-is-explorer/)(Proprietary), which you can request a free trial of on [AWS](https://go.aws/3VZay2I).

Graph Modeling is the abstraction and representation of real-world information in the form of a "vertex-> edge" graph. In our case, we will project the information parsed from Wikipedia as:

**Vertices**:

- player
- team
- group
- club

**Edges**:

- groupedin (the team belongs to which group)
- belongto (players belong to the national team)
- serve (players serve in the club)

The age of the players, the number of international caps, and the number of goals scored are naturally fit as properties for the player tag(type of vertex).

The following is a screenshot of this schema in **NebulaGraph Explorer** (I will just call it Explorer later).

![schema_fifa](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/schema_fifa.png)

Then, we can click the save icon in the upper right corner and the button: `Apply to Space`  to create a graph space with the [defined schema]( https://docs.nebula-graph.io/3.3.0/nebula-explorer/db-management/draft/):



## Ingesting into NebulaGraph

With the graph modeling, we can upload the [CSV file](https://github.com/siwei-io/talks/files/10152974/world_cup_squads_no_headers.csv) (the no-header version) into Explorer, by pointing and selecting the vid and properties that map the different columns to the vertices and edges.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/importer_config_mapping.png)

Click Import, we then import the whole graph to NebulaGraph, and after it succeeded, we could also get the entire CSV --> Nebula Importer configuration file: [nebula_importer_config_fifa.yml](https://github.com/siwei-io/talks/files/10164014/config_fifa.yml.txt), so that you reuse it in the future whenever to re-import the same data or share it with others.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/importer_log.png)

> Note: Refer to the [Import Data Document]( https://docs.nebula-graph.io/3.3.0/nebula-explorer/db-management/11.import-data/)

After importing, we can view the [statistics on the schema view](https://docs.nebula-graph.io/3.3.0/nebula-explorer/db-management/10.create-schema/#view_statistics) page, showing us that 831 players participated in the 2022 Qatar World Cup, serving in 295 different clubs.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/data_stats.png)


## Explore the graph

Let's see what insights we could get from the information/ knowledge in the form of a graph.

### Querying the data

Let's start by showing all the data and see what we will get.

First, with the help of NebulaGraph Explorer, I simply did drag and drop to draw any type of vertex type (TAG) and any type of edge between vertex types (TAG), here we know that all the vertices are connected with others, so no isolated vertices will be missed by this query pattern:

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/query-builder-0.png)

Let it generate the query statement for me. Here, it defaults to `LIMIT 100`, so let's change it to something larger (LIMIT 10000) and let it execute in the Console.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/query-builder-1.png)

### Initial observation

The result renders out like this, and you can see that it naturally forms a pattern of clusters.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/bird_view.png)

These peripheral clusters are mostly made up of players from clubs that are not traditionally strong ones (now we learned that they could win, though, who knows!). Many of those clubs have only one or two players who serve in one national team or region, so they are somewhat isolated from other clusters.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/edge_teams.png)

### Graph algorithm-based analysis

After I clicked on the two buttons(Sized by Degrees, Colored by Louvain Algorithm) in Explorer (refer to the [document](https://docs.nebula-graph.com.cn/3.3.0/nebula-explorer/graph-explorer/graph-algorithm/) for details), in the browser, we can see that the entire graph has become something like this:

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/Barcelona.png)

Two graph algorithms are utilized to analyze the insights here.

1. change the display size of vertices to highlight importance using their degrees
2. using Louvain's algorithm to distinguish the community of the vertices

You can see that the big red circle is the famous Barcelona, and its players are marked in red, too.

## Winner Prediction Algorithm

In order to be able to make full use of the graph magic(with the implied conditions and information on the graph), my idea([stolen/inspired from this post](https://cambridge-intelligence.com/fifa-world-cup-2022-prediction/)) is to choose a graph algorithm that considers edges for node importance analysis, to find out the vertices that have higher importance, iterate and rank them globally, and thus get the top team rankings.

These methods actually reflect the fact that excellent players have greater community and connectivity at the same time. Meanwhile, to increase the differentiation between traditionally strong teams, I am going to take into account the information on appearances and goals scored.

Ultimately, my algorithm is:

- Take all the `(player)-serve->(club)` relationships and filter them for players with too few goals and too few goals per game (to balance out the disproportionate impact of older players from some weaker teams)
- Explore outwards from all filtered players to get national teams
- Run the [Betweenness Centrality](https://en.wikipedia.org/wiki/Betweenness_centrality) algorithm on the above subgraph to calculate the node importance scores


## Process of the Prediction

First, we take out the subgraph in the pattern of `(player)-serve->(club)` for those who have scored more than ten goals and have an average of more than 0.2 goals per game.

```cypher
MATCH ()-[e]->()
WITH e LIMIT 10000
WITH e AS e WHERE e.goals > 10 AND toFloat(e.goals)/e.caps > 0.2
RETURN e
```

> Note: For convenience, I have included the number of goals and caps as properties in the serve edge, too.

![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/query_step0.png)

Then, we select all the vertices on the graph in the left toolbar, select the `belongto` edge of the outgoing direction, expand the graph outwards (traverse), and select the icon that marks the newly expanded vertices as flags.

![treversal_step1](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/treversal_step1.png)

Now that we have the final subgraph, we use the graph algorithm function within the browser to execute BNC (Betweenness Centrality):

![bnc_step2](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/bnc_step2.png)

The graph canvas then looks like this:

![bnc_predict](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/OpenGPT%20FIFA/bnc_predict.png)

# Result

Ultimately, we sorted according to the value of Betweenness Centrality to get the final winning team: Brazil! 🇧🇷, followed by Belgium, Germany, England, France, and Argentina, so let's wait two weeks to come back and see if the prediction is accurate :D.

The sorted data is as follows:

| Vertex                    | **Betweenness Centrality** |
| ------------------------- | -------------------------- |
| **Brazil**🇧🇷              | 3499                       |
| **Paris Saint-Germain**   | 3073.3333333333300         |
| **Neymar**                | 3000                       |
| **Tottenham Hotspur**     | 2740                       |
| **Belgium**🇧🇪             | 2587.833333333330          |
| **Richarlison**           | 2541                       |
| **Kevin De Bruyne**       | 2184                       |
| **Manchester City**       | 2125                       |
| **İlkay Gündoğan**        | 2064                       |
| **Germany**🇩🇪             | 2046                       |
| **Harry Kane (captain**   | 1869                       |
| **England**🏴󠁧󠁢󠁥󠁮󠁧󠁿              | 1864                       |
| **France**🇫🇷              | 1858.6666666666700         |
| **Argentina**🇦🇷           | 1834.6666666666700         |
| **Bayern Munich**         | 1567                       |
| **Kylian Mbappé**         | 1535.3333333333300         |
| **Lionel Messi (captain** | 1535.3333333333300         |
| **Gabriel Jesus**         | 1344                       |

> Feature Image Credit: The image was also generated with OpenAI, through the DALL-E 2 model & DALL-E 2 Outpainting. See the [original image](https://user-images.githubusercontent.com/1651790/205881462-ff007725-e270-4b1e-9062-7702f01021c1.png).


An attempt to use ChatGPT to generate code for a data scraper to predict sports events with the help of the NebulaGraph graph database and graph algorithms.

Use ChatGPT and NebulaGraph Database to Predict FIFA World Cup 2022

How could we model data in Tabular sources and ETL it to NebulaGraph? This article demonstrates an end-to-end example of doing so with dbt.

Graph Data Modeling and ETL with dbt in NebulaGraph Database

# Financial Fraud Detection: One of the Best Practices of Knowledge Graph

Hello, I'm Qingxin Kong from Bangsun Technology. I'm glad that I have this opportunity to share the topic **Financial Fraud Detection: Best Practices of Knowledge Graph **with NebulaGraph community members.

### Background: Problems with traditional fraud detection solutions

Financial frauds remain a hot topic in recent years. There's a new trend emerging that fraudsters are organized and ganged up on. It is difficult to verify whether the customer's information provided is fake or not. Such inconsistent information makes it difficult to identify the real frauds and leads to a lot of problems. The bank may control the passing rate by raising the requirements to handle loan applications due to the uncertainty of the customer's information, which is effective but brings losses.

The 2022 government work report said that financial inclusion will be further expanded and the customer base will include rural customers, which means there will be "white accounts" having no credit reports. Traditional fraudulent behaviors include fake credit reports, commission agencies, and collusion. 

Some agencies focus on a few banks for credit card applications or loans. Those agencies know the bottom line and red line of the banks. As long as the bottom line is not touched, they can successfully apply for loans. Those agencies may even know how to make the loan amount higher.

Traditional fraud detection solutions rely heavily on expert rules, that is, risk control experts set a batch of rules through experience and then adjust the trigger conditions to find a business balance between the pass rate and rejection rate. It costs little to forge things like ID numbers and bank card flows. There are no good means of control for mass and high-volume frauds.

### Empowering graphs: Paths to intelligent fraud detection solutions
![](http://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/financial-fraud-detection-one-of-the-best-practices-of-knowledge-graph/figure1.png)

Let's take a look at the development path of intelligent fraud detection solutions. 

The first step is to build expert rules, but it is difficult to cover all risks through expert experience. The second step is to build a [machine learning](https://www.nebula-graph.io/posts/fraud-detection-with-graph-machine-learning) model to prevent and control the overall risk and then use expert rules to help the model. Expert rules and machine learning models well complement each other. However, based on the current experience, we may face lots of problems when building the machine learning model. The first reason is insufficient data and the second is the limited growth of the sample data. Because there are probably be 6~7 ways to apply for loans or credit cards, manual inspection is required in the end, so the data growth is very low. There will be a case below to explain the problem.

Therefore, we need to use relational graphs to crack the problems of information inconsistency. Both group and agency frauds have a batch of application behaviors. We can use the visualized graphs to recognize fraud samples. In this way, we may capture dozens to hundreds of fraudulent applications. It is also a very effective way for sample replenishment. Graphs can also be used for consistency checks. The multiple sources of information prove the information inconsistency. Graphs are displayed from the dimension of relations which helps us to do secondary verification. For example, multiple applications sharing the same information like the same phone number and IP address are considered group fraud.

To detect these group frauds, we use graphs that help us find rules of relations. These rules are complementary to the original expert rules. On the other hand, we can extract graph information to optimize machine learning patterns which increases the pattern's ability to detect group frauds and label more suspicious objects. Therefore, the expert rules, machine learning models, and graphs construct an effective ecological closed loop and complement each other to detect frauds. 

In terms of the specific applications of relational graphs, I've listed the following applications:

**_Application Graph_**: An example is using graphs to detect fraudulent applicants for loans and credit cards. One characteristic of a group is that information is shared by the group members. From the visual queries of graph data, a lot of correlations will be found. The more common information includes identity card, cell phone number, device fingerprint, e-mail address, etc. We use this information to explore relationships among these data, and then visually analyze the specific fraudulent techniques.**_‍_**

**_Transaction Graph_**: For multi-level transfer relationships, we mainly focus on the final flow of funds and who are the beneficiaries. The recent gambling fraud issue is the main focus of concern for financial supervisors, which is also one of the main directions we focus on. We have many customer cases. Through the list of suspects provided by the supervision of our internal rule models, suspicious transactions can be detected. In this process, we use historical flow data to explore more layers of data starting with the collected accounts. We may also use the graph exploration feature or a graph platform to analyze existing data for incremental prevention and control. 

**_Enterprise Graph/Internal Control Graph_**: Nowadays, there are interconnections between two or more enterprise businesses. The impact of external risks on the enterprise is increasing. Using graphs can outline the whole picture of risks for an enterprise. Combined with the introduction of external risks, the impact of external risks on the enterprise can be predicted as early as possible. Internal control graphs are more often used for detecting ethical or operational risks, such as misuses or failure to do certain operations in accordance with rules and regulations. Moral risks occur more often in unusual financial transactions between employees and enterprises, such as illegal fundraising or misappropriation of funds. The prevention core is to use graphs to check the actual control accounts of employees. Because bank employees are well acquainted with the bank's business. Few employees take their own accounts for some crimes. The identification of these control accounts can be explored by the means of graphs. For example, the accounts of relatives. We have a case of a joint-stock bank. In this case, the target was located in the account of the employee's girlfriend, because from visual graphs we found that many funds actually flowed to the girlfriend's account. It is very difficult to detect a case like this by traditional means.

**_Money Laundering Graph_**: The money laundering problem has existed for a long time. There are many derivative money laundering ways in different periods. Popular ways include the underground money bank and the "order sneaking" platform. Criminals hide their money on the platform where order-sneaking customers' daily consumption transactions are normal, however, there is actually a fixed pattern on visual graphs to detect cases like this. So how creating business relations with graphs and making graph rules to perform real-time monitoring and detections have become an important task for Bangsun, taken as a graph technology platform, to empower business developers. We will talk about it later. 

**_Vehicle Insurance/Ops Graph_**: For the vehicle insurance/ops field, we combine the initialization network with the design of vertices and edges, then query for a subgraph where the data is relatively appropriate and related, and then combine the experience of business experts to perform some exploratory and analytical work.

### **Use case 1: Flow of funds**
![](http://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/financial-fraud-detection-one-of-the-best-practices-of-knowledge-graph/figure2.png)

Here, some specific application scenarios are also briefly listed. The first one is the flow of funds after the loan is approved.

We can focus on business and consumer loans through graphs, and track whether the approved loans flow into the housing market or the stock market. In this process, we will use the penetration ability of graphs to calculate the amount and proportion of transferred funds. Thereafter, we can easily find risks and violations and provide a basis for investigation. The regulators currently follow the principle of "three approaches and one guideline" for internet credits, so we establish real-time risk control through graphs and focus on the fund flows.

Next, we can show the specific methods. For example, loans flow directly into the blacklisted accounts or their own accounts are transferred to an account not under their name, but the account may be a controlled account. The account is used for investment and financial management. These can be analyzed with graphs and the specific pattern of the funds because the fund flows will form a very obvious distribution or pattern on visual graphs.

This pattern is difficult to portray with the previous expert rules, because the expert rules can only find a one-dimensional relationship, up to two-dimensional relationships, while the graphs are more adept at multi-dimensional relationship detecting. In massive transaction structures, we can also find some patterned structures. For example, the above figure shows 4-5 vertices, which involve some centralized transfer in, decentralized transfer out, and chain transaction structures like tree branches and ant nests, which are all abnormal funding patterns. Through visual graphs and combination with business knowledge, in many cases, we can see that it is problematic at a glance. As for where the specific problem is, we can check it through some functions provided by the graph, such as the k-degree query or the discovery algorithm of important nodes.

### **Use case 2: Insurance frauds** 
![](http://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/financial-fraud-detection-one-of-the-best-practices-of-knowledge-graph/figure3.png)

The second use case is frauds in insurance, such as vehicle insurance or health insurance. The primary vehicle insurance issues are collusive claim frauds. By monitoring and associating shared identity information, it is easier to find group members and excludes irrelevant data. As for health insurance, we generally focus more on the improper relationship between doctors and patients or providers. Once there is a large number of discrete values on a drug or disease, it represents fraud. This is the common way to detect individual fraud in the past. Now the group fraud problem is severe. Group fraudsters have a shift in thinking that dozens of accounts buy the same drug through individual payment accounts. There are also aggregation accounts and many other commonalities. Combined with graph analysis and algorithms, those group frauds can be detected.

### **Summary: Advantages of graph fraud detection**
![](http://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/financial-fraud-detection-one-of-the-best-practices-of-knowledge-graph/figure4.png)

The advantages of graph fraud detection mainly include the following four aspects.

Association analysis: This means that people take the initiative to explore and discover suspicious features, and then use graphs to make some visual associations. Business developers take the initiative to use graph tools for visual exploration and then combine the features of different business areas, such as money laundering, credit card applications, and loan applications to find some suspicious points. The suspicious points are certainly different in different areas, but the business developers who often use graph tools have a certain sensitivity to such suspicious points.

Graph rules: This aspect is more for incremental monitoring. Unlike the previous expert rules, graph rules use the means of graph exploration associated with knowledge comparison to draw some conclusions. We conclude graph rules from the dimension of relationships, and then the system uses the rules to detect risks.

Pattern analysis: We recently called the same model, because many fraud patterns have a solidified pattern, such as cash-out from credit cards. After analyzing these solidified patterns, we convert the pattern in a graph query language into a graph library for a traversal query. Compared to the graph rules introduced above, it is a method for detecting existing data.

Community analysis: As mentioned above, insufficient data is lacking for modeling graph data, the community analysis is actually a very good feature to discover fraudulent groups by algorithms and combination with data labels.

The above four graph fraud detection advantages can help us detect frauds in real-time, on time, and afterward. They are both the main graph functions for anti-frauds and are considered our strengths. These advantages are currently applied by about dozens of our customers.

This is all we have for today's article! Please look forward to the next topic Introduction of Bangsun Knowledge Graph Platform and Practical Cases of State-owned Banks.

‍

A knowledge Graph specialist from Bungsun shares his best Practices of Knowledge graph in the Financial Fraud detection field.

Financial Fraud Detection: One of the Best Practices of Knowledge Graph

Real-time recommendation is a very popular topic in the retail industry. The goal of real-time recommendation is to provide personalized recommendations for users immediately after they search for products on e-commerce websites. It is a challenging task since the user has not yet decided which product they want to buy, so it requires a deep understanding of their preferences and behaviors.

In order to achieve this goal, real-time recommender systems need to be able to process big data at near real-time speed. Real-time systems are often used as part of e-commerce websites or mobile apps, where they can recommend products based on what the user is currently doing. For example, if an e-commerce site is selling shoes, it might want to show the user different styles of shoes when they are looking at the "men's sneakers" page.

Graph technology is a good choice for real-time recommendation. It has the ability to predict user behavior and make recommendations based on it. Graph databases like NebulaGraph provide a flexible data model that allows you to represent any kind of relationship between entities. This includes not only the typical "product" and "user" relationships, but also any other relationships that are important to your application. For example, you can use a graph database to represent complex groupings of users (such as "friends") or categories of products (such as "books"). As long as there's an edge between two entities, you can use graph technology.

The ability to represent any kind of relationship also makes it possible for graph databases to be more expressive than relational databases. For example, if you have a large number of users who share similar interests but belong to different groups, then in relational databases this might mean having a separate table for each group and then joining them together into one big table at query time. In contrast, in a graph database, it would be possible for each user to have multiple edges representing their interests in different groups.

# **What is a graph database?**

A graph is known as a diagram that illustrates a relationship between two things. Thus, [a graph database](https://www.nebula-graph.io/posts/what-is-a-graph-database) can be assumed to be a database to understand relationships between data.

Graph databases are unlike traditional Relational Database Management Systems (RDBMSs) most people are familiar with. RDBMSs house data in a table of columns and rows with little or no relation to each other.

Graph databases are fundamentally designed with a focus on the relationship between data sets. So, they require more intense and specialized processing capabilities. As a result, graph databases like NebulaGraph use sophisticated design and architecture.
 ![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/use-cases-of-graph-databases-in-real-time-recommendation/figure1.png)


Graph databases are designed to uncover important relationships between many big data sets. In other words, it can connect the dots between multiple datasets that might otherwise just be sitting useless in silos. And graph databases are already widely used for such purposes.

# **What is a real-time recommender system?**

recommender systems are used to predict customer preferences and provide them with products or services that they might like. It helps companies reduce costs and increase revenues by providing personalized recommendations for their customers.

Recommender systems are used in a variety of industries including retail, media and entertainment, travel, and the public sector.

For example, Netflix uses an algorithm to recommend movies based on what you’ve watched previously. Amazon uses recommendations to help shoppers find new products they might be interested in purchasing. And Facebook uses its own algorithm to show you relevant ads based on your interests and likes.

# **Why should you use NebulaGraph database for real-time recommendation?**

NebulaGraph is a highly performant linearly scalable graph database available for use via a shared-nothing distributed model. In several real production environments, data mining and technical performance has been proven to beat the performance of competing graph databases by multiple times over.

The goal behind NebulaGraph is to unleash the power of exponentially growing connected data.

NebulaGraph securely processes data sets of at least twice the size and twice as fast as any competing graph databases

*   It is the only database that can store and process billions of data points with trillions of relational connections
*   NebulaGraph is designed for scalability and recovery without disruption, ensuring the best business continuity available
   ![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/use-cases-of-graph-databases-in-real-time-recommendation/figure2.png)

# **How graph technology is used in recommendation systems**

Recommender systems aim to provide users with personalized recommendations based on their tastes and preferences. The characteristics of a user can be represented by their preferences for movies and books, for example, or their shopping habits on an e-commerce website. There are two main types of recommender systems: 1) collaborative filtering methods and 2) model-based methods.

A collaborative filtering method, such as KNN, can predict the movie rating without knowing the attributes of the movies and users. To address this challenge, the graph factorization approach combines the model-based method with the collaborative filtering method to improve prediction accuracy when the rating record is sparse.

Graph factorization methods have been widely used in many online recommender systems. Graph factorization is a graph-based model that can be used to represent user preferences as well as the relationship between users, items, and attributes. The goal of graph factorization is to extract the latent features from user ratings and recommendations so that these features can be used to predict users' preferences in an unsupervised manner.

Graph factorization is done by breaking down the original dataset into smaller datasets or clusters. This process can be done using graph databases because they are designed to support highly connected data structures and relationships between data points.

# **How to build a recommender system using NebulaGraph database**

If you want to build your own recommender system, there are many different approaches you can take. One of the most popular is using a graph database like NebulaGraph. This section will walk through how to build a recommender system using a graph database.

**Define the data model**

In this step, we define the data model for our recommender system. The first thing we need to define is what type of data can be recommended. For example, we want to recommend movies so we will use movies as our entity. The second thing is what kind of information do we need about a movie? For example, if you are building a recommender system for books, maybe you need to know what genre it belongs to or what language it was written in. But in our case, all we need is the title, the year of release, the genre, and the country. So our movie entity is defined like this:

‍
![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/use-cases-of-graph-databases-in-real-time-recommendation/figure3.png)

‍

**Define relationships**

The next thing we need to do is define what type of relationship exists between movies and users (or people). There are two types of relationships that exist between movies and users: user likes movie and user has watched movie (or user has not watched movie). In this example, we will simplify it to just user likes movie.

Now imagine two users James and Kelsey, who are both fans of Sci-Fi Movie A, which has a genre of Science fiction. James also likes Sci-Fi Movie B, but Kelsey hasn’t watched it, so we don’t know whether Kelsey likes it or not.
![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/use-cases-of-graph-databases-in-real-time-recommendation/figure4.png)

But using this graph modeling of data, we may easily find out that Kelsey may like Sci-Fi Movie B. The recommender system would recommend Sci-Fi Movie B to Kelsey because James — who likes the same things as Kelsey — likes Sci-Fi Movie B.
![](https://www-cdn.nebula-graph.io/nebula-website-5.0/images/blogs/use-cases-of-graph-databases-in-real-time-recommendation/figure5.png)

**Making recommendations**

There are a few graph algorithms that you can use to make recommendations within a graph database.

**The PageRank algorithm**: The [PageRank algorithm](http://pi.math.cornell.edu/~mec/Winter2009/RalucaRemus/Lecture3/lecture3.html) is used to rank web pages in search results. The purpose of this algorithm is to determine which web pages should be displayed first when someone searches Google or any other major search engine.

The basic idea behind PageRank is that if you link to another page, then that page is more important than your own page. So if you are linking to a page about shopping for shoes, and someone else links to your shoe shopping site, then that person thinks your site is more important than it really is.

If everyone is linking to the same sites, then the people who link most get more weight.&nbsp;

Continuing with the movie analogy, while watching movies one can either carry on watching movies of a similar genre (and hence follow their most expected journey), or skip to a random movie in a totally different genre. It turns out that this is exactly how Google ranks websites by popularity using the _PageRank_**_ _**algorithm.

The popularity of a website is measured by the number of links it points to (and is referred from). In our movie use case, the popularity is built as the number hashes a given movie shares with all its neighbors.

**Collaborative filtering:** Collaborative Filtering (CF) is a method for recommender systems based on information regarding users, items, and their connections. Recommendations are made by looking at the neighbors of the user at hand and their interests. Since they are similar, the assumption is made that they share the same interests.

Read more: [Build a Recommendation Engine With Collaborative Filtering](https://realpython.com/build-recommendation-engine-collaborative-filtering/)

# Conclusion

In conclusion, it's worthwhile to use graph databases to build real-time recommender systems.&nbsp; Since graph databases are well designed for representing the relationship among users and products in the recommender system, they can be used to build real-time recommendation systems. Graph databases' support of powerful data analysis makes it useful in constructing recommender systems that need to take into account users and products with different preferences.

# How NebulaGraph Works
1. [NebulaGraph Architecture — A Bird’s Eye View](https://www.nebula-graph.io/posts/nebula-graph-architecture-overview)
2. [Benchmark: NebulaGraph vs Dgraph vs JanusGraph](https://www.nebula-graph.io/posts/benchmarking-mainstraim-graph-databases-dgraph-nebula-graph-janusgraph)
3. [Comparison: Neo4j vs NebulaGraph vs JanusGraph](https://www.nebula-graph.io/posts/performance-comparison-neo4j-janusgraph-nebula-graph)
4. [Deploy the Graph Database on Kubernetes](https://www.nebula-graph.io/posts/how-to-deploy-nebula-graph-in-kubernetes)


Why graph database is a good choice for recommendation engines? How to implement a recommender system using graph technology? How graph algorithms like PageRank and Collaborative filtering are used in real-time recommendation?

Graph Database For Recommendation Systems

I would like to share my opinionated reference data infrastructure stack with some of those best open-source projects with modern ETL, Dashboard, Metadata Governance, and Data Lineage Management.

Introducing a metadata management system that uses NebulaGraph as graph database

In this article, I will try to walk you through four different ways that you can make NebulaGraph and Apache Spark work together: Spark Connector, Nebula Exchange, Nebula Algorithm, and PySpark.