NebulaGraph Database v3.5.0 Benchmark Report

Disclaimer: We used the LDBC SNB benchmark as a starting point. However, the test results aren’t audited, so we want to be clear that this is not an LDBC Benchmark test run, and these numbers are not LDBC Benchmark Results.

This article provides the performance test report for NebulaGraph Community Edition v3.5.0.

In the v3.5.0 release, NebulaGraph has introduced significant enhancements including performance optimization for FIND ALL PATH queries, as well as support for index-free searches. Detailed changes and updates can be found in NebulaGraph v3.5.0 Release Note.

Here is a summary of key performance improvements in this version:

• The performance of FIND ALL PATH queries has seen a substantial improvement of approximately 50-500% across varying depths, with an impressive increase of around 600% observed for 1 to 5 hops.

• The Match2HOP_count operation's performance has registered an improvement of approximately 15%.

• An issue where incorrect results were returned when using property filtering in GO queries has been addressed. It's worth noting that a slight performance decrease may be observed in some cases, specifically with Go1~3 StepEdge and Go1~3 StepEdge_count operations.

Additionally, for those requiring high-performance results for queries with a depth exceeding 10 hops, it's recommended to consider the enterprise version of NebulaGraph.

Test Environment

Both the servers and the stress-testing machine are physical machines with the following configurations:

Test Data

The test data is derived from the LDBC-SNB SF100 dataset. The SF100 dataset size is 100GB, consisting of 282,386,021 vertices and 1,775,513,185 edges. The test graph space was partitioned into 24 partitions with 3 replicas each.

About LDBC-SNB

The Linked Data Benchmark Council (LDBC) sets the benchmarks for Graph and RDF data management. The Social Network Benchmark (SNB) is one of the software benchmarks developed by LDBC. For detailed information about the LDBC-SNB dataset, see the following links:

• LDBC-SNB Specification
• LDBC-SNB Docs
• LDBC-SNB Test Dataset Generation Tool

NebulaGraph Commit

• nebula-graphd version d1e2118
• nebula-storaged version d1e2118
• nebula-metad version d1e2118

Test Notes

• The performance testing tool utilized is k6, which is written in Go language. See the k6 official website for more details. The client used is nebula-go.

• The 50_vu, 100_vu, etc., seen on the x-axis of the graphs refer to the virtual user, a concept used by k6, which represents the concurrency in performance testing. 50_vu denotes 50 concurrent users, 100_vu denotes 100 concurrent users, and so forth.

• The performance baseline uses the officially released version v3.4.0.

• ResponseTime is the sum of Server Processing Duration (Latency), Network Result Return Duration, and Client Result Deserialization Duration.

Baseline Test

Terms and Definitions

The following terms and definitions are used in the performance test report:

• QPS: Queries Per Second, which refers to the number of queries processed per second.
• Latency: The time taken by the server to process a query.
• ResponseTime: The time taken by the client to process a query.
• RowSize: The rows returned by the query.

Test Cases and Results

Test Case 1: Retrieve the specified property of an edge

GO {} STEP FROM {} OVER KNOWS yield KNOWS.creationDate;

• 1-hop QPS

• 1-hop Latency (ms)

  

• 1-hop ResponseTime (ms)

• 1-hop RowSize

• 2-hop QPS

• 2-hop Latency (ms)

• 2-hop ResponseTime (ms)

• 2-hop RowSize

• 3-hop QPS

• 3-hop Latency (ms)

• 3-hop ResponseTime (ms)

• 3-hop RowSize

Test Case 2: Retrieve the specified property of the destination vertex

GO {} STEP FROM {} OVER KNOWS yield $$.Person.firstName;

• 1-hop QPS

• 1-hop Latency (ms)

• 1-hop ResponseTime (ms)

• 1-hop RowSize

• 2-hop QPS

• 2-hop Latency (ms)

• 2-hop ResponseTime (ms)

• 2-hop RowSize

• 3-hop QPS

• 3-hop Latency (ms)

• 3-hop ResponseTime (ms)

• 3-hop RowSize

Test Case 3: Retrieve the properties of the destination vertex and an edge

GO {} STEP FROM {} OVER KNOWS yield DISTINCT KNOWS.creationDate as t, $$.Person.firstName, $$.Person.lastName, $$.Person.birthday as birth | order by $-.t, $-.birth | limit 10;

• 1-hop QPS

• 1-hop Latency (ms)

• 1-hop ResponseTime (ms)

• 2-hop QPS

• 2-hop Latency (ms)

• 2-hop ResponseTime (ms)

• 3-hop QPS

• 3-hop Latency (ms)

• 3-hop ResponseTime (ms)

Test Case 4: Retrieve vertex properties using LOOKUP ON

LOOKUP ON Person WHERE Person.firstName == '{}' YIELD Person.firstName, Person.lastName, Person.gender, Person.birthday, Person.creationDate, Person.locationIP, Person.browserUsed;

• QPS

• Latency (ms)

• ResponseTime (ms)

• RowSize

Test Case 5: Retrieve vertex properties using FETCH PROP ON

FETCH PROP ON Person {} YIELD Person.firstName, Person.lastName, Person.gender, Person.birthday, Person.creationDate, Person.locationIP, Person.browserUsed;

• QPS

• Latency (ms)

• ResponseTime (ms)

• RowSize

Test Case 6: Retrieve an edge property using FETCH PROP ON

FETCH PROP ON KNOWS {} -> {} YIELD KNOWS.creationDate;

• QPS

• Latency (ms)

• ResponseTime (ms)

• RowSize

Test Case 7: Retrieve vertices with a specified tag and property using MATCH

MATCH (v:Person) WHERE v.Person.firstName == '{}' RETURN v;

• QPS

• Latency (ms)

• ResponseTime (ms)

• RowSize

Test Case 8: Retrieve vertices with a specified tag, edge type, and VID on a 1-hop path using MATCH

MATCH (v1:Person)-[e:KNOWS]->(v2:Person) WHERE id(v1) == {} RETURN v2;

• QPS

• Latency (ms)

• ResponseTime (ms)

• RowSize

Test Case 9: Retrieve vertices with a specified tag, edge type, and VID on a 2-hop path using MATCH

MATCH (v1:Person)-[e:KNOWS*2]->(v2:Person) WHERE id(v1) == {} RETURN v2;

• QPS

• Latency (ms)

• ResponseTime (ms)

• RowSize

Test Case 10: Retrieve the number of vertices with a tag and property using MATCH

MATCH (v:Person) WHERE id(v) == {} RETURN count(v.Person.firstName);

• QPS

• Latency (ms)

• ResponseTime (ms)

Test Case 11: Retrieve the sum of the lengths of the specified vertex properties using MATCH

MATCH (v:Person)-[e:KNOWS]-(v2) WHERE id(v) == {} AND v2.Person.locationIP != 'yyy' RETURN length(v.Person.browserUsed) + length(v2.Person.gender);

• QPS

• Latency (ms)

• ResponseTime (ms)

• RowSize

Test Case 12: Retrieve the sum of the lengths of the specified vertex properties using MATCH and WITH.

MATCH (v:Person)-[e:KNOWS]-(v2) WHERE id(v) == {} and v2.Person.locationIP != 'yyy' WITH v, v2 as v3 return length(v.Person.browserUsed) + (v3.Person.gender);

• QPS

• Latency (ms)

• ResponseTime (ms)

• RowSize

Test Case 13: Retrieve properties of vertices on a 2-hop path and sort them using MATCH

MATCH (m)-[:KNOWS]-(n) WHERE id(m)=={} OPTIONAL MATCH (n)<-[:KNOWS]-(l) RETURN length(m.Person.lastName) AS n1, length(n.Person.lastName) AS n2, l.Person.creationDate AS n3 ORDER BY n1, n2, n3 LIMIT 10;

Test Case 14: Retrieve properties of vertices on a 2-hop path and sort them using MATCH and WITH

MATCH (m)-[:KNOWS]-(n) WHERE id(m)=={} MATCH (n)-[:KNOWS]-(l) WITH m AS x, n AS y, l RETURN x.Person.firstName AS n1, y.Person.firstName AS n2, CASE WHEN l.Person.firstName is not null THEN l.Person.firstName WHEN l.Person.gender is not null THEN l.Person.birthday ELSE 'null' END AS n3 ORDER BY n1, n2, n3 LIMIT 10

• QPS

• Latency (ms)

• ResponseTime (ms)

Test Case 15: Retrieve the property of an edge and count the number of results

GO {} STEP FROM {} OVER KNOWS YIELD KNOWS.creationDate | RETURN count(*);

• 1-hop QPS

• 1-hop Latency (ms)

• 1-hop ResponseTime (ms)

• 2-hop QPS

• 2-hop Latency (ms)

• 2-hop ResponseTime (ms)

• 3-hop QPS

• 3-hop Latency (ms)

• 3-hop ResponseTime (ms)

Test Case 16: Retrieve the property of the destination vertex and count the number of results

GO 1 STEP FROM {} OVER KNOWS YIELD $$.Person.firstName | RETURN count(*);

• 1-hop QPS

• 1-hop Latency (ms)

• 1-hop ResponseTime (ms)

• 2-hop QPS

• 2-hop Latency (ms)

• 2-hop ResponseTime (ms)

• 3-hop QPS

• 3-hop Latency (ms)

• 3-hop ResponseTime (ms)

Test Case 17: Retrieve the vertex property using LOOKUP ON and count the number of results

LOOKUP ON Person WHERE Person.firstName == '{}' YIELD Person.firstName | count(*);

• QPS

• Latency (ms)

• ResponseTime (ms)

Test Case 18: Retrieve vertices with a specified tag and property using MATCH and count the number of results

MATCH (v:Person) WHERE v.Person.firstName == '{}' RETURN count(*);

• QPS

• Latency (ms)

• ResponseTime (ms)

Test Case 19: Retrieve vertices with a specified tag, edge type, and VID on a 1-hop path using MATCH and count the number of results

MATCH (v1:Person)-[e:KNOWS]->(v2:Person) WHERE id(v1) == {} RETURN count(*);

• QPS

• Latency (ms)

• ResponseTime (ms)

Test Case 20: Retrieve vertices with a specified tag, edge type, and VID on a 2-hop path using MATCH and count the number of results

MATCH (v1:Person)-[e:KNOWS*2]->(v2:Person) WHERE id(v1) == {} RETURN count(*);

• QPS

• Latency (ms)

• ResponseTime (ms)

Test Case 21: Insert vertices

INSERT VERTEX Comment (creationDate, locationIP, browserUsed, content, length) VALUES {}:('{}', '{}', '{}', '{}', {});

• QPS

• Latency (ms)

• ResponseTime (ms)

Test Case 22: Insert edges

INSERT EDGE LIKES (creationDate) VALUES {}→{}:('{}');

• QPS

• Latency (ms)

• ResponseTime (ms)

Test Case 23: Insert vertices with indexes

INSERT VERTEX Person (firstName,lastName,gender,birthday,creationDate,locationIP,browserUsed) VALUES {0}:(\"{1}\", \"{2}\",\"{3}\",\"{4}\",datetime(\"{5}\"), \"{6}\",\"{7}\");

• QPS

• Latency (ms)

• ResponseTime (ms)

Test Case 24: Insert edges with indexes

INSERT EDGE WORK_AT (workFrom) VALUES {0}→{1}:({2});

• QPS

• Latency (ms)

• ResponseTime (ms)

Monitoring Server Status

192.168.15.8

192.168.15.9

192.168.15.10

192.168.15.14

---

Thanks for reading through (///▽///)

Welcome to go to GitHub Repo and download the latest NebulaGraph v3.5.0：https://github.com/vesoft-inc/nebula/releases/tag/v3.5.0

Learn more about NebulaGraph

1. NebulaGraph Architecture — A Bird’s Eye View
2. Benchmark: NebulaGraph vs Dgraph vs JanusGraph
3. Comparison: Neo4j vs NebulaGraph vs JanusGraph