# Data Challenges in the AI Era

As intelligence transforms industries, data is exploding in volume and variety. Banks generate structured transactions, unstructured customer call recordings, and semi-structured JSON profiles. Hospitals manage free-text medical notes, numerical lab results, and diagnostic images. This flood of multi-source, heterogeneous data is no longer an exception—it’s the norm.

Legacy data systems, built for siloed, single-format processing, can’t keep up. They handle one data type at a time, missing the rich connections between them. But modern AI demands more: it needs holistic, context-rich insights drawn from all available data dimensions.

The challenge has shifted. It’s no longer just about storage—it’s about understanding. In the AI era, systems must mimic human cognition, linking disparate data points across modalities to form meaningful networks.

At present, the integration of multi-source heterogeneous data has become an inevitable trend, and Graph Database is one of the key technologies to solve this problem.

# Why Do We Need Graph Databases?  

## The Limits of Traditional Data Approaches

Traditional data processing methods are struggling to keep up in today’s complex landscape. Early storage models created fragmented, isolated “data silos” with little to no connection between them—making it nearly impossible to see the full picture or unlock the true value hidden in the data.

Taking enterprise customer management as an example, a customer’s profile might live in one table, purchase history in another, and service interactions in yet another. To understand their full journey, you need cross-table joins. But as data grows, these queries become slow and unwieldy with latency jumps from milliseconds to minutes. Worse, mismatched fields during joins can introduce errors, leading to inaccurate insights and flawed business decisions.

The result? Slow, inefficient analytics, overlooked relationships, and a widening disconnect between raw data and actionable insights.

## New Demands in the AI Era: Semantic Understanding and Multimodal Fusion

Traditional databases have inherent shortcomings when dealing with multimodal data. Complex implicit associations exist between multimodal data. However, the two-dimensional table structure of traditional databases cannot intuitively express such associations, making the fusion analysis of multimodal data difficult to achieve. AI’s demand for deep semantic understanding further highlights the shortcomings of traditional databases in handling complex and nonlinear relationships.

## From Multimodal Data to Relationships Analytics in One Step

To solve the data connectivity challenge, graph databases reframe relationships intuitively: disparate data points become “nodes”, and their logical connections are explicitly modeled as “edges”. This structure enables “one-click” data association—no complex joins required.

Graph databases seamlessly integrate structured and unstructured data into a unified model. For instance, when analyzing how a product’s visual features relate to user sentiment, an “image node” can be directly linked to a “comment text node” via an edge. By combining AI-driven image and text analysis, these connections reveal hidden patterns between visuals and emotions—enabling deeper semantic understanding and powerful cross-modal analytics in the AI era.

# How Do Graph Databases Empower Intelligent Data Foundations?

The data intelligence foundation is the core infrastructure for enterprises to achieve intelligent transformation, aiming to integrate multi-source heterogeneous data and provide unified, efficient data support for intelligent applications. Its construction follows a four-step framework: “content analysis, semantic alignment, domain modeling, and relationship map.” In this process, graph databases—natively designed to handle entities and relationships—play a critical role at every stage, serving as the cornerstone for multimodal data fusion and value extraction.

![https://www-cdn.nebula-graph.io/pic-upload/Multimodal%20Data%20Fusion%201-1.png](https://www-cdn.nebula-graph.io/pic-upload/Multimodal%20Data%20Fusion%201-1.png)

## Content Quarks: Turning Raw Data into Structured Building Blocks

Content analysis kicks off the data intelligence foundation. It’s all about unpacking massive, messy raw data—text, images, audio, documents—and extracting the essentials: entities, attributes, and relationships. Think of it as breaking data down into tiny, atomic units we call “content quarks”.

![https://www-cdn.nebula-graph.io/pic-upload/Multimodal%20Data%20Fusion%201-2.png](https://www-cdn.nebula-graph.io/pic-upload/Multimodal%20Data%20Fusion%201-2.png)

Advanced tools make this possible: OCR reads text in images, speech recognition converts audio to text, and LLM parse meaning from documents. Together, they transform unstructured data into clean, structured fragments.

By defining entity and relationship types upfront, graph databases provide a clear blueprint for extraction. For example, when processing payment records, a pre-built schema can guide the system to spot “User ID”, “Merchant Code”, or actions like “transfers to” with precision. This not only reduces errors but ensures consistency—setting the stage for smarter, more reliable insights down the line.

## Semantic Alignment: Breaking Down “Data Silos” and Building a Unified Semantic Space

The goal of semantic alignment is to map data from different systems and with different naming conventions into a unified semantic space, thereby achieving seamless cross-source data connection and interoperability.

![https://www-cdn.nebula-graph.io/pic-upload/Multimodal%20Data%20Fusion%201-3.png](https://www-cdn.nebula-graph.io/pic-upload/Multimodal%20Data%20Fusion%201-3.png)

This process combines the power of large language models (LLMs) for semantic understanding, data lineage analysis, and business-specific rules to identify synonyms across systems. For instance, “buyer ID” in an e-commerce platform and “account holder number” in a bank system can be recognized as the same core concept: a “user unique identifier.”  

Graph databases are a natural fit for this task. Using their native node-edge structure, they can merge different names for the same real-world entity into a single, unified node. Attributes on that node preserve the original labels from each source—like a “User X” node tagged with Customer ID: 123 and User Number: 456.

This approach allows systems to automatically recognize that different names refer to the same entity—effectively breaking down long-standing data silos and paving the way for powerful, cross-scenario analytics.  

## Domain Modeling: Flexible Data Structures for Every Use Case

Different business scenarios demand different data perspectives. Risk control focuses on user networks, suspicious transactions, and blacklisted merchants, while marketing looks at user preferences, behaviors, and campaign engagement. Domain modeling tailors the data structure to these specific needs by defining relevant concepts and business rules.

![https://www-cdn.nebula-graph.io/pic-upload/Multimodal%20Data%20Fusion%201-4.png](https://www-cdn.nebula-graph.io/pic-upload/Multimodal%20Data%20Fusion%201-4.png)

Here, graph databases act like a “customizable shelf”—flexible and easy to rearrange. Instead of rigid table schemas, they represent core ideas as nodes and connections as edges. This makes it simple to model complex relationships, like linking “blacklisted merchants” to “abnormal transactions” in fraud detection.

Best of all, the model can evolve with the business. Need to add “logistics information”? Just introduce a new node and connect it—no schema overhaul required. This agility makes graph databases ideal for building scalable, future-proof data models.

## Relationship Map: Connecting the Dots at Scale

The relationship map is the culmination of the four-step data intelligence framework—bringing together all entities and connections uncovered during content analysis, semantic alignment, and domain modeling. It forms a unified, global graph that integrates multimodal data into a single, coherent network, enabling deep data fusion and efficient querying.

![https://www-cdn.nebula-graph.io/pic-upload/Multimodal%20Data%20Fusion%201-5.png](https://www-cdn.nebula-graph.io/pic-upload/Multimodal%20Data%20Fusion%201-5.png)

This integrated graph consolidates fragmented data into one connected space. Powered by a robust graph computing engine, it uncovers hidden patterns and complex relationships that traditional systems miss.

The graph database becomes the central “hub” for storage and computation. It efficiently handles billions of nodes and edges while enabling fast, multi-hop traversals and complex pattern searches. For example, in fraud detection, querying “User A” can instantly reveal their transactions, linked merchants, triggered risk rules, and even indirect connections to known bad actors—like tracing a detective’s case map in real time.

With everything interconnected, the graph turns scattered data into actionable intelligence, unlocking the full value of enterprise multimodal data and empowering smarter, faster decisions.

# Graph Databases: The Engine of Data Intelligence

Graph databases provide a standardized framework for content extraction, enable a unified semantic layer for data alignment, offer a flexible structure for domain-specific modeling, and serve as a high-performance engine for storing and querying the relationship map.

More than just a database, graph databases represented by [NebulaGraph](https://www.nebula-graph.io/) is the core enabler of multimodal, heterogeneous data fusion—turning fragmented information into connected knowledge. By unlocking deep relationships and hidden patterns, graph databases power advanced applications like intelligent analytics, real-time risk detection, and precision marketing, laying a strong, scalable foundation for the intelligence of enterprise.


>In the first half, this article delved into the core role of graph databases in addressing the challenges of multimodal and heterogeneous data and detailed how they empower the intelligent data foundation. In the next part, we will further explore application innovation based on this and the exciting potential of integrating graph databases with AI. Stay tuned.

This article explores the core role of graph databases in addressing the challenges of multimodal and heterogeneous data and explains how they empower the intelligent data foundation.

When AI Meets Graph Databases: Innovating with Multimodal Data Fusion (Part I)


> Chunhui Huang is a Senior Development Engineer of ZKH Big Data Platform. Since 2024, he has been promoting the deep integration of NebulaGraph into multiple business scenarios at ZKH, including intelligent procurement decisions and supply chain risk control systems. This article introduces these application practices.



# About ZKH

ZKH is an online industrial products service platform dedicated to digital transformation. We aim to provide customers with transparent, efficient, and low-cost one-stop industrial product solutions. Leveraging digital management tools and big data technologies, we collaborate with partners to optimize the entire supply chain, covering the entire process from procurement and manufacturing to delivery. Additionally, ZKH offers high-quality industrial product selection services. Combined with self-built warehousing and logistics system, we create an efficient and reliable industrial product procurement and delivery experience for online and offline customers.


# Business Challenges


Before adopting [NebulaGraph](https://www.nebula-graph.io/), our main business pain points were as follows:
* **Data Scale and Complexity**: Business expansion led to rapid growth of complex relational data. Traditional databases struggled with efficient storage, querying, and real-time analysis.
* **Relational Database Limitations**: Significant performance bottlenecks in multi - table joins made it impossible to support real - time analysis of complex relational data.
* **Data Value Realization Difficulty**: Applications of complex relational data faced challenges in data modeling, algorithm optimization, and effectiveness verification.


# NebulaGraph Selection and Architecture



## Why NebulaGraph?


* **Efficient Relational Expression**: Graph structures intuitively represent entities and their relationships. This is suitable for ZKH's complex supply chain network.
* **Powerful Path Analysis**: Supports efficient graph traversal algorithms for rapid path analysis.
* **Real - time Queries and Responses**: Capable of high - concurrency, low - latency queries, supporting real - time recommendations and risk warnings.
* **Scalability and High Performance**: Distributed and storage - computing separated architecture, supporting horizontal scalability.




## Technical Architecture

![https://www-cdn.nebula-graph.io/pic-upload/ZKH.png](https://www-cdn.nebula-graph.io/pic-upload/ZKH.png)

* **Data Collection**: Data from ERP, MySQL, etc., is synchronized to Alibaba Cloud MaxCompute via DataX or Flink - CDC. Client behavior data is collected via SDK and uploaded to Alibaba Cloud DataHub. Unstructured data is stored via OSS.
* **Data Processing**: Offline computing uses MaxCompute and Spark for large - scale batch data processing. Real - time computing uses Flink for stream data processing.
* **Data Storage:** Two clusters are deployed. The online service cluster handles real - time queries, and the graph algorithm cluster deals with complex graph algorithms.
* **Data Services**: API gateway provides unified interfaces. Customized APIs meet different business needs.


# Application Practices



## Precision Recommendation


Previously, our precision recommendation in e - commerce faced challenges. We planned to introduce NebulaGraph to build a smarter recommendation system.

* **Data Input and Modeling**: User behavior data and product data are used as inputs, cleaned and transformed, and loaded into NebulaGraph for graph modeling.
* **Graph Database Structure**: A graph structure centered on products and users is constructed. Relationships between users and products are established through behavioral trajectories.
* **Real - time Feature Extraction and Recommendation Optimization**: Through NebulaGraph's nGQL query language, we can real - time extract user behavioral features. These features are used to optimize the recommendation process.


### Outcomes


After introducing NebulaGraph, we achieved a 15% increase in search recommendation click - through rates and a 5% improvement in order conversion rates. Moving forward, we aim to expand into more relational data dimensions and incorporate AI real - time learning mechanisms.


## Supply Chain


ZKH's supply chain is product - centered and driven by customer demand. With the support of NebulaGraph, we have built an efficient and visual supply chain network.

* **Procurement Decisions**: After a customer order is submitted, the system intelligently selects the most suitable supplier channels within NebulaGraph, which can help to optimize the procurement costs.
* **Inventory Management**: Products are managed in inventory based on various attributes, such as batch, quantity, and status. Real - time data sharing lets warehouse staff instantly check product status across warehouses, improving stocking operations, reducing stockpiling, and enabling joint inventory management and smart replenishment.
* **Intelligent Fulfillment**: The system optimizes the entire process from customer orders to logistics delivery. With NebulaGraph, intelligent management not only enhances the inventory turnover but also reduces warehousing costs and optimizes the overall supply chain efficiency.
* **Accurate Delivery**: We utilize real-time logistics sharing and NebulaGraph's shortest path algorithm to optimize delivery routes, ensuring that products can be delivered to customers on time and efficiently.


### Outcomes


* **Customer Satisfaction**: Transparent logistics info, on - time deliveries, and efficient order fulfillment significantly enhanced customer satisfaction, with a 20% drop in after – sales work orders.
* **Intelligence and Automation**: Graph algorithms based on NebulaGraph, such as the shortest path algorithm, collaborative filtering algorithm, etc., can automatically optimize inventory, procurement, fulfillment and delivery, reducing manual work. The intelligent supply chain system via NebulaGraph quickly responds to market shifts, boosting competitiveness.
* **Cost and Efficiency**: Optimizing inventory, procurement, and delivery has reduced operational costs, raised stock availability by 15%, and cut stagnant inventory by 10%.


# Future Plans



## Knowledge Base Management System


We're working with the [NebulaGraph GenAI](https://www.nebula-graph.io/posts/announcing_nebulagraph_RAG) team to build a knowledge base management system for ZKH. We aim to integrate structured and unstructured data efficiently. This will empower our internal teams, including sales, customer service, and operations staff, to use the knowledge base for intelligent Q&A. They can quickly access the info they need, which will make work more efficient.


## Intelligent Risk Control System


Leveraging NebulaGraph, we are exploring to build a unified customer-supplier risk-control network that fuses business registration, credit, fulfillment, and news data. At transaction time, the system instantly analyzes behaviors and legal risks, extracts risk features via nGQL, and applies algorithms to flag fraud, cutting supply-chain exposure.


# Summary


The implementation of NebulaGraph at ZKH has achieved significant results in business scenarios such as precision recommendation and supply chain optimization. In the future, ZKH and NebulaGraph will further explore the deep application of knowledge bases and intelligent risk control.

This article introduces the application practices of NebulaGraph at ZKH from aspects like selection and architecture of NebulaGraph, business scenarios, and future plans.

How ZKH Leveraged NebulaGraph to Achieve 15% Gains in Core Business Areas

# The New Era of Stablecoin Compliance

The passage of the GENIUS Act marks a watershed moment for stablecoin issuers—no longer operating in regulatory gray areas, they are now formally designated as "financial institutions" under the Bank Secrecy Act with stringent obligations. This transformation brings three critical challenges that traditional risk systems are ill-equipped to handle:

First, issuers must implement KYC/AML protocols capable of binding real-world identities to pseudonymous on-chain addresses—a near-impossible task when a single entity manipulates hundreds of wallets to launder funds. Second, they need to maintain real-time transparency between stablecoins and their 1:1 reserve assets, despite transactions fragmenting across multiple blockchains. Third, with stablecoin holders gaining priority claim status in insolvency, issuers must proactively model systemic risk exposure across interconnected user networks.

The core problem? Stablecoin transactions form complex relationship networks, but legacy systems treat them as isolated events. When a user's funds hop across Bitcoin, Ethereum, and Solana chains, relational databases see disconnected silos—while money launderers exploit these very gaps.

This is where graph technology becomes indispensable. Unlike traditional databases, graph databases natively model the relationships between entities —mapping how user identities connect to addresses, how transactions flow across chains, and how risks propagate through networks. By transforming fragmented data into intelligent relationship graphs, issuers can:

- Automatically detect hidden wallet clusters through community detection algorithms
- Reconstruct end-to-end transaction paths across chains in seconds, not hours
- Simulate insolvency scenarios by modeling issuer-reserve-user dependency networks

# How NebulaGraph Enhances Stablecoin Fraud Detection

[NebulaGraph](https://www.nebula-graph.io/) has been validated in production environments by leading stablecoin platforms, including Binance, OKLink and BlockSec. Its technical strengths address the three systemic challenges:

## Dynamic Graph Analytics: Real-Time Data for Cross-Border Compliance
Stablecoins’ cross-chain nature fragments transaction paths, and traditional risk systems—due to data silos—cannot quickly reconstruct end-to-end trails. 

NebulaGraph’s multi-chain transaction graph modeling aggregates and correlates addresses, timestamps, and transaction amounts from disparate blockchains (e.g., Bitcoin, Ethereum, Solana) into a unified fund flow visualization. It updates address relationships in milliseconds, capturing criminals’ frequent wallet-switching behaviors.

For example, a cross-border payment platform used shortest-path analysis to trace a stablecoin’s complete transfer path from TRON to Ethereum in seconds, assessing risk levels in real time. This dynamic graph capability also supports one-click generation of compliance evidence chains aligned with multi-jurisdictional regulations, drastically shortening compliance response times.

## Graph Computing: Penetrating Complex Relationships to Expose Hidden Risks

A core challenge for stablecoin issuers is deep-linking user identities to on-chain addresses. Traditional systems rely on static rules (e.g., blacklist matching) or simple associations, failing to detect sophisticated "cluster attacks" where criminals manipulate hundreds of addresses to disperse funds.

NebulaGraph’s graph computing models user identities, on-chain addresses, device fingerprints, and transaction timestamps into a dynamic relationship network. It leverages community detection algorithms (e.g., Louvain, PageRank) to automatically identify hidden fund pools.

In a cross-border money laundering case, the system analyzed transaction patterns across 200+ addresses , flagging anomalous fund flows that deviated significantly from normal user behavior. It locked down the hidden controller (back-end operator) in milliseconds. Additionally, graph computing simulates issuer-bank-creditor dependency networks to predict liquidity crisis transmission paths during stablecoin runs, providing institutions with systemic risk early-warning models.

## Real-Time Graph Features: Millisecond Response at Scale
In high-concurrent scenarios (e.g., market volatility surges), stablecoin issuers must handle traffic spikes while meeting regulatory demands for real-time risk interception. Traditional databases often lag under complex joins at thousands of queries per second, but NebulaGraph’s distributed architecture has proven its capability for trillion-edge real-time analysis.

NebulaGraph supports tens of thousands of concurrent queries per second with stable millisecond latency, even under extreme scenarios. This real-time capability not only meets regulatory deadlines but also proactively intercepts systemic risks like money laundering and bank runs, ensuring institutional stability.


# Building Resilient Risk Infrastructure for the Stablecoin Era

As U.S. regulators tighten stablecoin oversight, graph databases like NebulaGraph are no longer optional—they’re the foundation of modern risk infrastructure. By combining real-time cross-chain analytics, advanced graph algorithms, and high-concurrency scalability, NebulaGraph turns compliance from a cost center into a strategic advantage.

The GENIUS Act imposes direct compliance obligations on stablecoin issuers, but its ripple effects extend to the broader ecosystem, including compliance and security firms serving virtual asset transactions, financial institutions doing exchanges, payment gateways, as well as DeFi Protocols and NFT platforms.

For technical teams, this isn’t just about meeting deadlines—it’s about future-proofing systems against the next generation of financial threats. Ready to see how graph technology can transform your stablecoin risk strategy? Schedule a [1:1 demo](https://www.nebula-graph.io/contact) with our technical consultants to explore more.

NebulaGraph’s graph technology helps stablecoin issuers meet U.S. GENIUS Act compliance by addressing identity linking complexity and systemic risk through dynamic graph analytics and high-concurrency processing.

Bridging Compliance and Security: Graph Technology for Stablecoin Transaction Monitoring

Interact with biomedical knowledge graphs using graph-based retrieval augmented generation.

BioGraphRAG - Biomedical Knowledge Graph Retrieval Augmented Generation

As the Web3 ecosystem continues to expand, blockchain-based transactions have introduced a new paradigm of decentralized finance (DeFi) characterized by anonymity, intricate interaction paths, and a lack of centralized oversight. While this innovation offers unparalleled freedom and flexibility, it also creates significant challenges for risk management and compliance. **Traditional tools like rule engines, relational databases, and statistical analysis struggle to keep pace with the dynamic and the hidden nature of Web3 threats. Enter graph databases—a technology uniquely suited to model and analyze the complex relationships inherent in blockchain networks.**

In this article, we’ll explore why graph databases are becoming indispensable for Web3 risk management, how they address the shortcomings of traditional solutions, and their transformative impact on combating fraud, money laundering, and other illicit activities.

# The Challenges of Web3 Risk Management
## Anonymity and Lack of Context
In Web3, user identities are represented by blockchain addresses, which do not require real-name registration. This anonymity, combined with privacy-enhancing protocols and privacy coins, makes it nearly impossible to trace the origins of transactions or identify malicious actors. For example, attackers often exploit these features to launder money through multi-layered transfers or obfuscate transaction paths using time delays and decoy addresses. Traditional risk management systems, which rely on static rules and predefined thresholds, are ill-equipped to handle such sophisticated evasion tactics.

## Limitations of Rule Engines
Traditional rule-based systems operate on simple heuristics, such as flagging an address after a certain number of consecutive transactions or identifying high-value transfers as suspicious. While effective against predictable patterns, these rules falter when faced with the adaptive strategies of Web3 attackers. For instance, a money-laundering operation might split large sums into smaller amounts distributed across multiple addresses, making it difficult for static rules to detect the underlying scheme. Without the ability to understand the broader context of transactions, rule engines are rendered ineffective.

## Isolated Data Analysis
Statistical methods used in traditional risk management focus on individual metrics, such as transaction frequency or average transfer amounts. However, these approaches fail to capture the interconnected nature of Web3 transactions. Many fraudulent activities, such as Sybil attacks or airdrop farming, involve coordinated actions across multiple addresses controlled by a single entity. Analyzing each address in isolation misses the bigger picture, leaving organizations vulnerable to "group-based" attacks.

## Performance and Real-Time Constraints
The sheer scale and complexity of blockchain data pose additional challenges. Traditional relational databases rely on complex joins and queries to map relationships between entities, leading to slow performance and high resource consumption. In contrast, Web3 demands real-time risk detection and mitigation, which legacy systems cannot deliver.

# How Graph Databases Handle Web3 Risk Challenges
## Native Representation of Complex Relationships
At its core, Web3 operates as a vast network of interconnected entities—addresses, transactions, smart contracts, and protocols. Unlike relational databases, which require multiple tables and joins to represent these connections, graph databases model this data natively. Addresses become nodes, while transactions and interactions become edges, creating a semantic-rich and intuitive representation of the blockchain ecosystem. This structure simplifies modeling and enables lightning-fast queries, even across billions of nodes and edges.

For example, tracking a fund flow from a high-risk address involves traversing multiple layers of transactions. A graph database can perform this task in milliseconds, identifying all downstream addresses within 10 hops of the original source. This capability is crucial for real-time risk assessment and response.

## Pattern Recognition and Structural Analysis
Web3 fraud often exhibits distinct structural characteristics, such as circular transactions where funds loop back to their origin, star-shaped aggregations (multiple addresses funneling funds to a central account), or shared-source clusters (multiple users linked to the same IP). Graph databases excel at detecting these patterns through subgraph matching techniques, allowing systems to identify suspicious behaviors with remarkable accuracy.

By integrating with risk decision platforms, graph databases enable real-time alerts and dynamic analysis. As soon as an abnormal transaction or connection to a known blacklist node is detected, the system can trigger immediate warnings, far outpacing traditional batch-processing methods.

## Advanced Graph Algorithms for Fraud Detection
Graph databases come equipped with powerful algorithms that uncover hidden risks in large-scale networks. Take Community Detection Algorithms and Connected Components Analysis for instance. Techniques like Louvain clustering group highly interconnected addresses into potential fraud clusters. These clusters may represent orchestrated attacks, such as airdrop abuse or coordinated money laundering. Connected Components Analysis is often leveraged by Identifying isolated subgraphs to help detect rogue groups operating independently from the broader network. Such structures are often indicative of automated scripts or bot-controlled addresses.

These advanced graph algorithms allow graph databases to go beyond simple anomaly detection, providing deep insights into the structural vulnerabilities of Web3 ecosystems.

# The Future of Web3 Risk Management
As Web3 continues to reshape the financial landscape, the need for robust risk management solutions has never been greater. Graph databases offer a transformative approach to addressing the unique challenges posed by decentralized, anonymous, and complex blockchain environments. By enabling precise, real-time analysis of transaction networks, they empower organizations to stay ahead of evolving threats while ensuring regulatory compliance.

**For businesses operating in the Web3 space, adopting [graph database](https://www.nebula-graph.io/) technology is no longer optional—it’s essential.** Whether you’re a cryptocurrency exchange, a DeFi protocol, or a blockchain analytics firm, leveraging graph databases will help you unlock the full potential of your data and build a safer, more resilient Web3 ecosystem.


In the rapidly evolving world of Web3, where blockchain transactions are decentralized, anonymous, and complex, graph databases are emerging as the critical solution to tackle fraud, money laundering, and other risks with precision and speed.

Why Graph Models Outperform Traditional Tools in Web3 Risk Management

In the age of AI, competitive advantage hinges on how deeply organizations can understand their data. **With the release of NebulaGraph Enterprise v5.1, we're introducing native vector processing capabilities that address a critical demand in today’s AI and large language model development: hybrid queries combining graph structures and vector semantics.**

This enhancement significantly expands NebulaGraph Enterprise’s ability to handle unstructured data such as text, audio, and video—unlocking richer insights across domains like financial risk control and knowledge graph construction. The result is a more agile and intelligent way for enterprises to extract value from their data, backed by enterprise-grade reliability through distributed architecture improvements and cross-region disaster recovery.

# Native Vector Support Uncovers Hidden Connections  
In AI-driven applications, the gap between structured relationships and unstructured semantics often limits the depth of insight. With native vector support, NebulaGraph Enterprise v5.1 bridges this divide by integrating vector search into the graph model.

**Vectors are essentially becoming part of a node’s “attribute DNA.” This allows users to perform relationship traversal and vector similarity searches within the same query.** For example, entity relationships in an equity ownership chain can be analyzed alongside semantic intent extracted from contract texts, while sensor networks in industrial equipment graphs can be automatically linked to anomaly patterns in log messages. This unified approach opens up entirely new dimensions for data exploration and interpretation.

![image](https://www-cdn.nebula-graph.io/blogs/product_architecture_5.1.jpg)
*NebulaGraph Enterprise v5.1 Architecture*

# Up to 5.5x Performance Boost
On the performance front, NebulaGraph Enterprise v5.1 delivers major advancements. **Benchmarks based on the LDBC SF100 dataset show a 550% increase in query throughput compared to the previous generation.** Complex deep-link queries see dramatic improvements—for instance, a 10-hop equity penetration query now runs in just 1.5 seconds, down from 8.2 seconds. This effectively creates direct paths through intricate relationship networks, enabling faster, more responsive decision-making.

Additionally, full backup efficiency for ultra-large-scale graphs has been drastically improved. **A trillion-node dataset that previously required 72 hours can now be backed up in just 9 hours—greatly enhancing business continuity and operational resilience.**

# Enterprise-Grade Resilience and Security
To meet the stringent availability requirements of industries like finance and government, NebulaGraph Enterprise v5.1 introduces a robust disaster recovery and security framework.

The new cross-Zone active-active architecture leverages a multi-site, three-center design along with Raft consensus to ensure seamless traffic failover and zero data loss in case of local outages. For long-distance disaster recovery, manual switching and zone-aware scheduling minimize network latency and resource consumption—ensuring rapid service restoration even under extreme conditions.

On the security front, v5.1 offers end-to-end protection across access control, data transmission, and storage. Fine-grained permission management follows the principle of least privilege, allowing administrators to restrict access to specific subgraphs or attributes. Full TLS/mTLS encryption ensures secure data-in-transit, safeguarding sensitive information at every stage.

# Future-Ready Intelligence Infrastructure
With NebulaGraph Enterprise v5.1, the fusion of graph and vector technologies reaches a new level of maturity. By uniting structured relationships with unstructured semantics, and combining it with breakthroughs in performance, security, and enterprise readiness, we’re empowering businesses across finance, manufacturing, healthcare, and beyond to build smarter, more responsive data infrastructures.

Ready to explore what the future of data intelligence looks like? [Contact us](https://www.nebula-graph.io/contact) to unlock the power of integrated graph and vector analytics.


With the release of NebulaGraph Enterprise v5.1, we're introducing native vector processing capabilities that address a critical demand in today’s AI and large language model development: hybrid queries combining graph structures and vector semantics.

NebulaGraph Enterprise v5.1 Embeds Vector Search for AI-Grade Data Fusion

# About EasyCash
EasyCash is a leading Indonesian fintech platform that leverages AI, big data risk assessment models, and core credit systems to provide accessible, secure, and inclusive financial services for SMEs and individuals. As of July 2022, the platform has reached 11 million registered users and disbursed over $1 billion in loans to 2 million borrowers.

# Business Challenges: Bottlenecks in Real-Time  Fraud Detection
However, with rapid growth came complex operational challenges—especially when it came to fraud detection. **Traditional relational databases struggled under the pressure of high-concurrency scenarios, often taking more than three seconds to return critical query results.** This delay led to a 30% miss rate on high-risk applications, leaving gaps in the company’s defense against organized fraud rings. **Static rule-based systems were also ineffective in detecting evolving tactics such as “one device, multiple accounts” or “one ID, multiple users,” where bad actors distributed fraudulent behavior to avoid detection.**

Compounding these issues was the company’s offline data architecture, which required more than eight hours to generate graph-based features needed for machine learning models. This bottleneck severely limited the responsiveness of their AI-driven risk engine, making it difficult to deploy up-to-date fraud detection strategies in real time.

# Solution: A Graph-Powered Risk Control Upgrade
To overcome these limitations, EasyCash turned to NebulaGraph Enterprise , a distributed graph database trusted by leading enterprises for mission-critical workloads. By leveraging the power of graph technology, EasyCash built a real-time risk detection system capable of analyzing vast, interconnected datasets with low latency and high throughput.

**At the core of the new system is a real-time risk graph containing over 1 billion nodes and 19 billion edges , dynamically updated with information about users, devices, phone numbers, emergency contacts, and binding relationships.** Powered by NebulaGraph Enterprise , this infrastructure enables two key fraud detection workflows.					

## Credit Assessment: 8ms Decisions at Scale
In the credit assessment phase, when a user submits a loan application, the risk engine performs real-time queries into the user's 1–3 degree network. For example, it checks whether the applicant’s emergency contact appears on a blacklist or if the same device has been associated with more than five IDs. Based on dynamic graph rules, the system generates a risk score in under 8 milliseconds per query, even under 300+ concurrent requests.

## Disbursement: Unmasking Hidden Networks
During the disbursement phase, EasyCash combines real-time graph features generated by NebulaGraph Enterprise —such as community risk density and fund flow paths—with machine learning models to detect complex fraud patterns. Using subgraph embedding techniques, the system uncovered a spider-web-like network involving 20 devices and 100 phone numbers, enabling precise interception of coordinated loan fraud attempts.

![image](https://www-cdn.nebula-graph.io/blogs/risk_control_platform.jpg)

# Why NebulaGraph Enterprise?
EasyCash chose NebulaGraph Enterprise for its proven ability to deliver high-performance, scalable, and reliable graph computing in production environments. Some key reasons behind EasyCash’s adoption include:

- **Distributed Architecture**: Built from the ground up for horizontal scaling, NebulaGraph Enterprise allows organizations to grow their datasets seamlessly without compromising performance. Its multi-node architecture ensures high availability and fault tolerance.

- **High Concurrency & Low Latency**: The system excels in high-throughput environments where thousands of queries must be processed simultaneously with minimal response time. This makes it ideal for real-time applications such as fraud detection engines.

- **Seamless Integration**: NebulaGraph offers flexible connectors for Spark, Flink, Kafka, and other big data technologies, making it easy to integrate with existing data pipelines and machine learning workflows.

These features enabled EasyCash to build a unified, intelligent risk control platform that scales with business growth while maintaining responsiveness and accuracy.

# Customer Value Delivered: Enhanced Fraud Detection and Operational Efficiency
As a result of this transformation, EasyCash saw measurable improvements across its operations. **Risk screening was moved earlier in the process, allowing the company to intercept 60% of high-risk applications during the credit assessment stage, reducing manual review workload by 70%.** Despite increasing the overall fraud interception rate by 240%, EasyCash achieved a 15% increase in user approval rates, demonstrating that strong security does not have to come at the cost of user experience.

With NebulaGraph Enterprise, EasyCash has built a robust, intelligent, and scalable risk control system—one that supports sustainable business growth while protecting users from increasingly sophisticated threats. Its automated approval system reduces traditional financing barriers by 40%, with an average loan processing time under 8 minutes, directly addressing Indonesia’s demand for efficient, transparent, and inclusive financial services.


This article dives into how EasyCash built a robust, intelligent, and scalable risk control system based on NebulaGraph Enterprise.

Real-Time Risk Detection at High Concurrency: How EasyCash Secures Growth with NebulaGraph Enterprise


> The author,Xin Hao, is a risk management  engineer at Airwallex and a maintainer of nebula-go. This article will introduce the applications of NebulaGraph at Airwallex.


#  Airwallex  

As a renowned global payment and financial platform, Airwallex provides one-stop cross-border payment solutions for enterprises of all sizes and stages of development.  

When shopping, gaming, subscribing to video platforms, or using paid software, you encounter various cross-border payment scenarios, including card payments or recurring subscriptions. These payment services are facilitated by game companies, e-commerce platforms, or merchants on other platforms.  Behind the scenes, Airwallex supports these enterprises by offering services such as payment acceptance, fund collection, and treasury management.  

![](https://nebula-website-cn.oss-cn-hangzhou.aliyuncs.com/nebula-website-5.0/share/Airwallex/1744943392895.jpg)

In cross-border scenarios, our clients are highly diverse.  On one hand, some clients are gaming companies providing in-game purchase products to users across different countries. Others are e-commerce platforms, including self-operated platforms and those like Taobao that serve multiple merchants.  On the other hand, our clients extend beyond the platforms themselves to include the merchants on these platforms. These merchants are legally independent entities, necessitating clear and precise identification.  


#  Application Scenarios of NebulaGraph  

We categorize NebulaGraph’s use application scenarios into four main areas:  

## Know Your Customer (KYC)  

It is well known that cross-border operations involve unique legal frameworks across jurisdictions. Commercial entities are the fundamental legal units, and we must classify and link these entities to ensure compliance with local regulations and business scopes.  

1. **IPersonal Information Association:** 
    - Authorized Representatives and Legal Persons of Commercial Entities: Their information must be linked to the entities to identify who actually controls and manages them.  
    - Operators of Registered Accounts: When individuals register and use our services, their email addresses, phone numbers, and other details are associated with their accounts, aiding identity verification and risk assessment when needed.  

2. **Device Information Association:** Device information association involves client devices, such as IP addresses and device IDs. This helps detect abnormal login behaviors, such as access from unusual geographic locations or devices.  

3. **Network Information Association:** Network information association covers clients’ online behaviors and transaction patterns. By analyzing network activities, we identify anomalous transaction patterns or potential risks—for instance, frequent large transactions or multiple transactions from different locations within a short timeframe.  

4. **Facial Recognition Association:** Facial recognition association leverages biometric technology to verify client identities.  

## Real-Time Transaction Correlation  

During real-time transactions, NebulaGraph is used to correlate transaction data.  

Device information association here differs from the KYC context. Here, device data originates from payment components integrated with Airwallex SDK. When users initiate payments via Airwallex SDK (mobile, web, etc.), we track device IDs, models, IP addresses, and other details.  

Payment and collection information association ensures transaction legitimacy and security. In e-commerce, billing information association is key to detecting abnormal purchase patterns and potential fraud, while logistics information association tracks product flows and ensures transaction integrity.  

## Model Feature Extraction  

For model feature extraction, raw data is preprocessed into features used for model training and real-time inference. During online graph service operations, periodic data snapshots are generated for training, while real-time interfaces compute features for inference.  

## Visualization and Interaction  

Our current implementation is simplified, primarily serving internal operations teams. We provide web interfaces for operators to input queries and interact with the system on demand.  

# Why NebulaGraph?  

## Graph Relationships and KYC  

Know Your Customer (KYC) is a critical risk control process to verify client identities and assess risks throughout their lifecycle. It ensures compliance and prevents fraud by validating identities, understanding financial behaviors, and detecting anomalies.  

Graph databases naturally model KYC relationships. By representing clients, accounts, devices, and transactions as nodes and edges, NebulaGraph visualizes complex connections—such as legal-entity associations or account-device links. This structured approach enhances data readability and enables efficient anomaly detection, such as identifying fraudulent chains through relationship analysis.  

Thus, adopting NebulaGraph was the best choice.  

## Challenges  

Before NebulaGraph, we used a simple Python-based graph service for basic applications. During our early stages with smaller datasets, we extracted nodes and edges from offline data warehouses and stored them in memory.  


This architecture worked initially but faced issues as data grew:  
1. **Instability:** Periodic data updates every 10 minutes caused frequent failures.  
2. **Memory Inefficiency:** Increasing data volumes strained memory resources.  
3. **Limited Query Flexibility:** Basic query languages hindered complex scenario support, reducing development efficiency.  

These challenges drove us to adopt NebulaGraph.  

## NebulaGraph Selection  

During evaluation, we benchmarked multiple graph databases. Despite some competitors’ comparable performance, NebulaGraph excelled in functional completeness and enterprise adoption, making it the optimal choice.  

# System Architecture Design  

## Customer Relationship System Architecture  

### Graph Construction for Customer Relationships  


We built a heterogeneous graph, offering three key advantages:  
1. **Adaptability:** Supports diverse query scenarios, such as analyzing account linkages or transaction patterns.  
2. **Scalability:** New relationship types can be added effortlessly as business evolves.  
3. **TTL-Based Data Cleanup:** Automatic expiration of outdated data ensures relevance.  

Over time, we enriched the graph with additional relationship types, allowing granular analysis via edge attributes and weights.  

### System Architecture  


1. **Data Ingestion**  
   - Batch Processing (Full Data): Hourly bulk updates.  
   - Real-Time Data Supplement (Incremental): APIs fill missing data during new client registration.  

2. **Schema-First Approach**  
   - **Schema as Code:** Field definitions and changes are codified, ensuring consistency.  
   - **Unified Schema:** Shared across services and ETL pipelines for efficiency.  

Key applications of the graph service include:  
1. Model feature extraction for real-time client and transaction analysis.  
2. A rules engine for risk control decisions based on graph features.  
3. Internal business services leveraging graph data.  
4. Visualization interfaces for operational queries.  

## Business Complexities  

1. **Dynamic and Complex Schemas**  
   - Multiple node types (10+ variants) with diverse attributes.  
   - Frequent schema adjustments (monthly).  

2. **Data Logic Changes and Cleansing**  
   - Requires flexible workflows to accommodate rapid business shifts.  

### Mitigation Strategies  

1. **Centralized Schema Definition**  
   - Define schemas in code, auto-generating changes upon commits.  
   - Field modifications (renaming, additions) are code-driven and automated.  

2. **TTL + Full Data Writes**  
   - Combines TTL for data freshness with full writes for completeness.  

We adopted a declarative schema management approach. Field modifications trigger auto-generated schema change statements.  


For query result mapping, we use a "declare-and-get" method. Structs defined in code with annotations automatically map query results to fields, eliminating manual handling. SDKs or APIs dynamically fetch the latest schema, ensuring stability despite changes.  

## Transaction Correlation System Architecture  


This system prioritizes real-time performance, adopting a "compute-separate, merge-on-read" architecture.  

- **Historical Data (>4 hours):** Pre-processed via hourly batch jobs into subgraphs stored in the graph database.  
- **Real-Time Data (≤4 hours):** Ingested via Kafka, split, and written to the graph database after processing.  

# Future Outlook  

- **Resource Isolation:** Single Nebula clusters lack isolation, causing interference between applications during query peaks.  
- **TP/AP Compatibility:** A single cluster struggles to balance transactional processing (TP) and analytical processing (AP).  
- **Algorithm Ecosystem:** Limited openness and ecosystem for graph algorithms.  

Future improvements aim to enhance resource isolation, enrich graph algorithm support, and foster community collaboration to advance the technology and ecosystem.  

This article details Airwallex's use of NebulaGraph in cross-border payment scenarios. It covers application areas like KYC, real-time transaction correlation, model feature extraction, and visualization. 

How Airwallex Leverages NebulaGraph for Cross-Border Payment Solutions

Imagine a world where your data isn’t trapped in silos but connected like constellations, revealing hidden patterns that drive smarter decisions. Welcome to graph computing —the secret sauce powering today’s most innovative applications, from fraud detection to personalized recommendations. Let’s explore this transformative technology and how organizations are using it to turn complexity into competitive advantage.

# Decoding Graph Computing vs. Graph Databases
While often used interchangeably, these terms represent distinct concepts. Graph computing is the process of analyzing relationships to solve complex problems, like finding the quickest delivery route or detecting fraud rings. It’s the "brain" that applies graph theory to uncover insights.

Graph databases , on the other hand, are the "memory" that stores data as nodes (people, products) and edges (connections like "purchased" or "viewed"). They’re optimized to retrieve and manage these relationships efficiently. For example, a graph database might store billions of social media interactions, while graph computing algorithms analyze those connections to recommend new friends.

Think of it like cooking: the database holds the ingredients, and graph computing is the recipe that turns them into a meal.

# Why Graph Computing Matters Now More Than Ever
Traditional databases struggle with connected data. Relational systems require costly JOIN operations for relationship queries, while NoSQL solutions lack native support for multi-hop traversals. Graph computing solves this by:

- Speed : Analyzing millions of relationships in milliseconds.
- Flexibility : Adapting to evolving connections, like social networks or supply chains.
- Scalability : Handling billions of nodes and edges without slowdowns.

This capability is critical in modern applications. For example, Netflix uses graph computing to analyze viewing habits and suggest content that keeps users engaged. PayPal maps transaction patterns to stop fraud rings in real time.

# How Graph Databases Enable Graph Computing
Graph databases are the foundation for efficient graph computing. Leading solutions offer unique strengths:

[NebulaGraph](https://www.nebula-graph.io/) excels in real-time transaction processing at massive scale. It handles 10 billion+ edges natively, making it ideal for applications like ride-sharing route optimization or real-time fraud detection. Its SQL-like query language (nGQL) simplifies adoption, while open-source flexibility reduces vendor lock-in.

[Neo4j](https://neo4j.com/) focuses on deep analytical workloads, with a rich library of graph algorithms for tasks like community detection. However, its performance declines with datasets exceeding 1 billion nodes.

[TigerGraph](https://www.tigergraph.com/) targets enterprise-scale pattern matching, offering 35+ built-in algorithms and strong integration with machine learning tools. Its GSQL language appeals to SQL-proficient teams but requires more resource-intensive infrastructure.

[Amazon Neptune](https://aws.amazon.com/neptune/) simplifies cloud deployments for AWS-centric teams but lacks the hybrid flexibility of self-hosted solutions like NebulaGraph.

# Final Thoughts: From Complexity to Clarity
Graph computing isn’t about replacing your current systems—it’s about connecting them in ways that create exponential value. Whether you’re building a fintech startup or optimizing a manufacturing empire, the right solution turns data into discoveries.

Ready to Begin?
- Explore [NebulaGraph’s playground](https://explorer.nebula-graph.io/login) for hands-on learning
- Dive into the [Graph Database Community](https://join.slack.com/t/nebulagraph/shared_invite/zt-33nxrfan4-N_nhHDKOY1~C5YpHrJ3AiA) for peer insights
- Test your use case with the 14-day free trial offered by [NebulaGraph Cloud](https://www.nebula-graph.io/cloud-2025)


Let’s explore this transformative technology and how organizations are using it to turn complexity into competitive advantage.

Scaling Connections: How Graph Computing Solves Modern Data Complexity

Over the past few weeks, the Model Context Protocol (MCP) has sparked widespread discussion and attention within the technical community. Inspired by this momentum, I have contributed to several related open-source projects, including [implementing McpToolSpec for the official LlamaIndex repository](https://github.com/run-llama/llama_index/pull/17795) and collaborating with Xuanwo and FrostMing to develop [a Model Context Protocol Server for Apache OpenDAL™](https://github.com/Xuanwo/mcp-server-opendal).

As the only protocol currently offering a significant ecosystem advantage, MCP presents an exciting opportunity. In this context, [we have implemented a simple NebulaGraph MCP Server based on NebulaGraph 3.x](https://github.com/PsiACE/nebulagraph-mcp-server). This marks the first foundational component in the NebulaGraph X MCP ecosystem, opening up new avenues for exploring the potential of combining AI with graph databases. 

# What is Model Context Protocol (MCP)?

MCP, introduced by the Anthropic team, is an open protocol designed to provide AI systems with a unified and efficient way to access diverse data resources. By adopting a client-server architecture, MCP standardizes data access, addressing the fragmentation issues that traditional AI systems face when interacting with various data sources.

Key Features of MCP:

* **Standardized Communication**: MCP enables AI assistants to seamlessly access a wide range of data sources, including local data, remote services, and enterprise tools.
* **Openness and Ecosystem Advantage**: The MCP community already boasts nearly 1,000 MCP Server implementations, spanning search engines, weather services, databases, and more, facilitating rapid tool integration.
* **Flexible Architecture**: By defining clear interaction protocols between clients and servers, MCP empowers developers to build scalable and reusable systems.

While challenges around security and reliability remain, these issues are expected to be addressed as the protocol evolves and matures with improved development practices.


# Features of NebulaGraph MCP Server

Given my recent focus on Graph RAG and Agentic Workflow, leveraging large language models (LLMs) and intelligent agents to uncover insights from graph data has become a critical area of exploration. To facilitate this, I quickly developed the [NebulaGraph MCP Server](https://github.com/PsiACE/nebulagraph-mcp-server), enabling NebulaGraph 3.x to integrate into the MCP ecosystem as a tool for LLMs like Claude and GPT to invoke and utilize.

Built on FastMCP , the NebulaGraph MCP Server adheres to the core specifications of MCP, providing a lightweight and efficient connection service for graph databases. By exposing NebulaGraph's capabilities through standardized tool interfaces, this server allows LLMs to effortlessly interact with NebulaGraph data for basic graph exploration tasks.

### Multiple Transmission Modes:

* Supports both stdio and SSE (Server-Sent Events) transmission modes, catering to diverse development and deployment scenarios.

### Basic Graph Exploration Capabilities:

* Graph Space Listing: Enables models to query available graph spaces.
* Schema Querying: Supports querying schema definitions for specified graph spaces.
* Query Execution: Facilitates executing NebulaGraph queries via the MCP Server.

### Built-in Operator Templates:

* Includes templates for common operations like pathfinding and neighbor discovery, allowing LLMs to quickly retrieve preliminary insights from graph data.

To demonstrate its functionality, I built a simple example using McpToolSpec from LlamaIndex and ReActAgent. Below is a screenshot showcasing its capabilities:

![image](https://www-cdn.nebula-graph.io/blogs/3799122350000_llamaindex_mcptoolspec.png) 

# Conclusion
The NebulaGraph MCP Server represents an initial step in contributing to the MCP ecosystem, aiming to bridge the gap between language models and graph databases. As the protocol matures and its ecosystem expands, we hope it can play a vital role in real-world applications. If you're interested in this direction, we invite you to explore, experiment, and collaborate with us!

Together, let’s unlock the full potential of AI-powered graph databases!

The NebulaGraph MCP Server based on NebulaGraph 3.x is now open source. This marks the first foundational component in the NebulaGraph X MCP ecosystem, opening up new avenues for exploring the potential of combining AI with graph databases. 

Announcing the Open-Source Release of NebulaGraph MCP Server: Exploring the Infinite Possibilities of AI + Graph Databases

Storaged, the core storage component of NebulaGraph, is responsible for ensuring durability, availability, and scalability. It plays a critical role in handling reads and writes efficiently. This article explores the architecture of Storaged from the perspective of its underlying storage engine, RocksDB [1], and shares insights from real production use cases on diagnosing and resolving performance bottlenecks. 

One notable production example is Snapchat, which uses NebulaGraph to enhance its friend recommendation system at scale. (Read more [here](https://www.nebula-graph.io/posts/snapchat_enhances_friend_recommendations_with_nebulagraph).)

# How is data read and written in NebulaGraph

In NebulaGraph, both vertices and edges are stored as key-value pairs using RocksDB as the storage engine. In version 3.x, we use RocksDB 3.7.3. The data is partitioned and replicated, with each partition replica backed by a separate RocksDB instance.

![image](https://www-cdn.nebula-graph.io/blogs/storage.png)

To distribute data across multiple storage nodes, we use hash function:
- **Vertices** are partitioned based on their vertex ID.
- **Edges** are partitioned based on the source vertex ID.

This design ensures that a vertex and all its outgoing edges reside in the same partition, improving query performance. However, **super nodes** (e.g., highly connected entities such as celebrities or popular websites) can cause load imbalances. Hashing helps distribute these nodes more evenly across Storaged instances.

# Consistency and Replication via Raft

To maintain consistency between replicas, NebulaGraph employs the **Raft consensus algorithm** [2], which elects a **Leader** among the replicas. The Leader handles most read and write requests, while **Followers** serve as backups. If a Leader fails due to network issues or maintenance, a new Leader is elected.

The data write and replication process follows these steps:

1. The Leader writes data to the **Write-Ahead Log (WAL)**.
2. The write is replicated to all Followers, who also write to their own WAL.
3. Followers asynchronously persist the data to RocksDB and acknowledge the Leader.
4. Once the Leader receives acknowledgments from all Followers, it commits the write to RocksDB.

The diagram below illustrates this process.
![image](https://www-cdn.nebula-graph.io/blogs/storage_process.png)

# High Availability with Zones

For environments requiring high availability, we recommend distributing partition replicas across different zones. A zone refers to a distinct logical or physical grouping of infrastructure. In cloud environments, zones typically correspond to availability zones (AZs) [3] - geographically separate data centers within a region. These zones are close enough to support low-latency connections but far enough apart to minimize the risk of simultaneous failures due to localized disruptions (e.g., power outages or extreme weather). In this setup:

* Writes are always handled by the Leader, regardless of zone placement.
* Reads can be optimized with **zone affinity**, where queries prioritize a Storaged instance within the same zone as the compute node (Graphd) handling the query. 
* With zone affinity enabled, both Leaders and Followers can serve read requests, improving efficiency and reducing cross-zone data transfer costs. (E.g., In AWS, intra-zone data transfer is free.)

# Diagnosing High Storaged Latency: Common Causes and Solutions

In production environments, high P90 query latency and increased query failures often signal performance issues. While some problems originate from Graphd (our compute nodes) or Metad (our meta service nodes), when the root cause lies in Storaged, it always manifests as high storage latency. This section covers the most common causes of high storage latency and how to mitigate them.

## 1. Low Resource Utilization

If Storaged latency is high, but CPU usage, disk I/O, and disk throughput remain low, the software may not be fully utilizing available hardware resources. A key area to check is thread allocation for the Thrift server and RocksDB. 

Storaged relies on three main thread pools:
- Thrift IO thread pool – Handles incoming connections in an event loop. 
- Thrift worker thread pool – Handles service requests, including writes.
- Read handler thread pool – Manages disk reads via RocksDB.

The Thrift IO threads process the connections very fast and rarely cause latency issues. If latency persists, focus on the worker and read handler pools. In production, setting these pools to match the number of CPU cores on the machine or pod running Storaged typically yields optimal performance.

## 2. High CPU usage

If high CPU usage is observed across all Storaged instances, the root cause likely falls into one of the following categories.

### (1) CPU Spikes During Data Ingestion

If high CPU usage occurs primarily during data ingestion, the most likely cause is RocksDB compaction [4]. Compaction is an essential process that merges and reorganizes data in RocksDB’s Log-Structured Merge Tree (LSM tree) to optimize read performance. However, it can be resource-intensive.

To mitigate excessive CPU usage during compaction, consider the following optimizations:

* **Reduce compaction concurrency:**
    * Lower `max_subcompactions` and `max_background_jobs` to slow down compaction and distribute CPU load more efficiently.
* **Optimize custom compaction filtering**:
    * In NebulaGraph, expired data is garbage-collected using a **custom compaction filter** [5] when TTL is enabled [6]. This filter scans affected vertices and edges during compaction and verifies whether their TTL has expired. This additional check incurs CPU overhead.
    * The config `min_level_for_custom_filter` determines at which LSM level the custom compaction filter starts processing data.
    * A lower value of `min_level_for_custom_filter` will scan more levels of data and thus increase CPU load.
    * A higher value of `min_level_for_custom_filter` postpones TTL filtering to later stages, which reduces CPU usage but increases the temporary retention of expired data.

Since an excessively high value of `min_level_for_custom_filter` can result in bloated storage and potential availability issues, careful fine-tuning is necessary. The goal is to determine an optimal value that balances CPU efficiency with the retention of expired data. We will explore the impact of accumulated expired data later in this article.

### (2) Persistent High CPU Usage, Regardless of Writes

If high CPU usage persists even when there are no active writes, the cause is likely related to RocksDB read operations or thread contention within Storaged. To diagnose the issue more precisely, we recommend running `perf top` on a high-CPU Storaged instance.

* On physical or virtual machines, running `perf top` is straightforward.
* On Kubernetes deployments, additional privilege changes are required. Our Nebula Operator [7] includes the necessary configurations and can enable `perf top` with a single setting.

If `perf top` shows excessive thread context switching and preemption occurring within Thrift’s internal queues, this indicates over-threading - too many active threads competing for CPU resources. Reducing the number of worker and read handler threads can improve CPU efficiency.

## 3. High Disk I/O Utilization

If IOPS or I/O throughput reaches the disk's limits, query latency can degrade significantly. Identifying the underlying cause is crucial for maintaining performance.

### (1) High I/O During Data Ingestion

If excessive disk I/O occurs only during data ingestion, the likely causes are:
* High data ingestion speed
* High compaction concurrency

To alleviate this, consider:
* Reducing ingestion speed or adjusting the batch size
* Lowering the values of `max_subcompactions` and `max_background_jobs` to limit compaction concurrency

### (2) Constantly High I/O with Storage Errors

If disk I/O remains persistently high and approaches the disk's limit, and error logs such as `RPC failure in StorageClient` with `timeout` or `loadshedding` appear in Graphd, this likely indicates excessive amount of expired data..

In this case, Storaged is scanning the disk excessively to filter out expired data, which significantly increases I/O load. To verify this, check the statistics by running:

```
SUBMIT JOB STATS;
SHOW STATS;
```
If the amount of stale data is twice the normal level or higher, trigger a full compaction to clean up expired data:

```
SUBMIT JOB COMPACT;
```
# Conclusion

Storaged is a critical component of NebulaGraph, and its performance directly impacts query latency and system stability. This article provided insights into how data is stored, partitioned, replicated, and accessed in Storaged, along with best practices for diagnosing and resolving high storage latency.

#Reference

[1] Rocksdb. https://github.com/facebook/rocksdb 

[2] Raft. Ongaro, Diego, and John Ousterhout. "In search of an understandable consensus algorithm." 2014 USENIX annual technical conference (USENIX ATC 14). 2014.

[3] Availability Zone. https://learn.microsoft.com/en-us/azure/reliability/availability-zones-overview?tabs=azure-cli 

[4] Rocksdb Compaction. https://github.com/facebook/rocksdb/wiki/Compaction 

[5] Compaction Filter. https://github.com/facebook/rocksdb/wiki/Compaction-Filter 

[6] TTL in NebulaGraph. https://docs.nebula-graph.io/3.8.0/3.ngql-guide/8.clauses-and-options/ttl-options/

[7] Nebula Operator. https://github.com/vesoft-inc/nebula-operator 





This article explores the architecture of Storaged from the perspective of its underlying storage engine, RocksDB, and shares insights from real production use cases on diagnosing and resolving performance bottlenecks.

When NebulaGraph Slows Down: Diagnosing Storage-Related Performance Issues

Imagine asking ChatGPT, “How did the 2023 global chip shortage impact Tesla’s suppliers in Shenzhen through three degrees of separation?” Despite its eloquence, the LLM would likely falter—not because it lacks data, but because it lacks a way to map hidden, dynamic relationships. This is where graph databases like NebulaGraph become essential for the next generation of AI and LLMs.

As AI shifts from statistical pattern recognition to knowledge-driven reasoning, traditional databases (SQL, NoSQL) hit a wall. Their rigid tables and documents can’t model the interconnectedness of real-world data—supply chains, fraud networks, or molecular interactions. This blog explores how graph technology is rewriting the rules of intelligent systems.

# LLM Hallucinations: When “Confidently Wrong” Becomes Dangerous
LLMs hallucinate because they infer answers from patterns, not facts. Without grounding in verified knowledge, they risk generating plausible but dangerous outputs. Hallucinations aren’t just errors—they’re liabilities because they lack access to verified, structured knowledge. A medical chatbot might invent a drug interaction that doesn’t exist, risking patient safety.

# The Limitations of Traditional Databases
One of the biggest limitations of traditional databases is that they treat data points as isolated entities. Relational databases force data into tables, losing nuanced connections. For example, mapping a 10-layer corporate ownership structure requires 50+ JOINs in SQL, slowing queries to minutes. Vector databases retrieve semantically similar data but miss contextual relationships.

AI often struggles with the isolated data because so much of its power comes from understanding relationships. Graph databases, by contrast, are designed to store and manage data as interconnected entities, allowing AI to make sense of the links between seemingly disparate elements.

# Knowledge Graphs for Intelligent AI Systems
Graph databases are the backbone of such knowledge graphs, providing a powerful means of organizing and traversing complex webs of information. They allow AI systems to dynamically explore and connect various pieces of data, enhancing both the depth and the accuracy of the answers provided. Knowledge graphs are already being used in domains such as finance, healthcare, and education, where the complexity of the data requires more than a surface-level understanding.

Consider a fraud detection system for a major bank. Instead of treating each transaction as an isolated data point, a graph-powered AI can look at an entire network of transactions—cross-referencing user relationships, transaction frequency, locations, and any outliers that indicate something suspicious. It’s this networked understanding that enables real-time decision-making, stopping fraud as it happens rather than analyzing it after the damage is done.

# Graph RAG: Minimize Hallucinations of Traditional RAG
Graph RAG (Retrieval-Augmented Generation) is an advanced evolution of traditional RAG that integrates knowledge graphs into the retrieval process, significantly enhancing contextual understanding and multi-hop reasoning capabilities. Graph RAG addresses hallucinations of traditional RAG by leveraging the structured relationships in knowledge graphs, enabling more precise and contextually rich information retrieval. This structured approach minimizes hallucinations by grounding responses in factual, interconnected data, leading to more accurate and reliable answers.

# Two Major Methods to Implement Graph RAG

- **Build Your Graph RAG with NebulaGraph**: This method requires some coding work and solid AI knowledge. By leveraging NebulaGraph’s powerful graph database capabilities, you can construct a knowledge graph that integrates with your existing data sources. The approach of [Implementing Graph RAG with NebulaGraph](https://www.nebula-graph.io/posts/implement_graph_rag_with_nebulagraph) allows for customization and fine-tuning of the graph structure and retrieval algorithms, providing a robust solution for complex enterprise applications. It involves steps such as data ingestion, graph construction, and integration with large language models (LLMs) to enable sophisticated query handling and response generation.

- **Choose NebulaGraph RAG**: For those without coding experience, [NebulaGraph RAG](https://www.nebula-graph.io/posts/announcing_nebulagraph_RAG) offers an intelligent app-building platform that simplifies the process. This platform abstracts away the technical complexities, allowing users to build and deploy graph-powered AI applications with minimal effort. It provides a user-friendly interface for creating knowledge graphs, configuring retrieval parameters, and generating responses, making it accessible to a wider range of users within the organization.

# Build Tomorrow’s AI with NebulaGraph Today
The next AI breakthrough won’t come from bigger LLMs—it’ll come from smarter data infrastructure. NebulaGraph bridges the gap between LLMs’ linguistic brilliance and the structured intelligence of real-world knowledge. In the race to AGI, graph databases aren’t optional—they’re the bedrock. As OpenAI’s CTO once mused, “The future of AI isn’t just neural networks; it’s neural networks married to knowledge.”

Discover how graph databases like NebulaGraph are powering the future of AI and LLMs by enabling complex reasoning, reducing hallucinations, and unlocking scalable knowledge-driven applications.

Tag