ArangoDB集群查询任务分布、效率优化及配置咨询

Hey there, let's break down your ArangoDB questions one by one, based on my experience with distributed graph deployments:

1. Is this shortest path query executed on a single node or distributed? #

Your cluster uses an all-in-one deployment where each node serves as agent, coordinator, and dbserver. Here's how your query runs:

• The query is initiated by a node acting as a coordinator. This coordinator parses the execution plan, then identifies which dbservers hold the relevant vertex/edge data based on shard distribution.
• While the coordinator fetches necessary data from distributed dbservers, the BFS traversal logic itself runs locally on the initiating coordinator node. It doesn't distribute the traversal computation across all nodes; instead, it pulls the required graph data into its own memory to complete the shortest path search.

2. How to speed up queries for large-scale graphs? #

Since your execution plan shows no indexes or optimizations are being used, we'll focus on indexing, memory tuning, and query optimization:

Indexing Fixes (Critical!) #

• Edge Index for Traversal: For OUTBOUND shortest path queries, ArangoDB needs to quickly find all edges starting from a given vertex. Create a hash index on the _from field of your edge collection:

  CREATE INDEX idx_contact_edge_from ON contact_edge(_from)
  

• Vertex Filter Index: Your query filters on v.c_id == @c_id—add a hash index to the c_id field of your vertex collection to speed up this post-traversal filter (or even enable early filtering):

  CREATE INDEX idx_contact_vertex_c_id ON contact_vertex(c_id)
  

• Bonus: If you often filter c_id alongside traversal targets, consider a composite index (e.g., (_id, c_id)) but the single-field indexes above should cover most cases.

Memory & Configuration Tuning #

• Cache Size: With 254GB of RAM per node, set the ArangoDB cache to 60-70% of total memory (e.g., 150GB) via the --cache.size flag or config file. This reduces disk I/O by keeping frequently accessed vertices/edges in memory.
• Thread Count: Match the --server.maximal-threads setting to your CPU core count (40 cores → set to 40-48) to maximize parallel processing capacity.
• RocksDB Tuning: For dbserver nodes, increase RocksDB write buffers and level sizes to reduce disk contention:
  • --rocksdb.total-write-buffer-size=16G
  • --rocksdb.max-bytes-for-level-base=32G

Query Optimization #

• Filter Earlier: Instead of filtering after traversing the entire shortest path, narrow down target vertices first to reduce traversal scope:

  LET targetVertices = (FOR v IN contact_vertex FILTER v.c_id == @c_id RETURN v._id)
  FOR v,p IN outbound SHORTEST_PATH @startnode TO targetVertices GRAPH @graphname OPTIONS {bfs:true}
  RETURN v
  

• Confirm BFS Setting: You're already using OPTIONS {bfs:true}, which is correct for shortest path searches (BFS guarantees the shortest path in unweighted graphs, unlike DFS).

3. How to properly configure your current cluster? #

Your all-in-one deployment works for testing, but for production-scale graph workloads, we recommend splitting roles and optimizing sharding:

Split Node Roles #

• Agent Nodes: Dedicate 3 nodes (odd count for Raft consensus) as pure agents. They only manage cluster metadata, so you can use smaller resources (8GB RAM, 4 cores).
• Coordinator Nodes: Use 2-4 pure coordinators to handle query parsing, plan execution, and dbserver coordination. Allocate 64-128GB RAM and 20-40 cores per node based on query concurrency.
• DBServer Nodes: Keep your 3 existing nodes as pure dbservers (254GB RAM, 40 cores) — these handle all data storage and query execution for sharded data.

Sharding & Replication #

• Shard Count: Set shard count for contact_vertex and contact_edge to number of dbservers * 4-8 (e.g., 3 dbservers → 12-24 shards) to distribute load evenly.
• Replication Factor: Set replication-factor:2 to ensure data redundancy and high availability (if one dbserver goes down, another holds a copy).
• SmartGraph (If Applicable): If your c_id field represents a natural partition (e.g., customer groups), configure a SmartGraph with c_id as the shard key. This ensures related vertices/edges live on the same dbserver, eliminating cross-node data transfer for queries filtered by c_id.

Monitoring #

Set up Prometheus + Grafana with ArangoDB's built-in exporters to track CPU, memory, disk I/O, and query latency. This helps you spot bottlenecks and adjust configurations proactively.

内容的提问来源于stack exchange，提问作者feitianStyle