生产环境MongoDB内存不足，大量出现ClientCursor::staticYield递归锁警告求助

Alright, let's walk through how to resolve both your production MongoDB memory crunch and that pesky ClientCursor::staticYield can't unlock b/c of recursive lock warning tied to the findAndModify operation on users.person.

First: Understand the Warning Log #

That specific warning pops up when a long-running operation (in your case, a findAndModify) holds a lock and fails to yield it properly. Since findAndModify acquires a write lock for the update step, if the query part is slow (like doing a full collection scan), it'll hold the lock longer than expected—triggering the warning and adding unnecessary memory overhead from lock contention and operation queuing.

Step 1: Fix the findAndModify Operation (Root of the Warning) #

• Add a targeted compound index: Your findAndModify uses clientSN and status in the query. Without an index here, MongoDB does a full collection scan every time, which is slow, locks up resources, and eats memory. Create this index immediately:

  db.person.createIndex({ clientSN: 1, status: 1 })
  

  This will make the query part of findAndModify nearly instantaneous, reducing lock hold time and eliminating the yield warning.
• Tweak operation parameters:
  • If your business doesn't strictly need the updated document returned, remove new: true—this cuts down on data transfer and memory usage.
  • Double-check that lastUpdateTime is a date field (not a string) to avoid implicit type conversion during $currentDate, which adds unnecessary processing.
• Check for lock contention: Run db.currentOp() to see if other long-running reads/writes are fighting for locks with this findAndModify. If you spot slow queries, optimize those too (add indexes, limit result sets) to reduce overall lock pressure.

Step 2: Resolve Memory Exhaustion #

MongoDB (especially with WiredTiger) loves to use available memory, but we need to keep it in check:

• Limit WiredTiger cache size: By default, WiredTiger uses up to 50% of available RAM (or 1GB, whichever is larger). For production, set this explicitly in your mongod.conf to leave enough memory for the OS and other processes (aim for 50-60% of physical RAM):

  storage:
    wiredTiger:
      engineConfig:
        cacheSizeGB: 8  # Adjust based on your server's total RAM
  

  Restart MongoDB after making this change.
• Clean up stale data: Audit the users.person collection for expired or unused documents (e.g., entries with status: "Completed" that are older than your retention policy). Archive these to a separate collection or delete them—smaller datasets mean less memory used for caching and operations.
• Monitor and hunt for memory hogs: Use mongostat to track memory usage, lock percentages, and operation times. Use mongotop to see which collections/operations are consuming the most resources. Look for unindexed sorts, large aggregations, or frequent large queries that are forcing MongoDB to use in-memory temporary tables.
• Consider sharding (if needed): If your dataset is too large for a single node's memory, set up a sharded cluster to distribute data across multiple servers. This splits the memory load and improves scalability.

Temporary Emergency Fixes #

If your server is already on the brink of memory exhaustion:

• Run db.runCommand({ closeAllDatabases: 1 }) (note: this will disconnect all client connections—only do this if you can tolerate brief downtime or have a failover setup).
• If possible, switch traffic to a secondary node temporarily, then restart the primary to free up memory.
• Throttle application requests to reduce immediate load on the MongoDB server.

内容的提问来源于stack exchange，提问作者OTUser