Hey there! Let's walk through your Elasticsearch node stats one by one to make sense of what's happening, and figure out if any tweaks are needed.

First, let's break down your JVM metrics:

• Heap Used % (64%): This means your node is using 22GB of its allocated 34.2GB heap. On the surface, 64% usage isn't alarmingly high right now, but the bigger issue is your total heap size.
• GC Collections (1 Old / 46,372 Young): Young GC runs super frequently (46k+ times), while Old GC has only run once. This tells us most short-lived objects are getting cleaned up in the young generation before they move to the old gen—good news for avoiding major GC pauses. But frequent Young GC can still cause minor performance blips, so we'll want to address that.
• Threads (163 Peak / 147 Max): The peak thread count being higher than max is a bit odd (might be a reporting quirk), but 163 threads is reasonable for an Elasticsearch node—no red flags here as long as it doesn't keep climbing.

Is Your Heap Setup Problematic? #

Yes, but not because of the 64% usage—your 34.2GB max heap is too large. Here's why:
JVM uses something called Compressed Oops (compressed pointers) to save memory, which only works when the heap is under ~32GB. Once you go over that, JVM drops this optimization, making every object take more memory. This actually reduces memory efficiency and can increase GC pressure over time.

Fixes for Your JVM Heap #

• Resize the heap to ~31GB: Set -Xms31g and -Xmx31g in your jvm.options file. Matching the initial and max heap sizes prevents JVM from dynamically resizing the heap, which causes performance spikes. This will re-enable Compressed Oops and free up a bit of system memory for other uses.
• Tweak the young generation size: Frequent Young GC suggests your young gen might be too small. Try setting -XX:NewRatio=2 (this makes the young gen 1/3 of the total heap) or explicitly set -XX:NewSize=10g and -XX:MaxNewSize=10g to give more space for short-lived objects. This should reduce how often Young GC runs.
• Upgrade your JVM version: OpenJDK 1.8.0_171 is pretty old. Upgrading to OpenJDK 11 (recommended for Elasticsearch 7.x and above) gives you better GC algorithms (like improved G1GC) which handle memory more efficiently.
• Monitor GC over time: Keep an eye on GC metrics after making changes. If Young GC is still frequent, check if you're running heavy queries/writes that are creating tons of short-lived objects—you might need to optimize those or add more nodes to distribute the load.

Your system is using 59.4GB of its 65.8GB total memory—90% is definitely high. Remember, Elasticsearch relies on two types of memory:

1. JVM heap (for ES's internal operations)
2. Off-heap memory (for Lucene's file system cache, which is critical for fast query performance)

Right now, your JVM heap is taking up ~34.2GB (over half your system memory), leaving only ~31.6GB for Lucene and other system processes. That's tight, and likely why your system memory is maxed out.

Recommendations to Lower System Memory Usage #

• First, resize the JVM heap: As mentioned earlier, dropping the heap to 31GB will free up ~3GB of system memory immediately for Lucene or other processes.
• Check for other memory-hungry processes: Use top or htop to see if there are other apps (like log shippers, monitoring tools, or unused services) eating up memory. Shut down any unnecessary ones to free up resources.
• Optimize Lucene cache settings: Lucene uses the OS file cache by default, which is a good thing—but if memory is tight, you can limit some of ES's internal caches:
  • Adjust indices.fielddata.cache.size (if you're using field data for aggregations) to a percentage like 20% of the heap.
  • Tweak indices.queries.cache.size to limit the query cache (default is 10% of the heap).
    Note: These changes might slow down queries, so test them first in a non-production environment.
• Consider upgrading system memory: If your node handles heavy query or write loads, 65.8GB might not be enough. For optimal performance, aim to leave at least half your system memory (or more, if possible) for off-heap/Lucene usage. Upgrading to 128GB would let you keep a 31GB heap and give ~97GB to Lucene, which will make queries much faster.
• Track memory trends: Use Elasticsearch's built-in monitoring or system tools like vmstat to see if the 90% usage is constant or a temporary spike. Spikes might come from large one-time queries—optimize those by adding filters, limiting result sizes, or using scroll API instead of fetching all results at once.

内容的提问来源于stack exchange，提问作者Frederick Scott Smith