维护大量缓存的Spring Boot应用G1GC故障调优求助

生产环境运行的Spring Boot REST API应用进程突然无法接收请求，且无日志输出，出现挂起情况。当前使用G1GC垃圾收集器，通过ConcurrentHashMap维护内部缓存，怀疑缓存占用了大量堆内存。我们不想删除缓存数据，希望通过GC调优解决问题。

当前VM参数 #

-Xms5000m -Xmx8000m -Xss300m -XX:MetaspaceSize=1000M -XX:MaxMetaspaceSize=1000M -XX:G1HeapRegionSize=1000M -XX:+UseG1GC -XX:-DisableExplicitGC -verbose:gc -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -XX:+PrintAdaptiveSizePolicy -Xloggc:HOME/logs/GC.log

我们可以临时增大堆内存，但运行时缓存会持续新增数据，因此增大堆内存并非永久解决方案。

进程挂起时的GC日志片段 #

393875.584: [G1Ergonomics (Concurrent Cycles) initiate concurrent cycle, reason: concurrent cycle initiation requested]
2024-06-18T15:45:22.390+0200: 393875.584: [GC pause (G1 Evacuation Pause) (young) (initial-mark) 393875.584: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 0, predicted base time: 5.59 ms, remaining time: 194.41 ms, target pause time: 200.00 ms]
 393875.584: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 0 regions, survivors: 0 regions, predicted young region time: 0.00 ms]
 393875.584: [G1Ergonomics (CSet Construction) finish choosing CSet, eden: 0 regions, survivors: 0 regions, old: 0 regions, predicted pause time: 5.59 ms, target pause time: 200.00 ms]
 393875.590: [G1Ergonomics (Heap Sizing) attempt heap expansion, reason: recent GC overhead higher than threshold after GC, recent GC overhead: 99.98 %, threshold: 10.00 %, uncommitted: 0 bytes, calculated expansion amount: 0 bytes (20.00 %)]
, 0.0058720 secs]
   [Parallel Time: 5.3 ms, GC Workers: 8]
      [GC Worker Start (ms): Min: 393875584.5, Avg: 393875584.6, Max: 393875584.6, Diff: 0.1]
      [Ext Root Scanning (ms): Min: 4.5, Avg: 4.7, Max: 4.8, Diff: 0.2, Sum: 37.2]
      [Update RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
         [Processed Buffers: Min: 0, Avg: 0.1, Max: 1, Diff: 1, Sum: 1]
      [Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
      [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
      [Object Copy (ms): Min: 0.2, Avg: 0.2, Max: 0.3, Diff: 0.1, Sum: 1.8]
      [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.2, Diff: 0.2, Sum: 1.5]
         [Termination Attempts: Min: 1, Avg: 1.0, Max: 1, Diff: 0, Sum: 8]
      [GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
      [GC Worker Total (ms): Min: 5.0, Avg: 5.1, Max: 5.1, Diff: 0.1, Sum: 40.5]
      [GC Worker End (ms): Min: 393875589.6, Avg: 393875589.6, Max: 393875589.6, Diff: 0.1]
   [Code Root Fixup: 0.0 ms]
   [Code Root Purge: 0.0 ms]
   [Clear CT: 0.1 ms]
   [Other: 0.5 ms]
      [Choose CSet: 0.0 ms]
      [Ref Proc: 0.2 ms]
      [Ref Enq: 0.0 ms]
      [Redirty Cards: 0.1 ms]
      [Humongous Register: 0.0 ms]
      [Humongous Reclaim: 0.0 ms]
      [Free CSet: 0.0 ms]
   [Eden: 0.0B(384.0M)->0.0B(384.0M) Survivors: 0.0B->0.0B Heap: 7937.7M(8000.0M)->7937.7M(8000.0M)]
 [Times: user=0.02 sys=0.00, real=0.01 secs] 
2024-06-18T15:45:22.396+0200: 393875.590: [GC concurrent-root-region-scan-start]
2024-06-18T15:45:22.396+0200: 393875.590: [GC concurrent-root-region-scan-end, 0.0000293 secs]
2024-06-18T15:45:22.396+0200: 393875.590: [GC concurrent-mark-start]
 393875.591: [G1Ergonomics (Heap Sizing) attempt heap expansion, reason: allocation request failed, allocation request: 24 bytes]
 393875.591: [G1Ergonomics (Heap Sizing) expand the heap, requested expansion amount: 33554432 bytes, attempted expansion amount: 33554432 bytes]
 393875.591: [G1Ergonomics (Heap Sizing) did not expand the heap, reason: heap already fully expanded]
2024-06-18T15:45:22.397+0200: 393875.591: [Full GC (Allocation Failure) 

请求协助解决该GC故障。

内容的提问来源于stack exchange，提问作者Sankareswari M