排查Docker中SBT构建的java.lang.OutOfMemoryError创建本地线程失败问题

我们团队正在将基于SBT的构建迁移到自托管GitHub Runner，但测试阶段出现线程创建失败的报错：

报错信息 #

[106.424s][warning][os,thread] Failed to start thread "Unknown thread"
- pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached. [106.428s][warning][os,thread] Failed to start the native thread for java.lang.Thread "application-akka.actor.default-dispatcher-8"
[106.430s][warning][os,thread] Failed to start thread "Unknown thread"
- pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached. [106.431s][warning][os,thread] Failed to start the native thread for java.lang.Thread "application-akka.actor.default-dispatcher-9" Uncaught error from thread [application-akka.actor.default-dispatcher-6]: unable to create native thread: possibly out of memory or process/resource limits reached, shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[application] Uncaught error from thread [application-akka.actor.default-dispatcher-5]: unable to create native thread: possibly out of memory or process/resource limits reached, shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[application] java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:809)
at java.base/java.util.concurrent.ForkJoinPool.createWorker(ForkJoinPool.java:1465)
at java.base/java.util.concurrent.ForkJoinPool.signalWork(ForkJoinPool.java:1588)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.push(ForkJoinPool.java:979)
at java.base/java.util.concurrent.ForkJoinPool.externalSubmit(ForkJoinPool.java:2194)

当前配置 #

.sbtopts内容：

-J-XX:MaxMetaspaceSize=1g
-J-Xmx2g
-J-Xms800m
-J-Xss1m

本地复现差异 #

我没用过SBT，但已基于Ubuntu的Docker镜像用Podman本地复现问题，通过VisualVM观察到两个差异：

• 在容器内运行clean update test assembly并暴露JMX端口时，构建失败，但内存/线程消耗看似合理；
• 在本地macOS直接运行该命令时，构建成功，VisualVM显示两个SBT进程：sbt boot（与容器内观察类似）和sbt forkMain（实际执行任务，包含数千线程）。

具体问题 #

1. 能否在容器中暴露sbt forkMain进程以便用VisualVM观察，从而定位失败时触发的阈值？
2. 容器内ulimit -a输出如下，可调整容器或Podman的哪些配置来解决问题？

real-time non-blocking time  (microseconds, -R) unlimited
core file size              (blocks, -c) unlimited
data seg size               (kbytes, -d) unlimited
scheduling priority                 (-e) 0
file size                   (blocks, -f) unlimited
pending signals                     (-i) 31047
max locked memory           (kbytes, -l) 8192
max memory size             (kbytes, -m) unlimited
open files                          (-n) 1048576
pipe size                (512 bytes, -p) 8
POSIX message queues         (bytes, -q) 819200
real-time priority                  (-r) 0
stack size                  (kbytes, -s) 8192
cpu time                   (seconds, -t) unlimited
max user processes                  (-u) 1048576
virtual memory              (kbytes, -v) unlimited
file locks                          (-x) unlimited

问题1：暴露sbt forkMain进程用于VisualVM观察 #

可以通过配置SBT让fork出的测试进程开启JMX端口，步骤如下：

1. 在项目build.sbt中添加测试进程的JVM参数：

Test / javaOptions ++= Seq(
  "-Dcom.sun.management.jmxremote",
  "-Dcom.sun.management.jmxremote.port=9010",
  "-Dcom.sun.management.jmxremote.rmi.port=9010",
  "-Dcom.sun.management.jmxremote.authenticate=false",
  "-Dcom.sun.management.jmxremote.ssl=false",
  "-Djava.rmi.server.hostname=0.0.0.0"
)

2. 启动容器时映射端口：podman run -p 9010:9010 ...
3. 运行测试命令后，VisualVM即可通过宿主机IP+9010端口连接到forkMain进程，监控线程数、内存等指标定位阈值。

若仅需临时调试，可直接通过命令行传递参数：

sbt 'set Test/javaOptions ++= Seq("-Dcom.sun.management.jmxremote", "-Dcom.sun.management.jmxremote.port=9010", "-Dcom.sun.management.jmxremote.rmi.port=9010", "-Dcom.sun.management.jmxremote.authenticate=false", "-Dcom.sun.management.jmxremote.ssl=false", "-Djava.rmi.server.hostname=0.0.0.0")' clean update test assembly

问题2：调整容器/Podman配置解决线程创建失败 #

核心问题是容器内虚拟内存不足或线程数限制未放开，可调整以下配置：

1. 增大容器内存限制 #

Podman默认内存上限可能偏低，启动容器时添加内存参数：

podman run --memory=8g --memory-swap=-1 ...

--memory=8g可根据实际需求调整，--memory-swap=-1允许使用交换内存。

2. 调整内核线程数限制 #

虽然ulimit -u显示用户级进程数足够，但内核threads-max参数限制系统总线程数：

• 临时调整：sysctl -w kernel.threads-max=200000
• 永久生效：编辑/etc/sysctl.conf添加kernel.threads-max=200000，执行sysctl -p生效。

3. 降低JVM线程栈大小 #

当前.sbtopts中-J-Xss1m为1MB栈空间，可尝试降低到512KB减少内存占用：
修改.sbtopts：

-J-XX:MaxMetaspaceSize=1g
-J-Xmx2g
-J-Xms800m
-J-Xss512k

注意：若应用有深度递归逻辑，需测试验证避免StackOverflowError。

4. 限制Akka线程池大小 #

报错涉及Akka调度器线程，可在build.sbt中限制线程池最大并行数：

Test / javaOptions += "-Dakka.actor.default-dispatcher.fork-join-executor.parallelism-max=64"

数值可根据容器CPU核心数调整，避免创建过多线程。

内容的提问来源于stack exchange，提问作者Vincent F