389ds Docker容器启动后崩溃 无法找到ns-slapd进程PID

问题现象 #

使用389ds/dirsrv:2.1镜像启动容器时，容器在运行约30秒后自动崩溃，最小复现命令：

docker run 389ds/dirsrv:2.1

启动失败返回的核心报错：

Unable to find pid (/data/run/slapd-localhost.pid) of ns-slapd process

已做过的调试操作：

• 容器存活阶段通过docker exec进入容器shell环境，用ps命令检查ns-slapd进程状态，确认进程初期可以正常运行，会在某一时间点意外停止，未定位到崩溃触发点
• 完全相同的镜像在其他设备上可以正常启动运行

当前运行环境：Linux Mint 19.3 Cinnamon，使用系统软件源提供的Docker 19.03版本。

完整启动日志 #

INFO: The 389 Directory Server Container Bootstrap
INFO: Inspired by works of: ITS, The University of Adelaide
INFO: 389 Directory Server Version: 2.1.11
INFO: Initialising 389-ds-container due to empty volume ... 
DEBUG: Running setup with verbose
DEBUG: START: Starting installation ... 
DEBUG: READY: Preparing installation for localhost...
INFO: Validate installation settings ... 
DEBUG: PASSED: using config settings 999999999
DEBUG: PASSED: user / group checking
DEBUG: PASSED: prefix checking
DEBUG: list() localhost instance not found: missing /etc/dirsrv/slapd-localhost/dse.ldif

DEBUG: PASSED: instance checking
DEBUG: INFO: temp root password set to XQ1yfxlIKXCrI2Y5cYAyRKG3ouewNWy4Fsz16rqvKpav99iRPezd.rsidz4BOjUEf
DEBUG: PASSED: root user checking
DEBUG: PASSED: network avaliability checking
DEBUG: READY: Beginning installation for localhost...
DEBUG: ACTION: Creating dse.ldif
DEBUG: Container detected setting db home directory to db directory.
INFO: Create file system structures ... 
DEBUG: ACTION: creating /data/bak
DEBUG: ACTION: creating /etc/dirsrv/slapd-localhost
DEBUG: ACTION: creating /data/db
DEBUG: ACTION: creating /data/db
DEBUG: ACTION: creating /data/ldif
DEBUG: ACTION: creating /data/run/lock
DEBUG: ACTION: creating /data/logs
DEBUG: ACTION: creating /data/run
DEBUG: ACTION: Creating certificate database is /etc/dirsrv/slapd-localhost
DEBUG: Allocate <class 'lib389.DirSrv'> with None
DEBUG: Allocate <class 'lib389.DirSrv'> with /data/run/slapd-localhost.socket
DEBUG: Allocate <class 'lib389.DirSrv'> with localhost:3389
DEBUG: Allocate <class 'lib389.DirSrv'> with localhost:3389
DEBUG: nss cmd: /usr/bin/certutil -N -d /etc/dirsrv/slapd-localhost -f /etc/dirsrv/slapd-localhost/pwdfile.txt -@ /etc/dirsrv/slapd-localhost/pwdfile.txt
DEBUG: nss output: 
INFO: Create self-signed certificate database ... 
DEBUG: nss cmd: /usr/bin/certutil -N -d /etc/dirsrv/ssca/ -f /etc/dirsrv/ssca//pwdfile.txt -@ /etc/dirsrv/ssca//pwdfile.txt
DEBUG: nss output:
DEBUG: nss cmd: /usr/bin/certutil -S -n Self-Signed-CA -s CN=ssca.389ds.example.com,O=testing,L=389ds,ST=Queensland,C=AU -x -g 4096 -t CT,, -v 24 -2 --keyUsage certSigning -d /etc/dirsrv/ssca/ -z /etc/dirsrv/ssca//noise.txt -f /etc/dirsrv/ssca//pwdfile.txt
DEBUG: nss output: Is this a CA certificate [y/N]?
Enter the path length constraint, enter to skip [<0 for unlimited path]: > Is this a critical extension [y/N]?

DEBUG: nss cmd: /usr/bin/certutil -L -n Self-Signed-CA -d /etc/dirsrv/ssca/ -a
DEBUG: nss cmd: /usr/bin/openssl rehash /etc/dirsrv/ssca/
DEBUG: CSR subject -> CN=5d33ff7d9d7b,givenName=51269bd0-be73-4d45-b2a6-b80317ecf78a,O=testing,L=389ds,ST=Queensland,C=AU
DEBUG: CSR alt_names -> ['5d33ff7d9d7b']
DEBUG: nss cmd: /usr/bin/certutil -R --keyUsage digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment --nsCertType sslClient,sslServer --extKeyUsage clientAuth,serverAuth -s CN=5d33ff7d9d7b,givenName=51269bd0-be73-4d45-b2a6-b80317ecf78a,O=testing,L=389ds,ST=Queensland,C=AU -8 5d33ff7d9d7b -g 4096 -d /etc/dirsrv/slapd-localhost -z /etc/dirsrv/slapd-localhost/noise.txt -f /etc/dirsrv/slapd-localhost/pwdfile.txt -a -o /etc/dirsrv/slapd-localhost/Server-Cert.csr
DEBUG: nss cmd: /usr/bin/certutil -C -d /etc/dirsrv/ssca/ -f /etc/dirsrv/ssca//pwdfile.txt -v 24 -a -i /etc/dirsrv/slapd-localhost/Server-Cert.csr -o /etc/dirsrv/slapd-localhost/Server-Cert.crt -c Self-Signed-CA
DEBUG: nss cmd: /usr/bin/openssl rehash /etc/dirsrv/slapd-localhost
DEBUG: nss cmd: /usr/bin/certutil -A -n Self-Signed-CA -t CT,, -a -i /etc/dirsrv/slapd-localhost/ca.crt -d /etc/dirsrv/slapd-localhost -f /etc/dirsrv/slapd-localhost/pwdfile.txt
DEBUG: nss cmd: /usr/bin/certutil -A -n Server-Cert -t ,, -a -i /etc/dirsrv/slapd-localhost/Server-Cert.crt -d /etc/dirsrv/slapd-localhost -f /etc/dirsrv/slapd-localhost/pwdfile.txt
DEBUG: nss cmd: /usr/bin/certutil -V -d /etc/dirsrv/slapd-localhost -n Server-Cert -u YCV
DEBUG: asan_enabled=False
DEBUG: libfaketime installed =False
DEBUG: systemd status -> False
DEBUG: pid file /data/run/slapd-localhost.pid -> None
DEBUG: No pidfile found for localhost
DEBUG: systemd status -> False
DEBUG: DEBUG: starting with ['/usr/sbin/ns-slapd', '-D', '/etc/dirsrv/slapd-localhost', '-i', '/data/run/slapd-localhost.pid']
ERROR: Unable to find pid (/data/run/slapd-localhost.pid) of ns-slapd process

常见触发原因 #

• 最核心的诱因是使用的Docker 19.03版本过旧，默认的overlay2存储驱动存在已知的文件同步延迟bug：389ds的启动脚本会在拉起ns-slapd进程后轮询等待pid文件生成，旧版overlay2驱动在处理容器内上层可写层的小文件写入时，会出现数秒到数十秒的同步延迟，超过脚本的等待超时时间后就会抛出找不到pid的错误，直接终止容器。这也是同镜像在更新版本Docker的设备上可以正常运行的核心原因。
• 其次是宿主机安全策略拦截：Linux Mint 19.3自带的seccomp、AppArmor规则版本较旧，会拦截ns-slapd启动时需要的部分系统调用，导致进程在后台静默崩溃，自然不会生成pid文件。
• 少数情况是宿主机资源限制过低，比如给Docker分配的内存不足1G，ns-slapd初始化数据库时被OOM杀掉，也会出现相同报错。

排查&解决步骤 #

1. 优先验证存储驱动bug
   启动容器时给/data/run路径挂载tmpfs内存文件系统，绕过overlay2的文件同步问题，执行命令：

docker run --tmpfs /data/run 389ds/dirsrv:2.1

如果容器可以正常常驻不崩溃，就确认是旧版Docker存储驱动的问题。长期解决直接把Docker升级到20.10以上版本即可，临时使用可以每次启动都加上述tmpfs参数，或者在docker-compose配置中给对应服务添加tmpfs挂载规则。

2. 排查安全策略拦截
   如果加tmpfs参数后还是崩溃，用特权模式启动排除所有权限限制测试：

docker run --privileged 389ds/dirsrv:2.1

如果特权模式下启动正常，就逐一缩小权限范围定位问题：

• 先加--security-opt seccomp=unconfined参数启动，若恢复正常，说明是宿主机seccomp配置过旧，更新容器运行时的seccomp配置文件，或者给该容器单独放行ns-slapd需要的系统调用即可。
• 如果关闭seccomp还是报错，再加--security-opt apparmor=unconfined参数测试，定位到是AppArmor拦截的话，添加对应的AppArmor放行规则即可。

3. 直接获取进程崩溃的明确日志
   如果上述步骤都没定位到问题，直接进容器前台手动启动ns-slapd看原生报错，不要走容器自带的启动脚本：

# 先启动一个常驻的调试容器，直接进bash
docker run -it --entrypoint bash 389ds/dirsrv:2.1
# 进入容器后先手动执行初始化流程，再以前台debug模式启动ns-slapd
/usr/lib/dirsrv/dscontainer -r &
sleep 15
/usr/sbin/ns-slapd -D /etc/dirsrv/slapd-localhost -d 1

-d 1参数会让ns-slapd在前台运行并输出所有debug级别的日志，进程崩溃时会直接打印具体错误原因，比如端口占用、库依赖缺失、权限不足、OOM被杀、系统调用被拦截等，不需要盲猜问题。

内容的提问来源于stack exchange，提问作者bloxx