Systemd管理的无限循环JVM守护进程莫名显示成功停用的问题求助
我最近碰到了一个特别费解的问题,想请教各位大佬:我用systemd管理着一个Scala编写的JVM守护进程,它本质就是个无限循环,负责监控一堆RSS源并发送通知。
程序启动起来完全正常,但运行一段时间后就会莫名其妙被systemd标记为成功停用——最近一次刚好跑了49分57秒,systemctl status显示进程以exit code 0的状态正常退出。可我的程序是无限循环啊!根本不可能正常结束!我已经在main函数末尾加了兜底日志,不管是循环意外终止还是任何Throwable抛出,都会打SEVERE级别的日志,但日志里啥相关记录都没有。
如果是收到信号终止的话,journal里应该会有明确记录,而且也不会显示成功退出状态。我当前设置的是Restart=on-failure,虽然换成Restart=always可能能作为临时解决办法,但我真的想搞清楚这到底是怎么回事。
我的配置有个特殊点:这个守护进程在systemd里设置的是Type=forking而不是常规的Type=simple。因为我用mill构建工具(真心推荐!)来重新编译代码然后fork进程,这样我修改模板和配置文件后,能自动生成代码并编译进守护进程里。这部分工作都正常:每次systemctl start或者restart,mill都能成功重建并启动进程,子进程也能正常fork并运行一段时间——但就是会莫名退出。另外,PID文件是由子进程生成的,所以systemd一开始会报:
feedletter.service: Can't open PID file /home/feedletter/feedletter-local/feedletter.pid (yet?) after start: No such file or directory
不过后来应该找到了这个文件,因为没有出现超时退出的情况。
下面是最新的systemctl status feedletter输出:
# systemctl status feedletter ○ feedletter.service - Feedletter RSS-To-Mail-Etc Service Loaded: loaded (/etc/systemd/system/feedletter.service; enabled; preset: enabled) Active: inactive (dead) since Fri 2024-01-26 09:07:20 UTC; 10h ago Duration: 49min 57.712s Main PID: 10994 (code=exited, status=0/SUCCESS) CPU: 55.934s Jan 26 08:51:38 feedletter-play feedletter[10994]: 2024-01-26@08:51:38 [INFO] [com.mchange.feedletter.db.PgDatabase] Subscription with ID 2 has been marked confirmed. Jan 26 08:51:45 feedletter-play feedletter[10994]: 2024-01-26@08:51:45 [INFO] [com.mchange.feedletter.db.PgDatabase] Subscription with ID 3 has been marked confirmed. Jan 26 08:59:23 feedletter-play feedletter[10994]: 2024-01-26@08:59:23 [FINE] [com.mchange.feedletter.db.PgDatabase] Deleting any as-yet-unassigned items that have been deleted from feed with ID 1 Jan 26 08:59:23 feedletter-play feedletter[10994]: 2024-01-26@08:59:23 [INFO] [com.mchange.feedletter.db.PgDatabase] Updated/assigned all items from feed with ID 1, feed URL 'https://lawyersgunsdev.wpengine.com/feed' Jan 26 08:59:33 feedletter-play feedletter[10994]: 2024-01-26@08:59:33 [FINE] [com.mchange.feedletter.db.PgDatabase] Updated last-checked-time on a stable already-assigned, not yet cleared, item, feed ID 2, guid 'tag:blogger.com,1999:blog-3456975.post-7737119998523372540'. Jan 26 08:59:33 feedletter-play feedletter[10994]: 2024-01-26@08:59:33 [FINE] [com.mchange.feedletter.db.PgDatabase] Updated last-checked-time on a stable already-assigned, not yet cleared, item, feed ID 2, guid 'tag:blogger.com,1999:blog-3456975.post-7431934517490670965'. Jan 26 08:59:33 feedletter-play feedletter[10994]: 2024-01-26@08:59:33 [FINE] [com.mchange.feedletter.db.PgDatabase] Deleting any as-yet-unassigned items that have been deleted from feed with ID 2 Jan 26 08:59:33 feedletter-play feedletter[10994]: 2024-01-26@08:59:33 [INFO] [com.mchange.feedletter.db.PgDatabase] Updated/assigned all items from feed with ID 2, feed URL 'https://www.eschatonblog.com/feeds/posts/default?alt=rss' Jan 26 09:07:20 feedletter-play systemd[1]: feedletter.service: Deactivated successfully. Jan 26 09:07:20 feedletter-play systemd[1]: feedletter.service: Consumed 55.934s CPU time.
我的systemd unit文件内容如下:
[Unit] Description=Feedletter RSS-To-Mail-Etc Service After=syslog.target network.target [Service] Type=forking PIDFile=/home/feedletter/feedletter-local/feedletter.pid User=feedletter Group=feedletter WorkingDirectory=/home/feedletter/feedletter-local ExecStart=/home/feedletter/feedletter-local/feedletter daemon --fork TimeoutStopSec=90 Restart=on-failure RestartSec=10s StandardError=journal StandardOutput=journal StandardInput=null [Install] WantedBy=multi-user.target
系统是Ubuntu 23.10,希望有大佬能给点思路或者帮助,万分感谢!
备注:内容来源于stack exchange,提问作者Steve Waldman

