咨询PM2停止运行Node进程的原因、时间及排查日志

Hey there, let's break down how to pinpoint when your process stopped, why it happened, and where to dig up the logs to diagnose this issue.

1. Track Stop Time & Root Cause via PM2 Logs #

PM2 keeps detailed records of both its own operations and your application's behavior. Here's where to look first:

• PM2 Core Operation Log: Check ~/.pm2/pm2.log — this file logs every action PM2 takes, like starting, stopping, restarting, or handling crashed processes. Search for your app's name (or process ID) with:

  grep "your-app-name" ~/.pm2/pm2.log
  

  You’ll find entries with timestamps and exit details, like:

  PM2 | 2024-05-20 14:30:00: App [my-node-app:0] exited with code [1] via signal [SIGTERM]
  The timestamp tells you exactly when it stopped, and the exit code/signal hints at why (e.g., SIGKILL means the process was force-killed, non-zero exit codes usually point to app-level crashes).

• Application-Specific Error Logs: Navigate to ~/.pm2/logs/ — each process gets two log files: [app-name]-out.log (standard output) and [app-name]-error.log (error traces). The error log is your go-to for finding app-level issues like uncaught exceptions, memory leaks, or dependency failures that caused the crash.

2. Common Reasons Your Process Vanished #

Here are the most frequent culprits:

• App-Level Crash: Unhandled exceptions, uncaught promise rejections, or memory overflow will make the process exit abruptly. Check the app’s error log for stack traces to pinpoint the issue.
• System OOM Killer: If your server ran out of memory, the Linux kernel’s Out-of-Memory (OOM) Killer will terminate resource-heavy processes (including your Node app). Verify this by checking system logs:

  dmesg | grep -i oom
  

  Look for entries mentioning your app’s process name or PID.
• PM2 Daemon Restarted/Crashed: If the PM2 daemon itself stopped unexpectedly, all its managed processes go with it. If you didn’t run pm2 save to persist your process list and pm2 startup to set PM2 to launch on boot, restarting PM2 (or the server) will leave pm2 list empty. Run pm2 resurrect — if your process comes back, this is the issue.
• Manual Stop/Delete: Someone might have run pm2 stop all or pm2 delete all — check pm2.log for entries like "Stopping app [app-name] id [0]" or "Deleting app [app-name] id [0]".
• Server Reboot: If the server restarted and PM2 isn’t set to auto-start, your processes won’t launch on boot. Check the server’s uptime with uptime and compare it to the latest start time in pm2.log.

3. Quick Debug Steps to Confirm #

1. First, check if PM2 is even running:

   ps aux | grep pm2
   

   If no PM2 daemon is listed, that’s why your processes are gone — start PM2 with pm2 start (and run pm2 startup + pm2 save to prevent this in the future).
2. Run pm2 logs to stream all PM2 and app logs in real-time, which can help catch any ongoing issues.
3. If you suspect an OOM issue, check your server’s memory usage history with tools like free -h or htop.

内容的提问来源于stack exchange，提问作者kramer65