Apache httpd在Linux生产PHP服务器上挂起问题求助

Hey there, let's break down how to tackle this tricky httpd hang issue on your PHP servers. I've dealt with similar weirdness before, so here's a structured approach to dig into it:

1. First: Figure out why mod_wsgi (Python) is showing up on your PHP server #

This is the most confusing red flag at first glance—here's how to untangle it:

• Check if the mod_wsgi module is actually loaded in httpd:

  httpd -M | grep wsgi
  

• If it shows up, hunt down where it's configured. Look in httpd.conf or files under /etc/httpd/conf.d/ for lines starting with LoadModule wsgi_module. It might be a leftover from old deployments, or accidentally installed alongside other packages.
• Even idle modules can cause conflicts, especially when httpd tries to restart or recycle worker processes. Try temporarily commenting out the LoadModule line, restart httpd, and see if the Python-related logs and hangs stop. That's a quick way to rule out mod_wsgi as the culprit.

2. Fix the glibc "corrupted double-linked list" error #

This is a classic sign of memory corruption, which usually comes from buggy modules or outdated software:

• Your stack is extremely old: httpd 2.2.15, PHP 5.5.x, and RHEL 6 (since kernel 2.6.32 is RHEL 6) are all end-of-life with no security or bug fixes. If your business can tolerate it, prioritize upgrading to supported versions (e.g., httpd 2.4, PHP 7.4+, and a newer RHEL/CentOS release). This alone would eliminate hundreds of known memory leaks and crashes.
• Enable core dumps to pinpoint the exact module causing the corruption:

  # Allow unlimited core file size for the current session
  ulimit -c unlimited
  # Add this to your httpd startup script or httpd.conf to make it persistent
  LimitCORE unlimited
  

  When httpd crashes again, use gdb /usr/bin/httpd core.<pid> and run the bt command to get a call stack—this will show you exactly which module (mod_php, mod_wsgi, etc.) is triggering the memory issue.

3. Track down why your logs are being deleted #

This is suspicious and could be hiding critical clues:

• Check your cron jobs for log rotation scripts gone wrong. Look in /etc/cron.d/, /etc/cron.hourly/, or /etc/logrotate.d/httpd—a misconfigured script might be deleting logs instead of rotating them when server load spikes.
• Use inotifywait to monitor the log files in real time and catch the process deleting them:

  inotifywait -m -e delete /var/log/httpd/error_log /var/log/httpd/access_log
  

  This will output the PID and process name whenever someone tries to delete the logs, so you can trace it back to the source.

4. Troubleshoot the httpd hang itself #

When the server stops responding, run these checks immediately:

• Use ps aux | grep httpd to see process states. Look for processes stuck in D (uninterruptible sleep, usually waiting on IO) or Z (zombie) states—these can clog up the server.
• Check connection status with netstat -anp | grep httpd to see if you're hitting connection limits (e.g., hundreds of TIME_WAIT or ESTABLISHED connections that aren't being cleaned up).
• Adjust your httpd MPM settings to match your server's resources. The log message about StartServers/Min/MaxSpareServers suggests your current settings might be too low for the load. For a prefork setup, try something like this (tweak based on your CPU/memory):

  <IfModule prefork.c>
  StartServers       8
  MinSpareServers    5
  MaxSpareServers   20
  ServerLimit      256
  MaxClients       256
  MaxRequestsPerChild  4000
  </IfModule>
  

  Setting MaxRequestsPerChild ensures old processes get recycled, preventing memory leaks from building up over time.

5. Don't overlook PHP-specific issues #

Even if the logs don't point directly to PHP, outdated versions can cause chaos:

• Check your PHP error log (look for error_log in php.ini—usually /var/log/php-fpm.log or /var/log/php_errors.log) for fatal errors or memory limit hits around the time of the hang.
• Try disabling PHP opcache (if it's enabled) or updating it—old opcache versions have known memory bugs that can crash httpd processes.
• Use strace to trace a stuck httpd process and see what it's waiting on:

  strace -p <stuck_httpd_pid>
  

  This will show you the system calls the process is stuck on (e.g., waiting for a database connection, or reading a file that's locked).

内容的提问来源于stack exchange，提问作者user2033370