使用GDB调试DDNet-Server时因SIGPIPE崩溃求助

问题概况 #

• 未通过GDB启动时，DDNet-Server运行完全正常
• 执行gdb ./DDNet-Server启动服务器后，初期无异常，但运行数小时后会崩溃
• 崩溃根源疑似与curl组件相关，已尝试添加signal(SIGPIPE, SIG_IGN);忽略SIGPIPE信号，但问题仍未解决

崩溃时GDB栈跟踪信息 #

2023-03-02 19:37:25 I chat: *** 'nameless tee' has left the game (Timeout Protection over)
2023-03-02 19:37:25 I game: leave player='0:nameless tee'
[New Thread 0x7fffe6ffd700 (LWP 4022525)]
[Thread 0x7fffe6ffd700 (LWP 4022525) exited]

Thread 6 "DDNet-Server" received signal SIGPIPE, Broken pipe.
[Switching to Thread 0x7fffe77fe700 (LWP 3934801)]
__libc_write (nbytes=24, buf=0x7fffd80c0aa3, fd=14) at ../sysdeps/unix/sysv/linux/write.c:26
26      ../sysdeps/unix/sysv/linux/write.c: No such file or directory.
(gdb) bt
#0  __libc_write (nbytes=24, buf=0x7fffd80c0aa3, fd=14) at ../sysdeps/unix/sysv/linux/write.c:26
#1  __libc_write (fd=14, buf=0x7fffd80c0aa3, nbytes=24) at ../sysdeps/unix/sysv/linux/write.c:24
#2  0x00007ffff7d39605 in ?? () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#3  0x00007ffff7d3476a in ?? () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#4  0x00007ffff7d337c4 in ?? () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#5  0x00007ffff7d33c83 in BIO_write () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#6  0x00007ffff74df91a in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1
#7  0x00007ffff74e0809 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1
#8  0x00007ffff74ea1da in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1
#9  0x00007ffff74e8425 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1
#10 0x00007ffff7c42cd2 in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so.4
#11 0x00007ffff7c42d31 in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so.4
#12 0x00007ffff7c4960b in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so.4
#13 0x00007ffff7bff2aa in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so.4
#14 0x00007ffff7c1d720 in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so.4
#15 0x00007ffff7c1eab1 in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so.4
#16 0x00007ffff7c1f286 in curl_multi_perform () from /usr/lib/x86_64-linux-gnu/libcurl.so.4
#17 0x00007ffff7bf5a93 in curl_easy_perform () from /usr/lib/x86_64-linux-gnu/libcurl.so.4
#18 0x000055555568e4cf in CHttpRequest::RunImpl (this=0x555555ab7c10, pUser=0x7fffd8018f80) at /home/galactic/git/ddnet/src/engine/shared/http.cpp:295
#19 0x000055555568de05 in CHttpRequest::Run (this=0x555555ab7c10) at /home/galactic/git/ddnet/src/engine/shared/http.cpp:161
#20 0x000055555569267d in CJobPool::RunBlocking (pJob=0x555555ab7c10) at /home/galactic/git/ddnet/src/engine/shared/jobs.cpp:118
#21 0x000055555568a7a7 in IEngine::RunJobBlocking (pJob=0x555555ab7c10) at /home/galactic/git/ddnet/src/engine/shared/engine.cpp:124
#22 0x000055555559f6c6 in CRegister::CProtocol::CJob::Run (this=0x555555aafec0) at /home/galactic/git/ddnet/src/engine/server/register.cpp:417
#23 0x000055555569267d in CJobPool::RunBlocking (pJob=0x555555aafec0) at /home/galactic/git/ddnet/src/engine/shared/jobs.cpp:118
#24 0x00005555556923a8 in CJobPool::WorkerThread (pUser=0x5555559f4540) at /home/galactic/git/ddnet/src/engine/shared/jobs.cpp:73
#25 0x00005555556b9cfc in thread_run (user=0x5555559f68f0) at /home/galactic/git/ddnet/src/base/system.cpp:753
#26 0x00007ffff772aea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
#27 0x00007ffff764aa2f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) 

排查与解决建议 #

1. 确保SIGPIPE信号处理覆盖所有线程：
   signal()设置的信号处理为进程级，但如果线程在设置前已创建，可能无法继承该处理规则。建议在进程启动最早期调用signal(SIGPIPE, SIG_IGN);，或者在每个工作线程（如CJobPool的WorkerThread）中通过pthread_sigmask()主动屏蔽SIGPIPE信号。

2. 完善curl请求的错误处理：
   从栈跟踪可见，崩溃发生在curl_easy_perform()执行过程中，说明HTTP请求的连接已被远端关闭。需要在CHttpRequest::RunImpl中检查curl_easy_perform()的返回值，一旦检测到连接中断类错误，立即终止请求并清理相关资源，避免后续写入操作触发SIGPIPE。

3. 更新依赖库版本：
   当前使用的libcurl和openssl版本较旧（libssl.so.1.1），旧版本可能存在线程安全或信号处理的已知问题。尝试更新到最新稳定版的curl和openssl，验证是否能解决该崩溃。

4. 调整GDB的信号处理策略：
   GDB会默认拦截SIGPIPE信号并暂停进程，这可能与程序自身的信号处理逻辑冲突。在GDB中执行handle SIGPIPE nostop noprint pass，让GDB不拦截该信号，直接传递给进程处理。

内容的提问来源于stack exchange，提问作者GalacticRider