C pthread多线程场景下valgrind检测到still reachable问题

问题背景 #

使用pthread进行多线程程序开发时，程序功能运行完全正常，但valgrind检测到存在still reachable类型的内存残留：总大小固定为1654字节，共4个内存块，输出的调用栈涉及函数始终一致。

Valgrind检测日志 #

==29908== Memcheck, a memory error detector
==29908== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==29908== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==29908== Command: ./a.out
==29908== Parent PID: 23902
==29908== 
==29908== 
==29908== HEAP SUMMARY:
==29908==     in use at exit: 1,654 bytes in 4 blocks
==29908==   total heap usage: 8 allocs, 4 frees, 2,526 bytes allocated
==29908== 
==29908== 36 bytes in 1 blocks are still reachable in loss record 1 of 4
==29908==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==29908==    by 0x401F5CE: strdup (strdup.c:42)
==29908==    by 0x4019A81: _dl_load_cache_lookup (dl-cache.c:338)
==29908==    by 0x400A989: _dl_map_object (dl-load.c:2102)
==29908==    by 0x4015D36: dl_open_worker (dl-open.c:513)
==29908==    by 0x49DF8B7: _dl_catch_exception (dl-error-skeleton.c:208)
==29908==    by 0x40155F9: _dl_open (dl-open.c:837)
==29908==    by 0x49DE860: do_dlopen (dl-libc.c:96)
==29908==    by 0x49DF8B7: _dl_catch_exception (dl-error-skeleton.c:208)
==29908==    by 0x49DF982: _dl_catch_error (dl-error-skeleton.c:227)
==29908==    by 0x49DE994: dlerror_run (dl-libc.c:46)
==29908==    by 0x49DE994: __libc_dlopen_mode (dl-libc.c:195)
==29908==    by 0x486E99A: pthread_cancel_init (unwind-forcedunwind.c:53)
==29908== 
==29908== 36 bytes in 1 blocks are still reachable in loss record 2 of 4
==29908==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==29908==    by 0x400D5A7: _dl_new_object (dl-object.c:196)
==29908==    by 0x4006E96: _dl_map_object_from_fd (dl-load.c:997)
==29908==    by 0x400A61A: _dl_map_object (dl-load.c:2236)
==29908==    by 0x4015D36: dl_open_worker (dl-open.c:513)
==29908==    by 0x49DF8B7: _dl_catch_exception (dl-error-skeleton.c:208)
==29908==    by 0x40155F9: _dl_open (dl-open.c:837)
==29908==    by 0x49DE860: do_dlopen (dl-libc.c:96)
==29908==    by 0x49DF8B7: _dl_catch_exception (dl-error-skeleton.c:208)
==29908==    by 0x49DF982: _dl_catch_error (dl-error-skeleton.c:227)
==29908==    by 0x49DE994: dlerror_run (dl-libc.c:46)
==29908==    by 0x49DE994: __libc_dlopen_mode (dl-libc.c:195)
==29908==    by 0x486E99A: pthread_cancel_init (unwind-forcedunwind.c:53)
==29908== 
==29908== 384 bytes in 1 blocks are still reachable in loss record 3 of 4
==29908==    at 0x483DD99: calloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==29908==    by 0x401330A: _dl_check_map_versions (dl-version.c:274)
==29908==    by 0x40160EC: dl_open_worker (dl-open.c:577)
==29908==    by 0x49DF8B7: _dl_catch_exception (dl-error-skeleton.c:208)
==29908==    by 0x40155F9: _dl_open (dl-open.c:837)
==29908==    by 0x49DE860: do_dlopen (dl-libc.c:96)
==29908==    by 0x49DF8B7: _dl_catch_exception (dl-error-skeleton.c:208)
==29908==    by 0x49DF982: _dl_catch_error (dl-error-skeleton.c:227)
==29908==    by 0x49DE994: dlerror_run (dl-libc.c:46)
==29908==    by 0x49DE994: __libc_dlopen_mode (dl-libc.c:195)
==29908==    by 0x486E99A: pthread_cancel_init (unwind-forcedunwind.c:53)
==29908==    by 0x486EBB3: _Unwind_ForcedUnwind (unwind-forcedunwind.c:127)
==29908==    by 0x486CF05: __pthread_unwind (unwind.c:121)
==29908== 
==29908== 1,198 bytes in 1 blocks are still reachable in loss record 4 of 4
==29908==    at 0x483DD99: calloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==29908==    by 0x400D273: _dl_new_object (dl-object.c:89)
==29908==    by 0x4006E96: _dl_map_object_from_fd (dl-load.c:997)
==29908==    by 0x400A61A: _dl_map_object (dl-load.c:2236)
==29908==    by 0x4015D36: dl_open_worker (dl-open.c:513)
==29908==    by 0x49DF8B7: _dl_catch_exception (dl-error-skeleton.c:208)
==29908==    by 0x40155F9: _dl_open (dl-open.c:837)
==29908==    by 0x49DE860: do_dlopen (dl-libc.c:96)
==29908==    by 0x49DF8B7: _dl_catch_exception (dl-error-skeleton.c:208)
==29908==    by 0x49DF982: _dl_catch_error (dl-error-skeleton.c:227)
==29908==    by 0x49DE994: dlerror_run (dl-libc.c:46)
==29908==    by 0x49DE994: __libc_dlopen_mode (dl-libc.c:195)
==29908==    by 0x486E99A: pthread_cancel_init (unwind-forcedunwind.c:53)
==29908== 
==29908== LEAK SUMMARY:
==29908==    definitely lost: 0 bytes in 0 blocks
==29908==    indirectly lost: 0 bytes in 0 blocks
==29908==      possibly lost: 0 bytes in 0 blocks
==29908==    still reachable: 1,654 bytes in 4 blocks
==29908==         suppressed: 0 bytes in 0 blocks
==29908== 
==29908== For lists of detected and suppressed errors, rerun with: -s
==29908== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

复现规律 #

• 程序限制为1个线程运行时，从未出现该still reachable问题
• 线程数设为较低值（如2个）时问题偶现
• 线程数设为较高值（如48个）时问题几乎必现

最小复现代码 #

#include <stdlib.h>
#include <pthread.h>
#include <unistd.h>

#define NUMTHREADS  2 //can be anything higher than 1 to cause still reachables

void    *thread(void *arg)
{
    (void)arg;
    pthread_exit(EXIT_SUCCESS);
}

int main(void)
{
    int         i;
    pthread_t   threads[NUMTHREADS];

    i = -1;
    while (++i < NUMTHREADS)
        pthread_create(threads + i, NULL, &thread, NULL);
    i = -1;
    while (++i < NUMTHREADS)
        pthread_join(threads[i], NULL);
    return (0);
}

咨询问题 #

• 代码实现是否存在错误？
• 如果代码无问题、属于pthread的固有机制，为什么会产生still reachable内存？
• 为什么该问题会随线程数变化呈现半随机的出现特征？

1. 你的代码没有任何错误，这个内存残留是glibc pthread实现的正常固有行为，不属于需要修复的内存泄漏。
2. 从调用栈可以明确溯源内存来源：所有分配都来自pthread_cancel_init函数。这个函数是glibc的懒加载初始化逻辑，只有当线程退出/取消需要走强制栈展开流程时才会触发，作用是通过dlopen加载栈回溯依赖的libgcc_s.so动态库。
   dlopen加载动态库时，动态链接器会分配库对象结构、动态库缓存、版本校验表等内存，这些内存是进程级全局常驻资源，会一直保留到进程退出，不会在运行过程中释放。
   注意still reachable和真正的内存泄漏有本质区别：这类内存的指针没有丢失，进程全程都可以正常访问，只是库逻辑没有主动编写退出时的释放逻辑——这类内存大小固定不会持续增长，进程退出时操作系统会统一回收所有资源，完全不影响程序运行。
3. 出现概率和线程数相关的原因完全来自初始化触发的时序特性：
   • 单线程场景下，唯一的线程退出时就是进程最后一个线程，不需要走多线程场景下的强制栈展开流程，根本不会触发pthread_cancel_init，自然不会产生对应内存分配。
   • 线程数为2个的低线程数场景下，线程调度顺序是随机的：如果非最后退出的线程在进程完全处于正常运行阶段就走到退出栈展开逻辑，就会触发完整的初始化流程，分配对应内存；如果两个线程退出时序非常接近，触发初始化时进程已经开始走退出流程，dlopen流程会被提前终止，就不会完成那几块内存的分配，因此呈现偶现特征。
   • 线程数很高时，几乎必然存在线程在进程正常运行阶段就走到退出路径，触发完整的pthread_cancel_init流程，因此问题几乎必现。

这类glibc自身的常驻内存在通用系统上非常常见，valgrind官方默认的抑制规则库本来就包含这类pthread、dlopen相关的still reachable屏蔽项，你使用的Valgrind-3.15版本缺少对应版本glibc的抑制规则，才会把这部分内存报出，不需要做任何代码修改处理。

内容的提问来源于stack exchange，提问作者Modin