如何排查OpenSSL引擎程序退出时的段错误？

一、基础问题现象 #

开发的OpenSSL引擎运行阶段功能正常，但程序退出时触发段错误：

$ ./output/bin/async_test
1..1
PASS
[1]    24048 segmentation fault (core dumped)  ./output/bin/async_test

通过GDB分析core文件，栈回溯显示段错误发生在动态库卸载阶段：

$ gdb ./output/bin/async_test core.24048
# ...省略无关警告信息...
Program terminated with signal 11, Segmentation fault.
#0  0x00007f855b8ddfe8 in _dl_close_worker (map=map@entry=0x825b90) at dl-close.c:148
148       for (struct link_map *l = ns->_ns_loaded; l != NULL; l = l->l_next)
(gdb) bt
#0  0x00007f855b8ddfe8 in _dl_close_worker (map=map@entry=0x825b90) at dl-close.c:148
#1  0x00007f855b8dedac in _dl_close (_map=0x825b90) at dl-close.c:776
#2  0x00007f855b8d87d4 in _dl_catch_error (objname=0x825b40, errstring=0x825b48, mallocedp=0x825b38, operate=0x7f855adca060 <dlclose_doit>, args=0x825b90)
    at dl-error.c:177
#3  0x00007f855adca5ad in _dlerror_run (operate=operate@entry=0x7f855adca060 <dlclose_doit>, args=0x825b90) at dlerror.c:163
#4  0x00007f855adca08f in __dlclose (handle=<optimized out>) at dlclose.c:47
#5  0x00007f855b4bbcf9 in dlfcn_unload (dso=0x8259c0) at crypto/dso/dso_dlfcn.c:162
#6  0x00007f855b4bc59c in DSO_free (dso=0x8259c0) at crypto/dso/dso_lib.c:78
#7  0x00007f855b51684e in dynamic_data_ctx_free_func (parent=0x8256f0, ptr=0x825850, ad=0x825798, idx=1, argl=0, argp=0x0) at crypto/engine/eng_dyn.c:140
#8  0x00007f855b543f37 in CRYPTO_free_ex_data (class_index=10, obj=0x8256f0, ad=0x825798) at crypto/ex_data.c:354
#9  0x00007f855b51814d in engine_free_util (e=0x8256f0, not_locked=0) at crypto/engine/eng_lib.c:95
#10 0x00007f855b518952 in engine_list_remove (e=0x8256f0) at crypto/engine/eng_list.c:125
#11 0x00007f855b518d66 in ENGINE_remove (e=0x8256f0) at crypto/engine/eng_list.c:239
#12 0x00007f855b518616 in engine_list_cleanup () at crypto/engine/eng_list.c:38
#13 0x00007f855b518308 in engine_cleanup_cb_free (item=0x80e4c0) at crypto/engine/eng_lib.c:162
#14 0x00007f855b5c4d9b in OPENSSL_sk_pop_free (st=0x80e490, func=0x7f855b5182f3 <engine_cleanup_cb_free>) at crypto/stack/stack.c:368
#15 0x00007f855b517e40 in sk_ENGINE_CLEANUP_ITEM_pop_free (sk=0x80e490, freefunc=0x7f855b5182f3 <engine_cleanup_cb_free>) at crypto/engine/eng_int.h:54
#16 0x00007f855b51834b in engine_cleanup_int () at crypto/engine/eng_lib.c:169
#17 0x00007f855b547d7a in OPENSSL_cleanup () at crypto/init.c:601
#18 0x00007f855b006ce9 in __run_exit_handlers () from /lib64/libc.so.6
#19 0x00007f855b006d37 in exit () from /lib64/libc.so.6
#20 0x00007f855afef55c in __libc_start_main () from /lib64/libc.so.6
#21 0x00000000004023c9 in _start ()
(gdb)quit

二、ASAN工具排查异常 #

怀疑是内存越界破坏了libcrypto.so的内存结构，添加ASAN编译选项：

TARGET_LINK_LIBRARIES(qat ${crypto_path} libhct -fsanitize=address -fsanitize-recover=address -fno-omit-frame-pointer)

运行后ASAN在非预期位置触发终止：

build git:702a8b8 # ./output/bin/async_test
1..1
ASAN:DEADLYSIGNAL
==================================================================
==20978==ERROR: AddressSanitizer: SEGV on unknown address 0x00c09fff8000 (pc 0x7f891c950e27 bp 0x62d000012530 sp 0x62d000011cc0 T0)
    #0 0x7f891c950e26 in AddressIsPoisoned ../../.././libsanitizer/asan/asan_mapping.h:295
    #1 0x7f891c950e26 in QuickCheckForUnpoisonedRegion ../../.././libsanitizer/asan/asan_interceptors.cc:40
    #2 0x7f891c950e26 in __interceptor_memset ../../.././libsanitizer/asan/asan_interceptors.cc:451
    #3 0x7f8918453dad in cmd_passthru /home/usr2/git/tmp2/qat_crypto.c:126
    #4 0x7f89184ee4f9 in cipher_do_cipher /home/usr2/git/tmp2/qat_sm4.c:235
    #5 0x7f89187f069a in cipher_do_cipher /home/usr2/git/engine/e_qat.c:2335
    #6 0x7f891c57a7cc in evp_EncryptDecryptUpdate crypto/evp/evp_enc.c:333
    #7 0x7f891c57aa17 in EVP_EncryptUpdate crypto/evp/evp_enc.c:385
    #8 0x402b2a in enc_sw_dec /home/usr2/tmp2/async_test.c:95
    #9 0x7f891c485bfc in async_start_func crypto/async/async.c:152
    #10 0x7f891c05718f  (/lib64/libc.so.6+0x4818f)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ../../.././libsanitizer/asan/asan_mapping.h:295 in AddressIsPoisoned
==20978==ABORTING

三、尝试跳过ASAN检测无效 #

给cmd_passthru函数添加no_sanitize_address属性：

#if defined(__clang__) || defined (__GNUC__)
# define ATTRIBUTE_NO_SANITIZE_ADDRESS __attribute__((no_sanitize_address))
#else
# define ATTRIBUTE_NO_SANITIZE_ADDRESS
#endif

ATTRIBUTE_NO_SANITIZE_ADDRESS
int cmd_passthru(void *saddr,uint32_t slen,uint32_t first,uint32_t final)
{
...
}

重新编译运行后，ASAN仍报相同错误。

四、解决方案建议 #

1. 正确跳过ASAN检测的方法 #

• 确保函数的声明和定义都添加ATTRIBUTE_NO_SANITIZE_ADDRESS属性，编译器才能全局识别跳过该函数的ASAN检测。
• 针对qat_crypto.c单独设置编译选项，在CMake中添加：

  set_source_files_properties(qat_crypto.c PROPERTIES COMPILE_FLAGS "-fno-sanitize=address")
  

  这种方式会直接关闭整个编译单元的ASAN检测，比函数属性更可靠。

2. 排查段错误根源的其他方法 #

• 使用Valgrind Memcheck：Valgrind可以精准检测内存越界、重复释放、使用已释放内存等问题，适合退出阶段的段错误排查：

  valgrind --leak-check=full --track-origins=yes ./output/bin/async_test
  

• 检查OpenSSL引擎的资源清理逻辑：从栈回溯看，段错误发生在引擎清理和动态库卸载阶段，重点检查：
  • 引擎的销毁函数（通过ENGINE_set_destroy_function设置）中，是否存在重复释放资源、释放未分配内存的情况。
  • 引擎是否使用了全局静态资源，退出时未正确清理，导致OpenSSL后续清理时访问无效内存。
  • 确认所有通过OpenSSL内存分配函数（如CRYPTO_malloc、OPENSSL_malloc）分配的内存，都通过对应函数（CRYPTO_free、OPENSSL_free）释放，没有混用标准库的free。
• 启用OpenSSL调试日志：设置环境变量打印清理阶段的详细日志，定位异常步骤：

  OPENSSL_DEBUG_LEVEL=3 ./output/bin/async_test
  

• 检查cmd_passthru中的内存操作：针对触发ASAN的memset，确认saddr指针的有效性，以及slen是否超出了分配的内存大小，避免越界写入破坏OpenSSL内部的数据结构。

内容的提问来源于stack exchange，提问作者HnlyWk