Docker容器中Python cryptography库OpenSSL绑定触发段错误的排查求助
Docker容器中Python cryptography库OpenSSL绑定触发段错误的排查求助
大家好,我最近碰到了一个特别棘手的问题,想请各位大佬帮忙出出主意:
我的应用运行在Docker容器里,在AWS的VM上一直稳定得很,但部署到GCP的VM后,部分容器会毫无征兆地因为Segmentation Fault错误崩溃,而且完全找不到规律——有时候能正常跑好几天,有时候没几天就挂了。
我已经拿到了崩溃时的core dump,但分析的时候遇到了不少瓶颈:
- 我把core dump挂载到和原容器一致的环境里,用
gdb导出了回溯信息,里面能看到OpenSSL的相关函数,这些函数是被cryptography包的OpenSSL C绑定调用的; - 我还启用了Python的faulthandler捕获崩溃栈追踪,最后两行也指向了OpenSSL相关的密钥加载操作。
现在我卡在这几个点上:
- 没法给容器里的Python 3.10装调试符号——用的是Debian 11镜像,默认用Python3.10,但
apt install python3.10-dbg根本找不到,只有Python3.9的调试包可用; - 搞不清宿主机到底怎么影响容器的,毕竟用的OpenSSL绑定是cryptography包自带的;
- 我一直不敢随便重新构建Docker镜像调试,怕改动后反而没法重现问题(毕竟同一个镜像在AWS上完全没问题),但如果有必要的话也可以尝试。
目前用到的版本信息:
- cryptography: 38.0.4
- OpenSSL(cryptography捆绑版本): 3.0.7
GDB回溯信息
# gdb /usr/local/bin/python3.10 /core.dump (gdb) bt full #0 0x0000f93ee103cd78 in _PyErr_SetObject () from /usr/local/bin/../lib/libpython3.10.so.1.0 No symbol table info available. #1 0x0000f93ee10c6a24 in _PyErr_SetString () from /usr/local/bin/../lib/libpython3.10.so.1.0 No symbol table info available. #2 0x0000f93ee1044244 in ?? () from /usr/local/bin/../lib/libpython3.10.so.1.0 No symbol table info available. #3 0x0000f93ede93d5d8 in OPENSSL_LH_doall_arg () from /usr/local/lib/python3.10/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so No symbol table info available. #4 0x0000f93ede93f02c in ossl_namemap_doall_names () from /usr/local/lib/python3.10/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so No symbol table info available. #5 0x0000f93edea58a20 in OSSL_ENCODER_CTX_new_for_pkey () from /usr/local/lib/python3.10/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so No symbol table info available. #6 0x0000f93ede9a1cac in i2d_PUBKEY () from /usr/local/lib/python3.10/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so No symbol table info available. #7 0x0000f93ede8b3390 in ASN1_i2d_bio () from /usr/local/lib/python3.10/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so No symbol table info available. #8 0x0000f93ede83c524 in _cffi_f_i2d_PUBKEY_bio (self=<optimized out>, args=<optimized out>) at build/temp.linux-aarch64-3.6/cryptography.hazmat.bindings._openssl.c:43855 _save = 0xbeefaa544190 x0 = 0xbeefcea289d0 x1 = 0xbeefaa605e00 datasize = <optimized out> large_args_free = 0x0 result = <optimized out> pyresult = <optimized out> arg0 = 0xf93afffc2020 arg1 = 0xf93edd594d10 #9 0x0000f93ee10374b4 in ?? () from /usr/local/bin/../lib/libpython3.10.so.1.0 No symbol table info available. #10 0x0000f93ee104130c in _PyObject_Call () from /usr/local/bin/../lib/libpython3.10.so.1.0 No symbol table info available.
Python faulthandler最后两行栈追踪
Current thread 0x0000f8e17e8bbd40 (most recent call first): File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1024 in load_der_public_key File "/usr/local/lib/python3.10/site-packages/cryptography/hazmat/primitives/serialization/base.py", line 56 in load_der_public_key
如果各位有新的调试思路、能帮我理清宿主机在这个问题里的作用,或者有一些可能的原因猜测,都请不吝赐教,真的非常感谢!
备注:内容来源于stack exchange,提问作者luislhl
相关产品推荐
相关产品推荐

