iOS应用主线程卡死：pthread_from_mach_thread_np死循环排查

问题基本情况 #

正在排查一款iOS应用的主线程挂起问题，目前已在本地复现数次，但该问题复现概率较低、偶现性强，下文附上多份问题触发时采集到的堆栈样例。

初步现象分析 #

• 堆栈中可见dispatch_barrier_sync_f调用，该接口的作用是「提交屏障函数执行并等待函数执行完成」；问题触发时，大量其他线程均阻塞在ulock_wait状态。
• 所有挂起场景中，均存在一个执行dispatch_barrier_sync_f的活跃线程，经定位该线程卡在pthread_from_mach_thread_np函数内。目前仅能获取该函数的汇编代码，参考darwin-libpthread开源实现可知，该函数的核心逻辑是完成mach_thread到pthread的转换；单步调试汇编代码发现，程序卡在遍历pthread链表的循环逻辑中无法退出。
• 正常情况下该有限遍历循环不可能触发挂起，初步判断是内存损坏导致遍历的pthread链表非法，进而触发无限循环。该问题特征与苹果开发者论坛已公开的相关问题匹配。

问题堆栈样例 #

样例1 #

thread #104, queue = 'com.apple.coremedia.decompressionsession.clientcallback'
frame #0: 0x00000001f17fc100 libsystem_pthread.dylib`pthread_from_mach_thread_np + 68
frame #1: 0x0000000109139c98 libdispatch.dylib`_dispatch_introspection_continuation_get_info + 244
frame #2: 0x000000010913ae74 libdispatch.dylib`_dispatch_introspection_queue_item_enqueue_hook + 44
frame #3: 0x000000010913a42c libdispatch.dylib`_dispatch_introspection_queue_fake_sync_push_pop + 116
frame #4: 0x0000000109107f1c libdispatch.dylib`_dispatch_barrier_sync_f + 224
frame #5: 0x0000000197286720 VideoToolbox`__vtdsr_dequeueAllPendingFramesAndCallbackClientForEach_block_invoke + 1000
frame #6: 0x0000000109100c6c libdispatch.dylib`_dispatch_call_block_and_release + 32
frame #7: 0x00000001091027bc libdispatch.dylib`_dispatch_client_callout + 20
frame #8: 0x000000010910aa60 libdispatch.dylib`_dispatch_lane_serial_drain + 1428
frame #9: 0x000000010910b5e0 libdispatch.dylib`_dispatch_lane_invoke + 428
frame #10: 0x0000000109118168 libdispatch.dylib`_dispatch_workloop_worker_thread + 908
frame #11: 0x00000001f17f60bc libsystem_pthread.dylib`_pthread_wqthread + 288

样例2 #

thread #97
frame #0: 0x00000001f0b980ec libsystem_pthread.dylib`pthread_from_mach_thread_np + 48
frame #1: 0x00000001059f9c98 libdispatch.dylib`_dispatch_introspection_continuation_get_info + 244
frame #2: 0x00000001059fae74 libdispatch.dylib`_dispatch_introspection_queue_item_enqueue_hook + 44
frame #3: 0x00000001059fa42c libdispatch.dylib`_dispatch_introspection_queue_fake_sync_push_pop + 116
frame #4: 0x00000001059c7f1c libdispatch.dylib`_dispatch_barrier_sync_f + 224
frame #5: 0x0000000187fbbf54 CoreData`_perform + 176
frame #6: 0x0000000187e23a28 CoreData`-[NSPersistentStoreCoordinator _routeLightweightBlock:toStore:] + 204
frame #7: 0x0000000187ea71f4 CoreData`-[NSPersistentStoreCoordinator(_NSInternalMethods) newValuesForObjectWithID:withContext:error:] + 368
frame #8: 0x0000000187e89f7c CoreData`_PFFaultHandlerLookupRow + 296
frame #9: 0x0000000187ec357c CoreData`_PF_FulfillDeferredFault + 208
frame #10: 0x0000000187e25fc8 CoreData`_PF_ManagedObject_WillChangeValueForKeyIndex + 84
frame #11: 0x0000000187e3f260 CoreData`_sharedIMPL_setvfk_core + 152
frame #12: 0x000000010128f408 MyApp`-[GMSTileDataCache touchCachedTile:] + 100
frame #13: 0x000000010128fb7c MyApp`__72-[GMSTileDataCache loadTileForTileCoords:dataVersion:completionHandler:]_block_invoke + 216
frame #14: 0x0000000187e18834 CoreData`developerSubmittedBlockToNSManagedObjectContextPerform + 156
frame #15: 0x00000001059c27bc libdispatch.dylib`_dispatch_client_callout + 20
frame #16: 0x00000001059ca8a4 libdispatch.dylib`_dispatch_lane_serial_drain + 984
frame #17: 0x00000001059cb5e0 libdispatch.dylib`_dispatch_lane_invoke + 428
frame #18: 0x00000001059d8168 libdispatch.dylib`_dispatch_workloop_worker_thread + 908
frame #19: 0x00000001f0b920bc libsystem_pthread.dylib`_pthread_wqthread + 288

样例3 #

thread #99, queue = 'flight logger queue'
frame #0: 0x00000001f16040f8 libsystem_pthread.dylib`pthread_from_mach_thread_np + 60
frame #1: 0x0000000105909c98 libdispatch.dylib`_dispatch_introspection_continuation_get_info + 244
frame #2: 0x000000010590b0d0 libdispatch.dylib`_dispatch_introspection_queue_item_dequeue_hook + 44
frame #3: 0x000000010590a440 libdispatch.dylib`_dispatch_introspection_queue_fake_sync_push_pop + 136
frame #4: 0x00000001058d7f1c libdispatch.dylib`_dispatch_barrier_sync_f + 224
frame #5: 0x000000019ac06cac libswiftDispatch.dylib`merged implicit closure #2 (() -> ()) -> () in implicit closure #1 (__C.OS_dispatch_queue) -> (() -> ()) -> () in __C.OS_dispatch_queue.sync<τ_0_0>(execute: () throws -> τ_0_0) throws -> τ_0_0 + 180
frame #6: 0x000000019ac05d0c libswiftDispatch.dylib`partial apply forwarder for implicit closure #2 (() -> ()) -> () in implicit closure #1 (__C.OS_dispatch_queue) -> (() -> ()) -> () in __C.OS_dispatch_queue.sync<τ_0_0>(execute: () throws -> τ_0_0) throws -> τ_0_0 + 56
frame #7: 0x000000019ac069ec libswiftDispatch.dylib`__C.OS_dispatch_queue._syncHelper<τ_0_0>(fn: (() -> ()) -> (), execute: () throws -> τ_0_0, rescue: (Swift.Error) throws -> τ_0_0) throws -> τ_0_0 + 396
frame #8: 0x000000019ac05dbc libswiftDispatch.dylib`__C.OS_dispatch_queue.sync<τ_0_0>(execute: () throws -> τ_0_0) throws -> τ_0_0 + 168
frame #9: 0x0000000100324f8c MyApp`FlightData.flightDataComponents(self=0x000000010645aee0) at FlightData.swift:184:35
frame #10: 0x000000010029c014 MyApp`DDLogDataFactory.generateLogDataComponent(now=2022-06-30 22:54:33 UTC, flightData=0x000000010645aee0, self=0x0000000287465280) at DDLogDataFactory.swift:197:44
frame #11: 0x0000000100364688 MyApp`DDFlightLogFormatterV1.flightData(flightData=0x000000010645aee0, self=0x0000000283461040) at DDFlightLogFormatterV1.swift:45:57
frame #12: 0x0000000100364768 MyApp`@objc DDFlightLogFormatterV1.flightData(_:) at <compiler-generated>:0
frame #13: 0x00000001001d06b4 MyApp`__31-[FlightLogger logCurrentData:]_block_invoke(.block_descriptor=0x0000000283bc01b0) at FlightLogger.m:408:30
frame #14: 0x00000001001d0e40 MyApp`__41-[FlightLogger executeBlockInBackground:]_block_invoke(.block_descriptor=0x0000000283bc22b0) at FlightLogger.m:457:9
frame #15: 0x00000001058d0c6c libdispatch.dylib`_dispatch_call_block_and_release + 32
frame #16: 0x00000001058d27bc libdispatch.dylib`_dispatch_client_callout + 20
frame #17: 0x00000001058da8a4 libdispatch.dylib`_dispatch_lane_serial_drain + 984
frame #18: 0x00000001058db5e0 libdispatch.dylib`_dispatch_lane_invoke + 428
frame #19: 0x00000001058e8168 libdispatch.dylib`_dispatch_workloop_worker_thread + 908
frame #20: 0x00000001f15fe0bc libsystem_pthread.dylib`_pthread_wqthread + 288

可参考的排查方向 #

• 优先排查内存踩踏问题：挂起根因是pthread链表被破坏形成环，导致遍历无限循环。重点排查是否存在越界写、野指针写入、重复释放、非线程安全的跨线程对象操作这类常见内存问题，可开启Address Sanitizer、Zombie Objects、Thread Sanitizer在复现场景下测试，这类工具对偶现内存问题的捕获率较高。
• 注意Dispatch Introspection的触发条件：从堆栈看，卡顿时所有调用栈都经过了_dispatch_introspection_*系列函数，这类函数只有在挂载了调试工具、或者开启了dispatch队列监控的时候才会执行，排查时可先确认是否是调试工具本身触发了系统库的已知问题，可尝试在非调试模式下复现，看问题是否仍然存在。
• 核对系统版本匹配度：该类pthread链表损坏导致的死循环在部分iOS 15、iOS 16早期版本有公开的系统库bug记录，可统计出现问题的设备系统版本分布，如果集中在特定系统版本，可通过规避对应dispatch_barrier_sync的高频调用场景降低触发概率。
• 跨组件并发操作校验：三个堆栈分别涉及VideoToolbox回调、CoreData跨队列操作、Swift DispatchQueue同步调用，都是跨线程并发的高频场景，重点排查这几个业务模块是否存在队列误用、同步调用嵌套导致的优先级反转、未加保护的共享内存写入问题。

内容的提问来源于stack exchange，提问作者Andrew Delpit