内核升级后Kernel Panic求助：从3.18.19到3.18.140

Let's break down your kernel panic issue step by step—since you're new to kernel debugging, I'll start with foundational context and move to actionable troubleshooting steps.

Background #

You upgraded your MT8163 kernel from upstream 3.18.19 to 3.18.140 (using the mt8163_kernel_upstreamed source tree) and now face a Kernel Panic triggered by the Compiler driver process. The kernel is marked as tainted (Tainted: G D), and you're unable to use objdump on vmlinux to map the call trace to source files.

Relevant panic snippet:

[ 118.822910] <0> (0)[3176:Compiler driver]CPU: 0 PID: 3176 Comm: Compiler driver Tainted: G D 3.18.140-KFHD8-XTREME #1
[ 118.822920] <0> (0)[3176:Compiler driver]Hardware name: MT8163 (DT)
[ 118.822929] <0> (0)[3176:Compiler driver][name:traps&]Call trace:
[ 118.822944] <0> (0)[3176:Compiler driver][] dump_backtrace+0x0/0x15c
[ 118.822956] <0> (0)[3176:Compiler driver][] show_stack+0x14/0x1c
[ 118.822968] <0> (0)[3176:Compiler driver][] dump_stack+0x88/0xac
[ 118.822980] <0> (0)[3176:Compiler driver][] handle_IPI+0x18c/0x2bc
[ 118.822991] <0> (0)[3176:Compiler driver][] gic_handle_irq+0x80/0x84

1. Decode the Kernel Taint Flags #

First, let's unpack what the G D taint flags mean:

• G: The kernel loaded a GPL-incompatible module (this might not be the direct panic cause, but it's a critical context to note)
• D: The kernel crashed due to a double fault or severe low-level error

The call trace pointing to handle_IPI (Inter-Processor Interrupt handler) tells us this is likely an issue with cross-CPU communication or interrupt controller handling.

2. Fix objdump Address-to-Source Mapping #

If objdump isn't resolving addresses to source files, you're missing key debug setup:

• Ensure your kernel was compiled with debug symbols: Check your .config for CONFIG_DEBUG_INFO=y (enable this before recompiling if missing)
• Use the correct architecture-specific toolchain: For ARM64 MT8163, use aarch64-linux-gnu-objdump (not the default x86 binary)
• Run this command to generate a mixed source/disassembly output:

  aarch64-linux-gnu-objdump -S vmlinux > vmlinux_disasm.txt
  

• To look up a specific address (e.g., ffffffc000093e0c for handle_IPI), use addr2line for faster results:

  aarch64-linux-gnu-addr2line -e vmlinux ffffffc000093e0c
  

  This will return the exact source file and line number for that function.

3. Debug the IPI-Related Panic #

Since the panic originates in handle_IPI, focus on interrupt and cross-CPU communication changes between your kernel versions:

• Check upstream commits: Use git to compare IPI-related code between 3.18.19 and 3.18.140:

  git log v3.18.19..v3.18.140 -- arch/arm64/kernel/irq.c
  

  Look for commits modifying IPI handling or GIC (the interrupt controller referenced in gic_handle_irq), especially those related to MT8163.
• Enable debug logging: Recompile your kernel with CONFIG_IPI_DEBUG=y and CONFIG_DEBUG_IRQ=y to capture detailed IPI event logs before the panic.
• Verify platform patches: Ensure all MT8163-specific interrupt handling patches from your 3.18.19 tree were correctly applied to 3.18.140. Older platform patches often break when ported to newer upstream point releases.

4. Investigate the Compiler driver Process #

While this process triggered the panic, it's likely a victim rather than the root cause. Still:

• Identify what the process does: On a working 3.18.19 system, run ps aux | grep "Compiler driver" to get its path and arguments.
• Check for API changes: Verify if the process uses kernel interfaces (syscalls, memory mapping, etc.) that were modified between 3.18.19 and 3.18.140.

5. Advanced Debugging Steps #

• Use the crash tool: If you can enable kdump and capture a crash dump, use the ARM64 crash utility to load vmlinux and the dump. This lets you inspect registers, stack state, and kernel memory at the time of the panic.
• Git bisect: If you have the full git history, use git bisect between v3.18.19 and v3.18.140 to find the exact commit that introduced the panic. This is one of the most effective ways to narrow down kernel upgrade issues.
• Check for memory corruption: Enable CONFIG_DEBUG_KERNEL and CONFIG_SLUB_DEBUG_ON=y to catch slab allocator corruption, which can cause random panics in interrupt handlers.

内容的提问来源于stack exchange，提问作者Kai Jones