RedHat7.2程序调用popen触发非法指令核心转储，RIP指令异常

Hey folks, let's break down this odd issue where calling popen() on RedHat 7.2 triggers a Signal 4 (Illegal instruction) core dump, and the RIP register points to what looks like a non-existent instruction in glibc. First, here's the debug output we're working with:

Program terminated with signal 4, Illegal instruction.
#0 0x00007fb2f217b1f4 in popen@@GLIBC_2.2.5 () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-105.el7.x86_64 libgcc-4.8.5-4.el7.x86_64 libstdc++-4.8.5-4.el7.x86_64 zlib-1.2.7-15.el7.x86_64

Let's go through the most likely fixes and checks step by step:

1. First, install the missing debuginfos #

The debug message explicitly tells us we're missing debuginfo packages—this is critical because without them, gdb can't properly map memory addresses to actual assembly or source code. The "non-existent instruction" you're seeing is probably just gdb guessing because it doesn't have the right symbols.

Run this exact command to install the required debuginfos:

debuginfo-install glibc-2.17-105.el7.x86_64 libgcc-4.8.5-4.el7.x86_64 libstdc++-4.8.5-4.el7.x86_64 zlib-1.2.7-15.el7.x86_64

Once installed, re-load the core dump in gdb—you'll get much clearer info about where the crash is actually happening.

2. Check if glibc is corrupted or modified #

A SIGILL in a standard glibc function like popen() is almost never a bug in glibc itself (especially in a stable release like 2.17 on RHEL7). The far more likely scenario is that the /lib64/libc.so.6 library file is corrupted, or was accidentally modified.

Verify the integrity of the glibc package with:

rpm -V glibc-2.17-105.el7.x86_64

If you see flags like 5 (checksum mismatch) or S (file size mismatch), the library is definitely corrupted. Reinstall glibc to fix this:

yum reinstall glibc-2.17-105.el7.x86_64

3. Rule out CPU compatibility issues #

While LEA instructions are standard on x86_64, it's possible that your system (especially a virtual machine) is running on a CPU that doesn't support required extensions, or the hypervisor is hiding those extensions from the guest.

Check your CPU's supported features with:

cat /proc/cpuinfo | grep flags

For RHEL7, you should see at minimum sse, sse2, and mmx in the output. If you're on a VM, double-check that the hypervisor is configured to expose a compatible CPU model to the guest.

4. Look for memory corruption in your program #

Memory corruption (like buffer overflows, stack smashes, or use-after-frees) can overwrite critical parts of the program's memory—including the PLT/GOT entries that point to glibc functions, or even the code segment of glibc itself. This can cause the program counter (RIP) to jump to the middle of an instruction, leading to a SIGILL.

Use valgrind to hunt for memory errors:

valgrind --leak-check=full --track-origins=yes ./your-program

Once you have debuginfos installed, you can also disassemble the popen function in gdb to see what's actually at that address:

(gdb) disassemble popen@@GLIBC_2.2.5

If the RIP is pointing to the middle of a valid instruction, that's a dead giveaway that memory corruption caused the program counter to misalign.

5. Check for interfering LD_PRELOAD libraries #

Sometimes, third-party libraries loaded via the LD_PRELOAD environment variable can hook or interfere with glibc functions, causing unexpected crashes.

Check if any libraries are being preloaded:

echo $LD_PRELOAD

If there's output here, try running your program without the preload to see if the crash goes away:

unset LD_PRELOAD && ./your-program

内容的提问来源于stack exchange，提问作者Jian Zhang