解码后的Shellcode执行报错：Illegal instruction:4 技术求助

Let's break down what's going wrong here and how to fix it—this is a mix of environment mismatch and small implementation details tripping you up:

1. The Big One: You're Running Windows Shellcode on a Unix-Like System #

First off, your original shellcode is clearly Windows-targeted (look at the instruction sequence: \x6a pushes, \x0f\x05 is Windows x64 syscall, etc.), but your code uses mmap—a Unix/Linux system call. On top of that, you're using Metasploit's windows/base64.h for decoding.

When you run Windows-specific machine code on Linux (or any Unix-like OS), the CPU has no idea how to interpret those instructions correctly—they map to invalid operations for the system's ABI, hence the "Illegal instruction" error.

• If you want this to run on Windows: Ditch mmap and use Windows APIs like VirtualAlloc for memory allocation, and make sure your shellcode is compiled for Windows.
• If you're targeting Linux: Generate Linux-compatible shellcode (x86 or x64, match your system) instead of Windows shellcode, and use a Unix-friendly Base64 decoder if needed.

2. Base64 Decoding & Memory Copy Issues #

Even if you fix the environment mismatch, your current code has two critical flaws:

• Copying too much memory: You set bufferLen to 4096 and copy the entire buffer to executable memory, but your decoded shellcode is way shorter. This means you're copying uninitialized garbage data along with the valid shellcode. When the CPU finishes executing the shellcode, it'll run into that garbage and hit an illegal instruction.
  Fix this by capturing the actual decoded length (most Base64 decoders return this value) and only copy that many bytes:

  // Get actual decoded length (adjust based on your base64decode function's return value)
  int decoded_length = base64decode(buffer, shellcode, sizeof(shellcode) - 1); // Subtract null terminator
  memcpy(ptr, buffer, decoded_length);
  

• Misleading printf output: Your shellcode contains null bytes (\x00), which printf("%s", buffer) treats as a string terminator. So what you see printed is only the first part of the shellcode, not the full thing. To verify decoding is correct, print the buffer in hex:

  for (int i = 0; i < decoded_length; i++) {
      printf("\\x%02x", (unsigned char)buffer[i]);
  }
  printf("\n");
  

  This lets you compare the decoded byte-for-byte with your original shellcode to confirm it's intact.

3. Memory Permissions (Best Practice) #

While your mmap call sets PROT_READ | PROT_WRITE | PROT_EXEC, modern Linux systems often enforce W^X (Write XOR Execute) security policies—meaning memory shouldn't be both writable and executable at the same time. A safer approach is to:

1. Allocate writable/readable memory
2. Copy the shellcode
3. Flip permissions to readable/executable

void *ptr = mmap(NULL, decoded_length, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (ptr == MAP_FAILED) { perror("mmap failed"); exit(EXIT_FAILURE); }

memcpy(ptr, buffer, decoded_length);

if (mprotect(ptr, decoded_length, PROT_READ | PROT_EXEC) == -1) {
    perror("mprotect failed");
    munmap(ptr, decoded_length);
    exit(EXIT_FAILURE);
}

This avoids triggering security warnings and makes your code more compliant with modern system hardening.

内容的提问来源于stack exchange，提问作者Silent Person