使用RSA私钥解密15KB加密文件遇OpenSSL错误求助

Hey there, let's break down what's going wrong and how to fix this RSA decryption issue.

Why You're Seeing This Error #

The error RSA operation error 140360486789312:error:0406506C:rsa routines:rsa_ossl_private_decrypt:data greater than mod len is telling you that the data you're trying to decrypt is larger than your RSA key's modulus length.

RSA was never designed to encrypt/decrypt large files directly! The maximum data size you can process with RSA is roughly equal to your key's modulus length minus padding overhead (usually 11 bytes for PKCS#1 v1.5 padding). For example:

• A 2048-bit RSA key can only handle ~245 bytes of data
• A 1024-bit key tops out at ~117 bytes

Your 15KB encrypted file (plus the Base64 expansion which adds ~33% more size) is way beyond this limit — that's why OpenSSL is throwing an error.

Step-by-Step Fix #

1. Decode the Base64 First #

Your encrypted file contains a Base64-encoded string, so we need to convert it back to raw binary data first:

openssl base64 -d -in encrypted_file.enc -out encrypted_raw.bin

2. Understand the Correct Encryption Workflow #

Whoever encrypted this file almost certainly used the standard "hybrid encryption" approach:

1. A symmetric algorithm (like AES) was used to encrypt the large 15KB content
2. The symmetric key used for AES was then encrypted with your RSA public key
3. The encrypted AES key + encrypted content (and possibly an IV/nonce) were combined (and likely Base64-encoded) into your encrypted_file.enc

3. Split the Raw Encrypted Data #

You need to split encrypted_raw.bin into two parts:

• Encrypted AES key: This is the first N bytes of the file, where N is your RSA key's modulus length in bytes (256 bytes for 2048-bit keys, 128 bytes for 1024-bit keys)
• Encrypted content: The remaining bytes after the key (may also include an IV for AES, usually 16 bytes for CBC mode)

For example, if you have a 2048-bit key:

# Extract encrypted AES key (first 256 bytes)
dd if=encrypted_raw.bin of=encrypted_aes_key.bin bs=256 count=1
# Extract remaining content (skip first 256 bytes)
dd if=encrypted_raw.bin of=encrypted_content.bin bs=256 skip=1

4. Decrypt the AES Key with RSA #

Use your private key to decrypt the encrypted AES key:

openssl rsautl -decrypt -inkey ./id_rsa -in encrypted_aes_key.bin -out aes_key.bin

5. Decrypt the Content with AES #

Now use the decrypted AES key to unlock the actual content. You'll need to know the AES mode (CBC, GCM, etc.) and if an IV was used. For a common AES-256-CBC setup (with IV in the first 16 bytes of encrypted_content.bin):

# Extract IV (first 16 bytes)
dd if=encrypted_content.bin of=aes_iv.bin bs=16 count=1
# Extract actual encrypted content (skip first 16 bytes)
dd if=encrypted_content.bin of=actual_ciphertext.bin bs=16 skip=1

# Decrypt with AES-256-CBC
openssl aes-256-cbc -d -in actual_ciphertext.bin -out output.bin -K $(cat aes_key.bin | xxd -p) -iv $(cat aes_iv.bin | xxd -p)

Important Note for Future Use #

Never use RSA directly to encrypt large files! Always use hybrid encryption:

1. Generate a random symmetric key (AES is ideal)
2. Encrypt your large file with the symmetric key
3. Encrypt the symmetric key with the recipient's RSA public key
4. Send both the encrypted file + encrypted symmetric key to the recipient

内容的提问来源于stack exchange，提问作者gold foil