BCryptEncrypt WinAPI加密短消息触发C000000D参数无效错误

问题现象 #

使用WinAPI结合1024位PEM格式RSA公钥加密消息时：

• 当消息长度≥128字节，两次BCryptEncrypt调用均正常执行
• 当消息长度不足128字节，第二次BCryptEncrypt调用返回C000000D（STATUS_INVALID_PARAMETER）错误

相关代码如下：

//1024 bits public key
CHAR key[216] = "MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCAriYPFSlSVPprEN0jcg+K+AHMNAanHsfYELEIIrMRirFkzNQ3Zk+YSf1d3qZqGSqmUk6iaawMIh55Usfavnv9DCJtRnE9uHe1F1cgMnm+dug3dzOAiiSb5aIO5JqTK6LCtR+ccubCtw4geUGEk+nN0e0nk6BxRzjRIRGd5bJWDQIDAQAB";

//Get the length of the future base64 decoded public key
DWORD dwBufferLen = 0;
CryptStringToBinaryA(key, 0, CRYPT_STRING_BASE64, NULL, &dwBufferLen, NULL, NULL);

//Allocate a new buffer and redirect the output base64 decoded key to it
LPBYTE pbBuffer = NULL;
pbBuffer = (LPBYTE)LocalAlloc(0, dwBufferLen);

CryptStringToBinaryA(key, 0, CRYPT_STRING_BASE64, pbBuffer, &dwBufferLen, NULL, NULL);

//Get the size of the public key BLOB
DWORD cbKeyBlob = 0;
CryptDecodeObjectEx(X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, X509_PUBLIC_KEY_INFO, pbBuffer, dwBufferLen, 0, NULL, NULL, &cbKeyBlob);

//Allocate a new buffer and redirect the output key BLOB to it
LPBYTE pbKeyBlob = NULL;
pbKeyBlob = (LPBYTE)LocalAlloc(0, cbKeyBlob);

CryptDecodeObjectEx(X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, X509_PUBLIC_KEY_INFO, pbBuffer, dwBufferLen, 0, NULL, pbKeyBlob, &cbKeyBlob);

//Import that key BLOB to a BCRYPT_KEY_HANDLE
BCRYPT_KEY_HANDLE hKey = NULL;

CryptImportPublicKeyInfoEx2(X509_ASN_ENCODING, pbKeyBlob, 0, 0, &hKey);

//Get the size of the future encrypted message
char* secretMsg = "Hello this message will be sent to Alice";
ULONG secretMsgSize = strlen(secretMsg);

ULONG dwOutputSize = 0;

BCryptEncrypt(hKey, secretMsg, secretMsgSize, NULL, NULL, 0, NULL, NULL, &dwOutputSize , 0);

//Create a new buffer and allocate the dwOutputSize to the Heap
ULONG encryptedBufferSize = 0;
PUCHAR encryptedBuffer;
encryptedBuffer = (PUCHAR)HeapAlloc(GetProcessHeap(), 0, dwOutputSize);

BCryptEncrypt(hKey, secretMsg, secretMsgSize, NULL, NULL, 0, encryptedBuffer, dwOutputSize, &encryptedBufferSize, 0);

问题原因 #

1024位RSA的密钥长度为128字节，当未指定填充算法时，BCryptEncrypt默认采用无填充模式：

• 无填充模式要求输入明文长度必须等于密钥长度（128字节），因此当消息长度≥128字节时（实际取前128字节），调用可以正常执行
• 当消息长度不足128字节时，无填充模式不允许该输入，因此返回STATUS_INVALID_PARAMETER错误

解决方案 #

RSA加密必须使用填充算法（PKCS#1 v1.5或OAEP是常用标准），这里以PKCS#1 v1.5为例，修改BCryptEncrypt调用，明确指定填充参数：

1. 定义PKCS#1填充信息结构体：

BCRYPT_PKCS1_PADDING_INFO paddingInfo = { BCRYPT_RSA_ALGORITHM };

2. 修改两次BCryptEncrypt调用，传入填充信息并指定填充标志：

// 第一次调用获取输出长度
BCryptEncrypt(hKey, (PUCHAR)secretMsg, secretMsgSize, &paddingInfo, NULL, 0, NULL, NULL, &dwOutputSize, BCRYPT_PAD_PKCS1);

// 第二次调用执行加密
BCryptEncrypt(hKey, (PUCHAR)secretMsg, secretMsgSize, &paddingInfo, NULL, 0, encryptedBuffer, dwOutputSize, &encryptedBufferSize, BCRYPT_PAD_PKCS1);

注意事项 #

• 使用PKCS#1 v1.5填充时，1024位RSA的明文最大长度为128 - 11 = 117字节（11字节为填充占用空间），超过该长度的消息需要先使用对称加密算法加密，再用RSA加密对称密钥（这是RSA加密长消息的标准做法）
• 如果需要更高安全性，可以改用OAEP填充，对应结构体为BCRYPT_OAEP_PADDING_INFO，标志为BCRYPT_PAD_OAEP

修改后的完整代码片段 #

//1024 bits public key
CHAR key[216] = "MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCAriYPFSlSVPprEN0jcg+K+AHMNAanHsfYELEIIrMRirFkzNQ3Zk+YSf1d3qZqGSqmUk6iaawMIh55Usfavnv9DCJtRnE9uHe1F1cgMnm+dug3dzOAiiSb5aIO5JqTK6LCtR+ccubCtw4geUGEk+nN0e0nk6BxRzjRIRGd5bJWDQIDAQAB";

//Get the length of the future base64 decoded public key
DWORD dwBufferLen = 0;
CryptStringToBinaryA(key, 0, CRYPT_STRING_BASE64, NULL, &dwBufferLen, NULL, NULL);

//Allocate a new buffer and redirect the output base64 decoded key to it
LPBYTE pbBuffer = NULL;
pbBuffer = (LPBYTE)LocalAlloc(0, dwBufferLen);

CryptStringToBinaryA(key, 0, CRYPT_STRING_BASE64, pbBuffer, &dwBufferLen, NULL, NULL);

//Get the size of the public key BLOB
DWORD cbKeyBlob = 0;
CryptDecodeObjectEx(X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, X509_PUBLIC_KEY_INFO, pbBuffer, dwBufferLen, 0, NULL, NULL, &cbKeyBlob);

//Allocate a new buffer and redirect the output key BLOB to it
LPBYTE pbKeyBlob = NULL;
pbKeyBlob = (LPBYTE)LocalAlloc(0, cbKeyBlob);

CryptDecodeObjectEx(X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, X509_PUBLIC_KEY_INFO, pbBuffer, dwBufferLen, 0, NULL, pbKeyBlob, &cbKeyBlob);

//Import that key BLOB to a BCRYPT_KEY_HANDLE
BCRYPT_KEY_HANDLE hKey = NULL;

CryptImportPublicKeyInfoEx2(X509_ASN_ENCODING, pbKeyBlob, 0, 0, &hKey);

//Get the size of the future encrypted message
char* secretMsg = "Hello this message will be sent to Alice";
ULONG secretMsgSize = strlen(secretMsg);

ULONG dwOutputSize = 0;

// 定义PKCS#1填充信息
BCRYPT_PKCS1_PADDING_INFO paddingInfo = { BCRYPT_RSA_ALGORITHM };

// 获取加密输出长度（指定填充）
BCryptEncrypt(hKey, (PUCHAR)secretMsg, secretMsgSize, &paddingInfo, NULL, 0, NULL, NULL, &dwOutputSize, BCRYPT_PAD_PKCS1);

//Create a new buffer and allocate the dwOutputSize to the Heap
ULONG encryptedBufferSize = 0;
PUCHAR encryptedBuffer;
encryptedBuffer = (PUCHAR)HeapAlloc(GetProcessHeap(), 0, dwOutputSize);

// 执行加密（指定填充）
BCryptEncrypt(hKey, (PUCHAR)secretMsg, secretMsgSize, &paddingInfo, NULL, 0, encryptedBuffer, dwOutputSize, &encryptedBufferSize, BCRYPT_PAD_PKCS1);

内容的提问来源于stack exchange，提问作者Roderick Forest