Hey there! Let’s work through this AWS KMS encryption issue you’re hitting—totally get why having to decrypt every record just to find a matching email is killing your performance. The root cause here is that AWS KMS uses envelope encryption by default: every encryption call generates a unique data key, so even identical plaintexts result in different ciphertexts. That makes direct lookups impossible with standard encryption.

Here are three practical, optimized solutions tailored to your use case, with code snippets for the Java SDK:

1. Use Deterministic Encryption (Direct Ciphertext Lookups) #

KMS supports deterministic encryption, which generates the same ciphertext for identical plaintexts (when using the same CMK and encryption context). This lets you query directly using the encrypted value.

How to Implement It: #

First, make sure your CMK is configured to allow deterministic encryption (you can set this when creating the CMK or update it later via the AWS Console/API). Then, enable it in your Java encryption call:

import com.amazonaws.services.kms.AmazonKMS;
import com.amazonaws.services.kms.AmazonKMSClientBuilder;
import com.amazonaws.services.kms.model.EncryptRequest;
import com.amazonaws.services.kms.model.EncryptResult;
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.util.Base64;

public class KmsDeterministicEncryption {
    public static void main(String[] args) {
        AmazonKMS kmsClient = AmazonKMSClientBuilder.standard().build();
        String cmkKeyId = "your-cmk-arn-or-id";
        String plaintextEmail = "user@example.com";

        EncryptRequest encryptRequest = new EncryptRequest()
                .withKeyId(cmkKeyId)
                .withPlaintext(ByteBuffer.wrap(plaintextEmail.getBytes(StandardCharsets.UTF_8)))
                .withDeterministicEncryption(true); // Critical: Enable deterministic mode

        EncryptResult result = kmsClient.encrypt(encryptRequest);
        byte[] ciphertext = result.getCiphertextBlob().array();
        String base64Ciphertext = Base64.getEncoder().encodeToString(ciphertext);

        // Store base64Ciphertext in your database
        // To query: encrypt the input email the same way, then look up the matching ciphertext
    }
}

Important Notes: #

• Security Tradeoff: Deterministic encryption is riskier for high-entropy data like passwords (vulnerable to dictionary attacks), but it’s acceptable for emails if you’re aware of the risk.
• Encryption Context: If you use an encryption context, it must be identical for both encryption and lookup—any change will produce a different ciphertext.

2. Store Salted Hashes (Best Balance of Security and Performance) #

Instead of relying on ciphertext for lookups, store a salted hash of the email alongside the encrypted email. When querying, hash the input email with the same salt and use that hash to find matching records, then decrypt only those records to verify.

How to Implement It: #

import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.SecureRandom;
import java.util.Base64;

public class EmailHashStorage {
    public static void main(String[] args) throws Exception {
        String plaintextEmail = "user@example.com";
        
        // Generate a unique salt for each user (store this with the hash)
        byte[] salt = new byte[16];
        new SecureRandom().nextBytes(salt);
        
        // Compute salted SHA-256 hash
        MessageDigest digest = MessageDigest.getInstance("SHA-256");
        digest.update(salt);
        byte[] hashBytes = digest.digest(plaintextEmail.getBytes(StandardCharsets.UTF_8));
        
        String emailHash = Base64.getEncoder().encodeToString(hashBytes);
        String emailSalt = Base64.getEncoder().encodeToString(salt);
        
        // Store emailHash, emailSalt, and KMS-encrypted email in your database
        
        // Query Workflow:
        String inputEmail = "user@example.com";
        // Fetch records where emailHash matches the computed hash of inputEmail + stored salt
        byte[] retrievedSalt = Base64.getDecoder().decode(emailSalt);
        digest.update(retrievedSalt);
        byte[] inputHashBytes = digest.digest(inputEmail.getBytes(StandardCharsets.UTF_8));
        String inputHash = Base64.getEncoder().encodeToString(inputHashBytes);
        
        // Use inputHash to query the database, then decrypt the matched encrypted emails to confirm
    }
}

Why This Works: #

• Salted hashes prevent rainbow table attacks, and you only decrypt the small subset of records that match the hash—way faster than full-table decryption.
• You keep using KMS’s default secure random encryption, so ciphertexts remain unique even for identical emails (avoiding the security risks of deterministic encryption).

3. Client-Side Key Derivation (Advanced, For Custom Workflows) #

If you need more control, you can use KMS to generate a master key, then derive consistent data keys for identical plaintexts using a key derivation function (KDF) like HKDF. This ensures same plaintext → same ciphertext while keeping the master key secured by KMS.

Quick Implementation Outline: #

1. Use KMS to generate a data key (via GenerateDataKey API).
2. Store the encrypted version of this data key (never store the plaintext key).
3. For encryption, use HKDF to derive a sub-key from the plaintext data key + plaintext email (or a unique identifier).
4. Use the derived sub-key to encrypt the email with a symmetric algorithm like AES-256-GCM.

This is more complex but gives you full control over encryption consistency. I’d only recommend this if the first two options don’t fit your needs.

Final Recommendation #

For most email lookup scenarios, the salted hash approach is the best bet—it’s secure, easy to implement, and avoids the tradeoffs of deterministic encryption. If you absolutely need to query directly by encrypted value, go with deterministic encryption but make sure to assess the security risks for your data.

内容的提问来源于stack exchange，提问作者ankit