能否在Databricks中执行Azure Key Vault加解密操作？如何通过Scala或Python实现Azure Blobs的解密？

Absolutely! You can decrypt Azure Blob Storage blobs protected by Azure Key Vault right in Databricks using either Python or Scala—both languages have full support via Azure's official SDKs, which integrate seamlessly with Databricks' environment. Below are step-by-step examples for both languages, including setup and code snippets.

Python Implementation #

First, ensure you have the required Azure SDK libraries installed (Databricks may have some pre-installed, but you can add them explicitly if needed):

%pip install azure-storage-blob azure-keyvault-keys azure-identity

Then use this code to decrypt your blob:

from azure.identity import DefaultAzureCredential
from azure.keyvault.keys import KeyClient
from azure.storage.blob import BlobClient, BlobEncryptionClient

# Replace these values with your own resources
key_vault_url = "https://your-key-vault-name.vault.azure.net/"
key_name = "your-cmk-encryption-key"
storage_account_url = "https://your-storage-account.blob.core.windows.net/"
container_name = "your-target-container"
blob_name = "your-encrypted-blob.file"

# Authenticate to Azure (uses Databricks cluster MSI, service principal, or other valid auth methods)
credential = DefaultAzureCredential()

# Fetch the customer-managed key (CMK) from Key Vault
key_client = KeyClient(vault_url=key_vault_url, credential=credential)
encryption_key = key_client.get_key(key_name)

# Initialize Blob Client with encryption support
blob_client = BlobClient(
    account_url=storage_account_url,
    container_name=container_name,
    blob_name=blob_name,
    credential=credential
)

# Create encryption client to handle decryption
encryption_client = BlobEncryptionClient(blob_client, key=encryption_key)

# Download and decrypt the blob content
decrypted_bytes = encryption_client.download_blob().readall()

# Example: Convert bytes to string and save to DBFS
decrypted_text = decrypted_bytes.decode("utf-8")
dbutils.fs.put("/dbfs/mnt/decrypted/your-decrypted-file.txt", decrypted_text, overwrite=True)

Scala Implementation #

For Scala, we'll use Azure's Java SDK bindings (Databricks runtimes typically support these, but you can add dependencies if missing):

%maven com.azure:azure-storage-blob:12.20.0
%maven com.azure:azure-keyvault-keys:4.8.0
%maven com.azure:azure-identity:1.12.0

Here's the Scala code to decrypt your blob:

import com.azure.identity.DefaultAzureCredentialBuilder
import com.azure.security.keyvault.keys.KeyClientBuilder
import com.azure.storage.blob.{BlobClientBuilder, BlobEncryptionClientBuilder}

// Replace with your resource details
val keyVaultUrl = "https://your-key-vault-name.vault.azure.net/"
val keyName = "your-cmk-encryption-key"
val storageAccountUrl = "https://your-storage-account.blob.core.windows.net/"
val containerName = "your-target-container"
val blobName = "your-encrypted-blob.file"

// Authenticate to Azure (auto-detects cluster MSI, service principal, etc.)
val credential = new DefaultAzureCredentialBuilder().build()

// Retrieve the CMK from Key Vault
val keyClient = new KeyClientBuilder()
  .vaultUrl(keyVaultUrl)
  .credential(credential)
  .buildClient()
val encryptionKey = keyClient.getKey(keyName)

// Initialize Blob Client
val blobClient = new BlobClientBuilder()
  .accountUrl(storageAccountUrl)
  .containerName(containerName)
  .blobName(blobName)
  .credential(credential)
  .buildClient()

// Create encryption client for decryption
val encryptionClient = new BlobEncryptionClientBuilder()
  .blobClient(blobClient)
  .key(encryptionKey)
  .buildClient()

// Download and decrypt content
val decryptedBytes = encryptionClient.downloadContent().toBytes()

// Example: Convert to string and save to DBFS
val decryptedText = new String(decryptedBytes)
dbutils.fs.put("/dbfs/mnt/decrypted/your-decrypted-file-scala.txt", decryptedText, overwrite = true)

Critical Notes #

• Permissions Setup: Your Databricks cluster needs these IAM roles assigned:
  • For Azure Key Vault: Key Vault Crypto User (to perform decryption) + Key Vault Reader (to fetch key metadata)
  • For Azure Blob Storage: Storage Blob Data Reader (to access the encrypted blob)
• Authentication: DefaultAzureCredential automatically uses valid auth methods available in the Databricks environment (like cluster-managed identity, service principal credentials set in environment variables, etc.)
• Encryption Type: These examples work for blobs encrypted with Customer-Managed Keys (CMK) stored in Azure Key Vault. For Server-Side Encryption with CMK, the SDK handles decryption transparently when using the encryption client.

内容的提问来源于stack exchange，提问作者jukebox