AzCopy转换小体积Append Blob为Block Blob时遇InvalidBlobType错误

问题背景 #

需要将Append Blob转换为Block Blob，操作流程如下：

• 1. 使用AzCopy将目标Append Blob（例如publicstatistics_16.jsonl）复制为前缀带bb-的Block Blob
• 2. 再次使用AzCopy将这个Block Blob复制回原Append Blob的名称

遇到的异常：大小超过1000KB的Blob操作正常，但小于1000KB的Blob在步骤2失败，报错409 The blob type is invalid for this operation (InvalidBlobType)。手动下载上传小Blob、使用Python SDK操作时也出现相同错误。

AzCopy错误日志 #

2024/10/21 06:47:13 ==> REQUEST/RESPONSE (Try=1/69.6505ms, OpTime=69.771031ms) -- RESPONSE SUCCESSFULLY RECEIVED PUT https://storage-account.blob.core.windows.net/container/2024/filepath/bb-GOPACS-0.jsonl?si=rlwc&sig=-REDACTED-&sr=c&sv=2024-08-04 
Accept: application/xml Content-Length: 0 User-Agent: AzCopy/10.26.0 azsdk-go-azblob/v1.4.0 (go1.22.5; linux) X-Ms-Client-Request-Id: 
 x-ms-access-tier: Hot x-ms-blob-content-md5: x-ms-blob-content-type: application/jsonl;charset=UTF-8 
x-ms-blob-type: BlockBlob x-ms-copy-source: https://storage-account.blob.core.windows.net/container/2024/filepath/bb-GOPACS-0.jsonl?sas-token

2023-08-03 Status: 409 The blob type is invalid for this operation.Content-Length: 228Content-Type: application/xmlDate: Mon, 21 Oct 2024 06:47:13 GMTServer: Windows-Azure-Blob/1.0
 Microsoft-HTTPAPI/2.0X-Ms-Client-Request-Id: -Ms-Error-Code: InvalidBlobTypeX-Ms-Request-Id: -Ms-Version: 2023-08-03

2024/10/21 06:47:13 ERR: [P#0-T#0] COPYFAILED: https://storage-account.blob.core.windows.net/container/2024/filepath/bb-GOPACS-0.jsonl?si=rlwc&sig=-REDACTED-&sr=c&sv=2024-08-04/bb-GOPACS-0.jsonl : 409 : 409 The blob type is invalid for this operation.. When Put Blob from URL. X-Ms-Request-Id: d7601b94-801e-0030-4b85-23a96b000000

原Python SDK代码 #

def upload_small_block_blob(self, blob_service_client, container, blob_name):# Download the small blob using the SDK
   new_blob_client = container_client.get_blob_client(new_block_blob_name)
   downloaded_data = new_blob_client.download_blob().readall()
   blob_client = blob_service_client.get_blob_client(container=container, blob=blob_name)

        try:
            # Check if the blob already exists
            if blob_client.exists():
                log.error(f"Blob '{blob_name}' already exists. Choose a different name.")
                return
            # Upload the data as a new Block Blob
            blob_client.upload_blob(data)
            log.info(f"Uploaded new Block Blob '{blob_name}' successfully.")
        except Exception as e:
            log.error(f"Failed to upload new Block Blob '{blob_name}': {str(e)}")

错误原因 #

核心是目标位置的Blob类型与操作不匹配：

• 处理小文件时，AzCopy和Azure存储服务会默认使用Put Blob from URL直接创建完整Blob，若原Append Blob未被彻底删除（或残留类型标记），尝试覆盖为Block Blob会触发类型校验冲突。
• 小文件复制不会触发分块逻辑，直接的Put Blob操作会严格检查目标位置的Blob类型，而分块上传（Put Block + Put Block List）则无此限制。

解决方案 #

方案1：删除原Append Blob后再复制 #

执行步骤2前，先彻底删除原Append Blob：

azcopy rm "https://storage-account.blob.core.windows.net/container/2024/filepath/publicstatistics_16.jsonl?sas-token"

删除完成后再执行步骤2的复制操作。

方案2：强制AzCopy使用分块模式 #

通过指定块大小，强制小文件使用分块上传逻辑，绕过直接Put Blob的类型校验：

azcopy copy "https://storage-account.blob.core.windows.net/container/2024/filepath/bb-publicstatistics_16.jsonl?sas-token" "https://storage-account.blob.core.windows.net/container/2024/filepath/publicstatistics_16.jsonl?sas-token" --blob-type BlockBlob --block-size 1MiB

方案3：修正Python SDK代码 #

修复原代码的变量不一致、未指定Blob类型、未处理原Blob的问题：

def convert_append_to_block_blob(self, blob_service_client, container, original_blob_name):
    # 1. 创建临时Block Blob
    temp_block_blob_name = f"bb-{original_blob_name}"
    temp_blob_client = blob_service_client.get_blob_client(container=container, blob=temp_block_blob_name)
    
    # 从原Append Blob下载数据
    original_blob_client = blob_service_client.get_blob_client(container=container, blob=original_blob_name)
    downloaded_data = original_blob_client.download_blob().readall()
    
    # 上传为临时Block Blob
    temp_blob_client.upload_blob(data=downloaded_data, blob_type="BlockBlob", overwrite=True)
    
    # 2. 删除原Append Blob
    if original_blob_client.exists():
        original_blob_client.delete_blob()
    
    # 3. 将数据上传回原名称（指定BlockBlob类型）
    final_blob_client = blob_service_client.get_blob_client(container=container, blob=original_blob_name)
    final_blob_client.upload_blob(data=downloaded_data, blob_type="BlockBlob", overwrite=True)
    
    # 可选：清理临时Blob
    temp_blob_client.delete_blob()

关键改进点：

• 明确指定blob_type="BlockBlob"
• 上传前删除原Append Blob
• 直接复用下载的数据，避免二次复制的类型问题

内容的提问来源于stack exchange，提问作者Manoj N D