Azure跨ADLS Gen2存储账户复制Blob SAS可用其他授权报公共访问错误

我需要在两个同类型的ADLS Gen2存储账户之间复制Blob，目前使用SAS令牌授权的代码可正常运行，代码如下：

import com.microsoft.azure.storage.{CloudStorageAccount,StorageException,StorageCredentials,StorageCredentialsToken}
import com.microsoft.azure.storage.blob.{CloudBlobClient,CloudBlockBlob,CloudBlobDirectory,CloudBlobContainer,ListBlobItem}

val srcStorageAccountName = "<src-account-name>"
val destStorageAccountName = "<dest-account-name>"


// SAS 授权方式
val srcSas = "<src-sas-token>"
val srcCloudStorageAccount= CloudStorageAccount.parse(s"DefaultEndpointsProtocol=https;AccountName=${srcStorageAccountName};SharedAccessSignature=${srcSas};EndpointSuffix=core.windows.net")

val destSas = "<dest-sas-token>"
val destCloudStorageAccount= CloudStorageAccount.parse(s"DefaultEndpointsProtocol=https;AccountName=${destStorageAccountName};SharedAccessSignature=${destSas};EndpointSuffix=core.windows.net")

val destBlobClient = destCloudStorageAccount.createCloudBlobClient()
val srcBlobClient = srcCloudStorageAccount.createCloudBlobClient()

try {
  val srcPath = "<src-path>"
  val destPath = "<dest-path>"
  val srcContainerName = "<src-container>"
  val destContainerName = "<dest-container>"
  val srcContainer = srcBlobClient.getContainerReference(srcContainerName)
  val destContainer = destBlobClient.getContainerReference(destContainerName)
  val blobs = srcContainer.listBlobs(srcPath)
  import scala.jdk.CollectionConverters._
  copyBlobList(blobs.asScala, destContainer, srcContainer, srcContainerName, srcPath, destPath)
} catch {
  case e: StorageException =>
    e.printStackTrace()
}

def copyBlobList(blobs: Iterable[ListBlobItem], destContainer: CloudBlobContainer, srcContainer: CloudBlobContainer, srcContainerName: String, srcPath: String, destPath: String): Unit = {
  for (blob <- blobs) {
    blob match {
      case blockBlob: CloudBlockBlob => copySingleBlob(blockBlob, destContainer, srcContainer, srcContainerName, srcPath, destPath)
      case blobDirectory: CloudBlobDirectory => copyBlobDirectory(blobDirectory, destContainer, srcContainer, srcContainerName, srcPath, destPath)
      case _ => println(s"未知Blob类型")
    }
  }
}

def copySingleBlob(blob: CloudBlockBlob, destContainer: CloudBlobContainer, srcContainer: CloudBlobContainer, srcContainerName: String, srcPath: String, destPath: String): Unit = {
  val srcBlob = srcContainer.getBlockBlobReference(blob.getUri.getPath.replaceFirst(srcContainerName, "").substring(1).replace(srcPath, destPath))
  val destBlob = destContainer.getBlockBlobReference(blob.getUri.getPath.replaceFirst(srcContainerName, "").substring(1).replace(srcPath, destPath))
  destBlob.startCopy(srcBlob)
}

def copyBlobDirectory(blobDirectory: CloudBlobDirectory, destContainer: CloudBlobContainer, srcContainer: CloudBlobContainer, srcContainerName: String, srcPath: String, destPath: String): Unit = {
  val blobsFromDir = srcContainer.listBlobs(blobDirectory.getUri.getPath.replace(srcContainerName, ""))
  import scala.jdk.CollectionConverters._
  copyBlobList(blobsFromDir.asScala, destContainer, srcContainer, srcContainerName, srcPath, destPath)
}

使用非SAS授权方式时，会抛出如下异常：

com.microsoft.azure.storage.StorageException: Public access is not permitted on this storage account.

已尝试的两种授权方式代码如下：

// 账户密钥授权
val destKey = "<dest-account-key>"
val srcKey =  "<src-account-key>"

val srcCloudStorageAccount = CloudStorageAccount.parse(s"DefaultEndpointsProtocol=https;AccountName=${srcStorageAccountName};AccountKey=${srcKey};EndpointSuffix=core.windows.net")
val destCloudStorageAccount = CloudStorageAccount.parse(s"DefaultEndpointsProtocol=https;AccountName=${destStorageAccountName};AccountKey=${destKey};EndpointSuffix=core.windows.net")

// 访问令牌授权
val srcTokenCredentials = new StorageCredentialsToken(srcStorageAccountName, srcToken)
val srcCloudStorageAccount = new CloudStorageAccount(srcTokenCredentials, true)

val destTokenCredentials = new StorageCredentialsToken(destStorageAccountName, destToken)
val destCloudStorageAccount = new CloudStorageAccount(destTokenCredentials, true)

上述两种授权方式均支持同一存储账户内的Blob复制，但执行跨存储账户复制操作时就会触发前述报错。

问题原因 #

出现这个差异的核心是Azure Blob服务端跨账号复制的执行机制：

• startCopy触发的跨账号复制是目标存储账户的服务端主动向源存储账户拉取Blob内容，全程不需要本地客户端中转数据。
• 用SAS令牌授权时，构造的源Blob引用本身就携带了合法的鉴权签名，目标存储服务拉取源文件时会直接带上这个SAS签名，源存储服务验证SAS合法就会放行，不需要源开启公共访问，所以能正常运行。
• 用账户密钥、AAD访问令牌授权时，这两类凭证属于本地客户端持有的鉴权信息，不会被SDK自动附加到目标端发起的服务端拉取请求上。目标存储服务发起拉取请求时拿不到源端的访问凭证，只能尝试匿名访问源Blob，只要源存储账户禁用了公共匿名访问，就会抛出“Public access is not permitted on this storage account”异常。
• 同账号内复制不会触发这个问题，是因为同账号复制属于存储服务内部操作，不需要走公网拉取流程，服务内部直接完成权限校验和数据拷贝。

修复方案 #

如果要使用账户密钥或者访问令牌实现跨账号复制，不要直接把本地凭证生成的源Blob对象传给startCopy，按以下步骤调整即可：

• 用源端的密钥/令牌凭证，为待复制的源Blob生成至少包含Read权限的SAS，有效期要覆盖整个复制操作的周期，复制大文件建议适当延长有效期
• 拼接源Blob的完整访问URI和生成的SAS令牌，构造新的源Blob引用对象，再传给目标Blob的startCopy方法
• 复制大文件时建议轮询复制进度，避免SAS过期导致复制任务中断

内容的提问来源于stack exchange，提问作者Bartek Maciejewski