Windows Server 2019上传AWS S3 Glacier仍遇签名过期异常求助
AWS S3 Glacier大文件上传持续遭遇Signature Expired错误的排查方案
尝试将约15TB的大文件上传至AWS S3 Glacier,但持续遇到Signature Expired错误。已尝试两种时间设置方案:手动设置正确时钟并禁用时间同步,以及让Windows自动同步时间与时区,肉眼观察时钟显示正常。
编辑1
错误信息
Signature expired: 20231014T151454Z is now earlier than 20231014T151519Z (20231014T152019Z - 5 min.)
上传用VB代码(基于AWSSDK)
'4 GB per part Dim mPartSize = 4L * 1024L * 1024L * 1024L Dim InitiateUpload = Function() As Model.InitiateMultipartUploadResponse Dim GetInitialRequest = Function() As Model.InitiateMultipartUploadRequest Dim mRequest = New Model.InitiateMultipartUploadRequest With mRequest .VaultName = mVaultName .PartSize = mPartSize .ArchiveDescription = $"{mFileName}_{WhatObjectNamePrefix}" End With Return mRequest End Function Return mStorageClient.InitiateMultipartUpload(GetInitialRequest()) End Function Dim Upload = Function(WhatUploadID As String) As List(Of String) Dim mBinaryChecksums = New List(Of String) Dim mPosition = 0L While mPosition < WhatStream.Length Dim mStreamSize = Math.Min(mPartSize, WhatStream.Length - mPosition) Dim mTempStream = GlacierUtils.CreatePartStream(WhatStream, mStreamSize) Dim mUploadPartRequest = New Model.UploadMultipartPartRequest With mUploadPartRequest .VaultName = mVaultName .UploadId = WhatUploadID .Checksum = TreeHashGenerator.CalculateTreeHash(mTempStream) .Body = mTempStream .SetRange(mPosition, mPosition + mStreamSize - 1) End With mBinaryChecksums.Add(mUploadPartRequest.Checksum) mStorageClient.UploadMultipartPart(mUploadPartRequest) mPosition += mStreamSize End While Return mBinaryChecksums End Function Dim CompleteUpload = Sub(WhatUploadID As String, WhatChecksums As List(Of String)) Dim mCompleteRequest = New Model.CompleteMultipartUploadRequest With mCompleteRequest .VaultName = mVaultName .UploadId = WhatUploadID .ArchiveSize = WhatStream.Length.ToString .Checksum = TreeHashGenerator.CalculateTreeHash(WhatChecksums) End With mStorageClient.CompleteMultipartUpload(mCompleteRequest) End Sub Dim AbortUpload = Sub(WhatInitialReponse As Model.InitiateMultipartUploadResponse) Dim mRequest = New Model.AbortMultipartUploadRequest With mRequest .UploadId = WhatInitialReponse.UploadId End With mStorageClient.AbortMultipartUpload(mRequest) End Sub Dim mInitialResponse As Model.InitiateMultipartUploadResponse = Nothing Try mInitialResponse = InitiateUpload() Dim mCheckSums = Upload(mInitialResponse.UploadId) CompleteUpload(mInitialResponse.UploadId, mCheckSums) Catch ex As Exception If mInitialResponse IsNot Nothing Then Tryer(Sub() AbortUpload(mInitialResponse)) Throw New Exception($"Failed to upload file {mFileName}", ex) End Try
编辑2
参考AWS知识库建议改为异步上传,在抛出异常的UploadMultipartPart函数中添加了async/await,同步修改Sub Main以避免死锁,但问题依旧。
编辑3
调整代码提前计算树哈希(TreeHash),避免请求构造时等待哈希计算导致签名过期,但问题仍未解决。
请问还有哪些排查和解决方法?
内容的提问来源于stack exchange,提问作者Dan
相关产品推荐
相关产品推荐

