使用AzCopy导入Azure表失败，请求排查问题原因

我在使用AzCopy从Blob Storage恢复Azure Tables备份时，遇到了部分表恢复成功、部分表失败的问题。以下是成功和失败的AzCopy日志，想请教下出现这个错误的原因是什么？

成功恢复的AzCopy日志 #

[2018/05/29 12:01:49.463+03:00][VERBOSE] 6.1.0 : AzCopy /Source:https://arenaxtablesbackup.blob.core.windows.net/arenaxstorage/2018-05-28 /Dest:https://arenaxstoragedev.table.core.windows.net/LeaderboardDaily /SourceKey:****** /Destkey:****** /Manifest:arenaxstorage_LeaderboardDaily.manifest /EntityOperation:InsertOrReplace /Z /V 
[2018/05/29 12:01:52.531+03:00][VERBOSE] Uploaded entities: 0 Downloaded blobs: 1, transferred bytes: 191 B 
[2018/05/29 12:01:52.758+03:00] Validating the integrity of data file list... 
[2018/05/29 12:01:52.974+03:00] Validation of data file list passed. 
[2018/05/29 12:01:53.208+03:00] Transfer summary: 
----------------- 
Total entities transferred: 0 
Elapsed time: 00.00:00:03 

[2018/05/29 12:04:57.932+03:00] >>>>>>>>>>>>>>>> 
[2018/05/29 12:04:57.937+03:00][VERBOSE] 6.1.0 : AzCopy /Source:https://arenaxtablesbackup.blob.core.windows.net/arenaxstorage/2018-05-28 /Dest:https://arenaxstoragedev.table.core.windows.net/LeaderboardUser /SourceKey:****** /Destkey:****** /Manifest:arenaxstorage_LeaderboardUser.manifest /EntityOperation:InsertOrReplace /Z /V 
[2018/05/29 12:05:01.189+03:00][VERBOSE] Uploaded entities: 0 Downloaded blobs: 1, transferred bytes: 209 B 
[2018/05/29 12:05:01.440+03:00] Validating the integrity of data file list... 
[2018/05/29 12:05:01.679+03:00] Validation of data file list passed. 
[2018/05/29 12:05:01.903+03:00][VERBOSE] Uploaded entities: 0 Downloaded blobs: 1 of 2, transferred bytes: 209 B 
[2018/05/29 12:05:01.903+03:00][VERBOSE] Uploaded entities: 0 Downloaded blobs: 1 of 2, transferred bytes: 209 B 
[2018/05/29 12:05:01.904+03:00][VERBOSE] Uploaded entities: 0 Downloaded blobs: 1 of 2, transferred bytes: 209 B 
[2018/05/29 12:05:01.904+03:00][VERBOSE] Uploaded entities: 0 Downloaded blobs: 1 of 2, transferred bytes: 209 B 
[2018/05/29 12:05:04.149+03:00][VERBOSE] Uploaded entities: 0 Downloaded blobs: 1 of 2, transferred bytes: 4 MB 
[2018/05/29 12:05:23.741+03:00][VERBOSE] Uploaded entities: 0 Downloaded blobs: 1 of 2, transferred bytes: 8 MB 
[2018/05/29 12:05:23.747+03:00][VERBOSE] Uploaded entities: 0 Downloaded blobs: 1 of 2, transferred bytes: 9.53 MB 
[2018/05/29 12:05:23.753+03:00][VERBOSE] Uploaded entities: 0 Downloaded blobs: 1 of 2, transferred bytes: 9.53 MB 
[2018/05/29 12:05:23.753+03:00][VERBOSE] Uploaded entities: 0 Downloaded blobs: 1 of 2, transferred bytes: 9.53 MB 
[2018/05/29 12:05:23.756+03:00][VERBOSE] Uploaded entities: 0 
[2018/05/29 12:05:23.823+03:00] Validating the integrity of data file "arenaxstorage_LeaderboardUser_0_0_C0DC932F526179D7.json"... 
[2018/05/29 12:05:23.849+03:00] Validation of data file "arenaxstorage_LeaderboardUser_0_0_C0DC932F526179D7.json" passed. 
[2018/05/29 12:05:26.463+03:00][VERBOSE] Uploaded entities: 100 
[2018/05/29 12:05:27.667+03:00][VERBOSE] Uploaded entities: 500 
... 
[2018/05/29 12:08:18.882+03:00][VERBOSE] Uploaded entities: 21600 
[2018/05/29 12:08:19.024+03:00][VERBOSE] Uploaded entities: 21704 
[2018/05/29 12:08:19.040+03:00] Transfer summary: 
----------------- 
Total entities transferred: 21704 
Elapsed time: 00.00:03:21 

失败恢复的AzCopy日志 #

[2018/05/29 12:10:44.537+03:00] >>>>>>>>>>>>>>>> 
[2018/05/29 12:10:44.543+03:00][VERBOSE] 6.1.0 : AzCopy /Source:https://arenaxtablesbackup.blob.core.windows.net/arenaxstorage/2018-05-28 /Dest:https://arenaxstoragedev.table.core.windows.net/Leaderboard /SourceKey:****** /Destkey:****** /Manifest:arenaxstorage_Leaderboard.manifest /EntityOperation:InsertOrReplace /Z /V 
[2018/05/29 12:10:47.659+03:00][VERBOSE] Uploaded entities: 0 Downloaded blobs: 1, transferred bytes: 205 B 
[2018/05/29 12:10:47.895+03:00] Validating the integrity of data file list... 
[2018/05/29 12:10:48.276+03:00][ERROR] Failed to import Azure table. Detailed error: Data validation failed. Data files are missing or have been renamed. 

问题原因分析 #

从失败日志里的错误信息Data validation failed. Data files are missing or have been renamed.，结合AzCopy恢复Azure Table的工作逻辑来看，核心问题是AzCopy无法找到arenaxstorage_Leaderboard.manifest文件中指定的备份数据文件。

AzCopy恢复Table时，会先读取指定的manifest文件——这个文件里记录了该Table备份对应的所有数据文件（通常是.json格式）的名称和路径。然后AzCopy会去源Blob路径下查找这些数据文件，如果找不到，或者文件名/路径和manifest里的记录不匹配，就会触发这个验证失败的错误。

对比成功恢复的LeaderboardUser日志可以看到：AzCopy在验证通过后，会下载对应的data文件（比如arenaxstorage_LeaderboardUser_0_0_C0DC932F526179D7.json），然后逐步上传实体；但失败的Leaderboard日志里，只下载了205B的manifest文件，之后验证数据文件列表直接失败，说明manifest里的data文件根本不在源路径里。

具体可能的原因包括：

• 备份数据文件缺失：Leaderboard表的备份过程可能中断或失败，导致对应的data文件没有生成，或者生成后被误删除/移动到了其他路径。
• manifest文件内容错误：arenaxstorage_Leaderboard.manifest文件可能被修改过，或者备份时生成的manifest里记录的data文件名/路径有误，和实际存储的文件不匹配。
• 大小写或拼写问题：Azure Blob存储的文件名是大小写敏感的，如果manifest里的文件名大小写和实际文件不一致，也会导致AzCopy找不到文件。
• 权限问题（可能性较低）：虽然其他表能正常恢复，但也可以确认下SourceKey是否有读取Leaderboard相关所有文件的权限，不过这种概率很小。

排查建议 #

1. 检查manifest文件内容：下载arenaxstorage_Leaderboard.manifest文件，打开查看里面列出的data文件名称，比如应该类似arenaxstorage_Leaderboard_xxx.json的格式。
2. 验证文件存在性：登录Azure Portal，进入源Blob容器arenaxtablesbackup/arenaxstorage/2018-05-28，查找manifest里记录的data文件是否存在，路径和文件名是否完全一致（注意大小写）。
3. 检查备份完整性：确认Leaderboard表的备份任务是否正常完成，有没有备份日志显示失败或中断的情况。
4. 确认源路径正确性：检查AzCopy命令里的/Source参数是否指向了包含所有备份文件的正确文件夹（这里是2018-05-28），有没有路径拼写错误。

内容的提问来源于stack exchange，提问作者Evgeniy