FTP服务器CAGED文件读取报错:Corrupt input data问题求助
解压FTP下载的CAGED 7z微数据文件时出现「Corrupt input data」错误
问题背景
从指定FTP服务器下载CAGED项目的7z格式微数据文件后,解压读取阶段触发「Corrupt input data」错误,无法定位问题根源是代码还是FTP服务器:
- FTP地址:
ftp://ftp.mtps.gov.br/pdet/microdados/ - 数据字典文件路径:
ftp://ftp.mtps.gov.br/pdet/microdados/NOVO%20CAGED/Legado/Movimenta%C3%A7%C3%B5es/Layout%20Novo%20Caged%20Movimenta%C3%A7%C3%A3o.xlsx
已确认文件下载成功,但解压时始终报错,尝试过修改代码、检查服务器文件,问题仍未解决。
使用的Python代码
from os import remove from py7zr import SevenZipFile import pandas as pd import wget import numpy as np import warnings excel = pd.ExcelFile("data/dicionario_dados.xlsx") get_dict = lambda x: pd.read_excel(excel, sheet_name=x) data_dict = { sheet: {row[1]: row[2] for row in get_dict(sheet).itertuples()} for sheet in excel.sheet_names[1:] } url = lambda year, month: f"ftp://ftp.mtps.gov.br/pdet/microdados/NOVO CAGED/{year}/{year}{month:02d}/CAGEDMOV{year}{month:02d}.7z" dfs = [] start_year = 2020 start_month = 4 dates = [] #dates = data["competênciamov"].unique() for year in range(start_year, 2025): for month in range(start_month, 13): if f"{year}-{month:02d}-01" in dates: continue try: print(f"{month:02d}/{year}") wget.download(url(year, month), 'caged.7z') archive = SevenZipFile('caged.7z', mode = 'r') print('Microdata downloaded successfully, ready for reading') for name, fd in archive.read(name for name in archive.getnames() if name.endswith(".txt")).items(): caged_raw = pd.read_csv(fd, delimiter=";", decimal=",") caged_raw = caged_raw.loc[caged_raw["uf"] == 25, :].reset_index(drop=True) for col in caged_raw.columns: if col in data_dict: caged_raw[f"{col}_cod"] = caged_raw[col] caged_raw[col] = caged_raw[col].apply(lambda x: data_dict[col][x] if x in data_dict[col] else np.nan) dfs.append(caged_raw) archive.close() remove('caged.7z') print('Reading completed successfully') except Exception as e: print(f'Error processing {month:02d}/{year}: {e}') print('Microdata for the selected month is not yet available') break
代码执行输出
04/2020 Microdata downloaded successfully, ready for reading Error processing 04/2020: Corrupt input data Microdata for the selected month is not yet available 04/2021 Microdata downloaded successfully, ready for reading Error processing 04/2021: Corrupt input data Microdata for the selected month is not yet available 04/2022 Microdata downloaded successfully, ready for reading Error processing 04/2022: Corrupt input data Microdata for the selected month is not yet available 04/2023 Microdata downloaded successfully, ready for reading Error processing 04/2023: Corrupt input data Microdata for the selected month is not yet available 04/2024 Microdata downloaded successfully, ready for reading Error processing 04/2024: Corrupt input data Microdata for the selected month is not yet available
疑问
已尝试修改代码并检查FTP服务器上的文件,但问题仍未解决,请问是否有用户遇到过类似的FTP服务器相关问题?
内容的提问来源于stack exchange,提问作者vagas emprego
相关产品推荐
相关产品推荐

