Python中用for循环无法获取DataFrame名称，无法批量上传至ADLS Gen2

我需要将两个DataFrame（test1、test2）上传至ADLS Gen2目录，但无法获取DataFrame的变量名。此前单个上传时，我会直接调用以下upload_df_to_adls_path函数：

upload_df_to_adls_path(
    df,
    adls_dirname=staging_dirname,
    adls_filename=staging_filename,
    container_name=staging_container_name,
    storage_account_name=staging_account_name,
    storage_account_key=staging_account_key,
)

def upload_df_to_adls_path(
   df: pd.DataFrame, 
   adls_dirname: str,
   adls_filename: str,
   container_name: str,
   storage_account_name: str, 
   storage_account_key: str,
 ):
    """将pandas DataFrame以JSON Lines文件格式上传至指定ADLS路径"""
    json_data = df.to_json(orient='records', lines=True, date_format='iso')

    adls_service_client = get_adls_client(storage_account_name, storage_account_key)
    file_system_client = adls_service_client.get_file_system_client(file_system=container_name)

    try:
        file_system_client.create_directory(adls_dirname)
    except Exception as e:
        print(e)

    try:
        directory_client = file_system_client.get_directory_client(adls_dirname)
        file_client = directory_client.get_file_client(adls_filename)
        file_client.upload_data(json_data, overwrite=True)
    except Exception as e:
        print(e)

尝试用for循环批量处理时，我试图获取DataFrame的name属性，触发了报错：

final_df = [test1, test2]       
for files in final_df:
    print(files.name)
#     upload_df_to_adls_path(files, adls_dirname="FinalFilesToUse", adls_filename=files,
#         container_name=staging_container_name,
#         storage_account_name=staging_account_name,
#         storage_account_key=staging_account_key)       

# 报错信息：
# AttributeError: 'DataFrame' object has no attribute 'name'

解决办法：用字典关联DataFrame与目标文件名 #

Pandas DataFrame对象本身没有name属性，无法直接获取其变量名。可以通过字典来存储每个DataFrame对应的目标文件名，循环时同时获取DataFrame和文件名：

# 构建字典：键为ADLS上要保存的文件名，值为对应的DataFrame
df_mapping = {
    "test1.jsonl": test1,
    "test2.jsonl": test2
}

# 遍历字典完成批量上传
for target_filename, df in df_mapping.items():
    upload_df_to_adls_path(
        df,
        adls_dirname="FinalFilesToUse",
        adls_filename=target_filename,
        container_name=staging_container_name,
        storage_account_name=staging_account_name,
        storage_account_key=staging_account_key
    )

补充说明 #

• 字典的键可以直接设置为你需要在ADLS中保存的完整文件名（建议加上.jsonl后缀，匹配函数中生成的JSON Lines格式）
• 如果需要以变量名作为文件名基础，只需将键设为"test1"，再拼接后缀即可（比如f"{key}.jsonl"）

内容的提问来源于stack exchange，提问作者Lopa