如何在Pandas中处理路径权限错误并生成Last_Update列？

问题场景 #

现有如下DataFrame：

JOB_Command Parent_path
0   /data/ingestao_gpdb_nextdm_rep/execucao/call_r...   /data/ingestao_gpdb_nextdm_rep/execucao/call_r...
1   /data/processos/current/ingestao/ciclico/BACEN...   /data/processos/current/ingestao/
2   /data/processos/current/ingestao/ciclico/BACEN...   /data/processos/current/ingestao/
3   /data/processos/current/ingestao/ciclico/BACEN...   /data/processos/current/ingestao/
4   /data/processos/current/ingestao/ciclico/BACEN...   /data/processos/current/ingestao/
5   /data/processos/current/ingestao/ciclico/Inges...   /data/processos/current/ingestao/

使用以下代码获取文件/文件夹最后更新时间时，遇到权限不足的路径会抛出PermissionError：

from pathlib import Path
import datetime
import numpy as np
import pandas as pd

def get_fourth_elem(file_path):
    """Helper function.

    Args:
        file_path: file path as a string.

    Returns:
        absolute path to the fourth element (or last one if shorter) as a Pathlib object.
    """
    file_path_length = len(file_path.strip("/").split("/"))
    file_path = Path(file_path)
    if file_path_length > 4:
        for _ in range(file_path_length - 4):
            file_path = Path(file_path.parent)
        return file_path
    else:
        return file_path

df_test["Last_Update"] = df_test["JOB_Command"].apply(
    lambda x: datetime.datetime.fromtimestamp(
        get_fourth_elem(x).stat().st_mtime
    ).strftime("%Y-%m-%d %H:%H:%S")
    if Path(x).exists()
    else np.nan
)

错误信息：

PermissionError: [Errno 13] Permission denied: '/data/processos/current/ingestao/ciclico/IngestionManager_2_0_EXP/sbin/call_ingestion_manager_ciclico_exp.sh'

期望输出：

JOB_Command Parent_path Last_Update
0   /data/ingestao_gpdb_nextdm_rep/execucao/call_r...   /data/ingestao_gpdb_nextdm_rep/execucao/call_r...   2022-09-26 11:30:00
1   /data/processos/current/ingestao/ciclico/BACEN...   /data/processos/current/ingestao/                   2022-09-20 19:20:00             
2   /data/processos/current/ingestao/ciclico/BACEN...   /data/processos/current/ingestao/                   2022-09-20 19:20:00
3   /data/processos/current/ingestao/ciclico/BACEN...   /data/processos/current/ingestao/                   2022-09-20 19:20:00
4   /data/processos/current/ingestao/ciclico/BACEN...   /data/processos/current/ingestao/                   2022-09-20 19:20:00
5   /data/processos/current/ingestao/ciclico/Inges...   /data/processos/current/ingestao/                   NaN

修改后的代码方案 #

将更新时间的获取逻辑封装成独立函数，在函数内部添加异常捕获，处理路径不存在和权限不足的情况：

from pathlib import Path
import datetime
import numpy as np
import pandas as pd

def get_fourth_elem(file_path):
    """Helper function.

    Args:
        file_path: file path as a string.

    Returns:
        absolute path to the fourth element (or last one if shorter) as a Pathlib object.
    """
    file_path_length = len(file_path.strip("/").split("/"))
    file_path = Path(file_path)
    if file_path_length > 4:
        for _ in range(file_path_length - 4):
            file_path = file_path.parent
        return file_path
    else:
        return file_path

def get_last_update(file_path_str):
    file_path = Path(file_path_str)
    if not file_path.exists():
        return np.nan
    try:
        target_path = get_fourth_elem(file_path_str)
        mtime = target_path.stat().st_mtime
        return datetime.datetime.fromtimestamp(mtime).strftime("%Y-%m-%d %H:%M:%S")
    except PermissionError:
        print(f"permission denied: {file_path_str}")
        return np.nan
    except Exception as e:
        print(f"error accessing {file_path_str}: {str(e)}")
        return np.nan

df_test["Last_Update"] = df_test["JOB_Command"].apply(get_last_update)

关键说明 #

• 异常捕获：新增try-except块，专门捕获PermissionError，此时打印权限不足提示并返回NaN；同时捕获其他未知异常，避免程序崩溃。
• 逻辑拆分：把原lambda中的复杂逻辑抽成独立函数get_last_update，代码更易读和维护。
• 格式修正：原代码中%H:%H:%S是错误格式，改为%H:%M:%S，正确显示时分秒。

内容的提问来源于stack exchange，提问作者gfernandes