如何优化DataFrame处理代码格式？能否将修改融入创建环节？

问题背景 #

我有一段冗长且不美观的Pandas DataFrame处理代码，未来可能还会变长。我想了解这段代码的最佳格式化方式，除了封装为函数外是否有更优呈现方式，同时想知道能否将部分数据处理修改步骤融入DataFrame的创建环节。原始代码如下：

# Create data frames.
channel_data = pd.DataFrame(result_channel_statistics, columns=[
    'result_playlist_id' ,
    'result_channel_name' , 
    'result_channel_views' ,
    'result_channel_subscribers' ,
    'result_channel_total_videos' 
    ])
video_data = pd.DataFrame(video_details, columns=[
    'result_video_id' ,
    'result_video_upload_time' , 
    'Published Date' ,
    'Published Time' , 
    'result_video_name' , 
    'result_video_description' , 
    'result_video_views' , 
    'result_video_likes' , 
    'result_video_comments' 
    ])

# Data Frame Structure
channel_data['result_channel_subscribers'] = pd.to_numeric(channel_data['result_channel_subscribers'])
channel_data['result_channel_views'] = pd.to_numeric(channel_data['result_channel_views'])
channel_data['result_channel_total_videos'] = pd.to_numeric(channel_data['result_channel_total_videos'])
upload_time = video_data.groupby('result_video_upload_time', as_index = False).size()
published_date = upload_time['Dates'] = pd.to_datetime(upload_time['result_video_upload_time']).dt.date
published_time = upload_time['Time']  = pd.to_datetime(upload_time['result_video_upload_time']).dt.time
video_data['Published Date'] = published_date #Pull date from string
video_data['Published Time'] = published_time #Pull time from string
# Rename column headers
video_data.columns = ['Video ID', 'Published', 'Uplaod Date', 'Upload Time', 'Video Title', 'Video Description', 'Views', 'Likes', 'Comments']
channel_data.columns = ['Video ID', 'Channel Title', 'Total Views', 'Total Subs', 'Total Videos']

核心优化需求聚焦于这段处理代码的可读性、可维护性：

# Data Frame Structure
channel_data['result_channel_subscribers'] = pd.to_numeric(channel_data['result_channel_subscribers'])
channel_data['result_channel_views'] = pd.to_numeric(channel_data['result_channel_views'])
channel_data['result_channel_total_videos'] = pd.to_numeric(channel_data['result_channel_total_videos'])
upload_time = video_data.groupby('result_video_upload_time', as_index = False).size()
published_date = upload_time['Dates'] = pd.to_datetime(upload_time['result_video_upload_time']).dt.date
published_time = upload_time['Time']  = pd.to_datetime(upload_time['result_video_upload_time']).dt.time
video_data['Published Date'] = published_date #Pull date from string
video_data['Published Time'] = published_time #Pull time from string
# Rename column headers
video_data.columns = ['Video ID', 'Published', 'Uplaod Date', 'Upload Time', 'Video Title', 'Video Description', 'Views', 'Likes', 'Comments']
channel_data.columns = ['Video ID', 'Channel Title', 'Total Views', 'Total Subs', 'Total Videos']

优化方案 #

1. 封装为模块化函数（长期维护最优方案） #

将DataFrame的创建、处理逻辑拆分为独立函数，每个函数负责单一职责，既提升可读性，也方便后续扩展和测试。

import pandas as pd

def create_channel_data(result_channel_statistics):
    # 定义原始列名
    raw_cols = [
        'result_playlist_id',
        'result_channel_name',
        'result_channel_views',
        'result_channel_subscribers',
        'result_channel_total_videos'
    ]
    # 创建基础DataFrame
    df = pd.DataFrame(result_channel_statistics, columns=raw_cols)
    
    # 批量转换数值列（替代重复的pd.to_numeric调用）
    numeric_cols = raw_cols[2:]  # 提取需要转数值的列
    df[numeric_cols] = df[numeric_cols].apply(pd.to_numeric, errors='coerce')
    
    # 重命名列（用rename更安全，避免列顺序出错）
    rename_map = {
        'result_playlist_id': 'Video ID',
        'result_channel_name': 'Channel Title',
        'result_channel_views': 'Total Views',
        'result_channel_subscribers': 'Total Subs',
        'result_channel_total_videos': 'Total Videos'
    }
    df = df.rename(columns=rename_map)
    
    return df

def create_video_data(video_details):
    # 原始数据中不存在'Published Date'/'Published Time'，创建时无需包含
    raw_cols = [
        'result_video_id',
        'result_video_upload_time',
        'result_video_name',
        'result_video_description',
        'result_video_views',
        'result_video_likes',
        'result_video_comments'
    ]
    # 创建基础DataFrame
    df = pd.DataFrame(video_details, columns=raw_cols)
    
    # 直接生成日期/时间列（原groupby步骤冗余，可移除）
    df['Upload Date'] = pd.to_datetime(df['result_video_upload_time']).dt.date
    df['Upload Time'] = pd.to_datetime(df['result_video_upload_time']).dt.time
    
    # 批量转换数值列
    numeric_cols = ['result_video_views', 'result_video_likes', 'result_video_comments']
    df[numeric_cols] = df[numeric_cols].apply(pd.to_numeric, errors='coerce')
    
    # 重命名列
    rename_map = {
        'result_video_id': 'Video ID',
        'result_video_upload_time': 'Published',
        'result_video_name': 'Video Title',
        'result_video_description': 'Video Description',
        'result_video_views': 'Views',
        'result_video_likes': 'Likes',
        'result_video_comments': 'Comments'
    }
    df = df.rename(columns=rename_map)
    
    # 调整列顺序（匹配原输出结构）
    desired_order = [
        'Video ID', 'Published', 'Upload Date', 'Upload Time',
        'Video Title', 'Video Description', 'Views', 'Likes', 'Comments'
    ]
    return df[desired_order]

# 调用函数生成处理后的DataFrame
channel_data = create_channel_data(result_channel_statistics)
video_data = create_video_data(video_details)

2. 使用链式操作简化单块逻辑 #

如果不想拆分为函数，可通过assign()等方法实现链式调用，减少重复的变量名书写，提升代码紧凑性：

# 优化channel_data的处理逻辑
channel_data = (
    pd.DataFrame(result_channel_statistics, columns=[
        'result_playlist_id', 'result_channel_name',
        'result_channel_views', 'result_channel_subscribers',
        'result_channel_total_videos'
    ])
    .assign(
        result_channel_views=lambda x: pd.to_numeric(x['result_channel_views'], errors='coerce'),
        result_channel_subscribers=lambda x: pd.to_numeric(x['result_channel_subscribers'], errors='coerce'),
        result_channel_total_videos=lambda x: pd.to_numeric(x['result_channel_total_videos'], errors='coerce')
    )
    .rename(columns={
        'result_playlist_id': 'Video ID',
        'result_channel_name': 'Channel Title',
        'result_channel_views': 'Total Views',
        'result_channel_subscribers': 'Total Subs',
        'result_channel_total_videos': 'Total Videos'
    })
)

# 优化video_data的处理逻辑
video_data = (
    pd.DataFrame(video_details, columns=[
        'result_video_id', 'result_video_upload_time',
        'result_video_name', 'result_video_description',
        'result_video_views', 'result_video_likes', 'result_video_comments'
    ])
    .assign(
        Upload_Date=lambda x: pd.to_datetime(x['result_video_upload_time']).dt.date,
        Upload_Time=lambda x: pd.to_datetime(x['result_video_upload_time']).dt.time,
        result_video_views=lambda x: pd.to_numeric(x['result_video_views'], errors='coerce'),
        result_video_likes=lambda x: pd.to_numeric(x['result_video_likes'], errors='coerce'),
        result_video_comments=lambda x: pd.to_numeric(x['result_video_comments'], errors='coerce')
    )
    .rename(columns={
        'result_video_id': 'Video ID',
        'result_video_upload_time': 'Published',
        'result_video_name': 'Video Title',
        'result_video_description': 'Video Description',
        'result_video_views': 'Views',
        'result_video_likes': 'Likes',
        'result_video_comments': 'Comments',
        'Upload_Date': 'Upload Date',
        'Upload_Time': 'Upload Time'
    })
    [['Video ID', 'Published', 'Upload Date', 'Upload Time',
      'Video Title', 'Video Description', 'Views', 'Likes', 'Comments']]
)

3. 部分处理步骤可融入DataFrame创建环节 #

部分操作可以在创建DataFrame时完成，但需权衡可读性和灵活性：

• 直接指定最终列名：如果原始数据的列顺序与最终列完全对应，可以在pd.DataFrame()的columns参数中直接使用最终列名，省去后续重命名步骤，但如果原始数据结构变更，这种方式容易出错。
• 指定dtype：对于已知类型的列，可在创建时通过dtype参数指定，但对于需要从字符串转换为数值的列，创建后批量转换更稳妥（避免转换失败导致整个DataFrame创建失败）。
• 衍生列提前处理：如果原始数据是可迭代对象（比如列表），可以先在原始数据中生成日期/时间列，再传入pd.DataFrame()，但这种方式会增加原始数据处理的复杂度，不如在DataFrame层面处理清晰。

额外优化点 #

• 批量处理重复操作：避免重复调用pd.to_numeric，通过列列表批量转换，减少代码冗余。
• 移除冗余逻辑：原代码中groupby生成upload_time的步骤完全冗余，直接在video_data上处理日期时间即可。
• 修正拼写错误：原代码中Uplaod Date应为Upload Date，避免后续使用时出现列名错误。

内容的提问来源于stack exchange，提问作者Gigantesco