运行Pandas数据清洗代码得到空DataFrame及如何保存test()输出为CSV

一、为什么得到空DataFrame #

• 1. 函数返回值错误：你定义的test()函数返回类型是None，没有将清洗后的DataFrame返回，外部执行cleaned = pd.DataFrame(test())自然只能得到空对象
• 2. 缺失必填列触发全量删除：你在coerce_columns、clean_odds函数中都用到了datetime列，但你的原始数据只有date和time两列，全程没有将两列拼接为datetime字段，后续去重、删空操作会因为列缺失/全空直接删除所有行
• 3. dropna()误删有效数据：原始数据前4行date字段为NaN，在numerical_scores函数末尾执行全量df.dropna()时，这部分行被直接删除，如果你没有额外有效数据，自然会全量删空
• 4. 冗余过滤逻辑的潜在影响：你已经通过正则r"[a-zA-Z\s\D]"将home_score、away_score中的非数字全部替换为空，后续针对-的过滤逻辑完全冗余，反而可能因为逻辑判断错误误删数据

二、修复后可正常保存CSV的代码 #

修正test()函数的返回逻辑，补全datetime列生成逻辑，去掉冗余过滤后的代码如下：

import pandas as pd

def harmonize_game(df: pd.DataFrame) -> pd.DataFrame:
    df["game"] = df["game"].astype(str).str.replace(r"(\(\w+\))", "", regex=True)
    df["game"] = df["game"].astype(str).str.replace(r"(\s\d+\S\d+)$", "", regex=True)
    df["league"] = (
        df["league"].astype(str).str.replace(r"(\s\d+\S\d+)$", "", regex=True)
    )
    df[["home_team", "away_team"]] = df["game"].str.split(" - ", expand=True, n=1)
    df[["home_score", "away_score"]] = df["score"].str.split(":", expand=True)
    # 新增：拼接datetime列
    df['datetime'] = pd.to_datetime(df['date'] + ' ' + df['time'], errors='coerce')
    print("Data Harmonised")
    return df

def numerical_scores(df: pd.DataFrame) -> pd.DataFrame:
    df["away_score"] = (
        df["away_score"].astype(str).str.replace(r"[a-zA-Z\s\D]", "", regex=True)
    )
    df["home_score"] = (
        df["home_score"].astype(str).str.replace(r"[a-zA-Z\s\D]", "", regex=True)
    )
    df = df[df.home_odds != "-"]
    df = df[df.draw_odds != "-"]
    df = df[df.away_odds != "-"]
    m = (
        df[["home_odds", "draw_odds", "away_odds"]]
        .astype(str)
        .agg(lambda x: x.str.count("/"), 1)
        .ne(0)
        .all(1)
    )
    df = df[~m]
    df = df[df.home_score != ""]
    df = df[df.away_score != ""]
    # 仅针对关键字段删空，避免误删有效数据
    df = df.dropna(subset=['home_score','away_score','home_odds','draw_odds','away_odds','datetime'])
    print("Numerical data harmonised and cleaned")
    return df

def coerce_columns(df: pd.DataFrame) -> pd.DataFrame:
    df = df.loc[
        :,
        df.columns.intersection(
            [
                "datetime",
                "country",
                "league",
                "home_team",
                "away_team",
                "home_odds",
                "draw_odds",
                "away_odds",
                "home_score",
                "away_score",
            ]
        ),
    ]
    colt = {
        "country": str,
        "league": str,
        "home_team": str,
        "away_team": str,
        "home_odds": float,
        "draw_odds": float,
        "away_odds": float,
        "home_score": int,
        "away_score": int,
    }
    df = df.astype(colt)
    print("Data types recognized")
    return df

def strip_strings(df: pd.DataFrame) -> pd.DataFrame:
    return df.applymap(lambda x: x.strip() if isinstance(x, str) else x)

def clean_odds(df: pd.DataFrame) -> pd.DataFrame:
    df = df[df["home_odds"] <= 100]
    df = df[df["draw_odds"] <= 100]
    df = df[df["away_odds"] <= 100]
    df = df.drop_duplicates(
        [
            "datetime",
            "home_score",
            "away_score",
            "country",
            "league",
            "home_team",
            "away_team",
        ],
        keep="last",
    )
    df = df.applymap(lambda x: x.strip() if isinstance(x, str) else x)
    print("Dataframe Cleaned")
    return df

def clean(df: pd.DataFrame) -> pd.DataFrame:
    df = harmonize_game(df)
    df = numerical_scores(df)
    df = coerce_columns(df)
    df = strip_strings(df)
    df = clean_odds(df)
    print("All steps applied")
    return df

def test(csv_path: str, save_path: str) -> pd.DataFrame:
    # 补全read_csv的输入路径参数
    df = pd.read_csv(csv_path)
    cleaned_df = clean(df)
    # 保存CSV
    cleaned_df.to_csv(save_path, index=False, encoding='utf-8-sig')
    print(f"清洗后数据已保存至 {save_path}")
    return cleaned_df

if __name__ == "__main__":
    # 替换为你的输入文件路径和输出文件路径
    input_path = "你的原始数据文件路径.csv"
    output_path = "清洗后的数据.csv"
    cleaned_data = test(input_path, output_path)

三、保存CSV的核心注意点 #

• 必须保证清洗函数有有效的DataFrame返回值，不能用无返回的函数的结果构造DataFrame
• to_csv()方法必须指定保存路径，设置index=False可以避免输出多余的索引列，encoding='utf-8-sig'可以保证中文、特殊字符在Excel中打开不乱码

内容的提问来源于stack exchange，提问作者leonardo