处理Kaggle泰坦尼克数据集时，Test集Cabin列莫名全变为NaN

我在处理Kaggle泰坦尼克经典数据集时，创建了training、test数据集及其副本copytrain、copytest共4个DataFrame，但执行相同操作后，test集的Cabin列全部变为NaN，无法定位问题。

代码块1：数据读取与赋值 #

training=pd.read_csv("../input/titanic/train.csv")
copytrain = training
training=training.drop(columns=["Ticket","PassengerId"])

test=pd.read_csv("../input/titanic/test.csv")
test=test.drop(columns=["Ticket","PassengerId"])
copytest=test

代码块2：Cabin列初步处理 #

for i in range(len(training["Cabin"])):
    if "A" in str(training.loc[i,"Cabin"]):
        training.loc[i,"Cabin"] = "A"
    if "B" in str(training.loc[i,"Cabin"]):
        training.loc[i,"Cabin"] = "B"
    if "C" in str(training.loc[i,"Cabin"]):
        training.loc[i,"Cabin"] = "C"
    if "D" in str(training.loc[i,"Cabin"]):
        training.loc[i,"Cabin"] = "D"
    if "E" in str(training.loc[i,"Cabin"]):
        training.loc[i,"Cabin"] = "E"
    if "F" in str(training.loc[i,"Cabin"]):
        training.loc[i,"Cabin"] = "F"
    if "G" in str(training.loc[i,"Cabin"]):
        training.loc[i,"Cabin"] = "G"
    
for i in range(len(test["Cabin"])):
    if "A" in str(test.loc[i,"Cabin"]):
        test.loc[i,"Cabin"] = "A"
    if "B" in str(test.loc[i,"Cabin"]):
        test.loc[i,"Cabin"] = "B"
    if "C" in str(test.loc[i,"Cabin"]):
        test.loc[i,"Cabin"] = "C"
    if "D" in str(test.loc[i,"Cabin"]):
        test.loc[i,"Cabin"] = "D"
    if "E" in str(test.loc[i,"Cabin"]):
        test.loc[i,"Cabin"] = "E"
    if "F" in str(test.loc[i,"Cabin"]):
        test.loc[i,"Cabin"] = "F"
    if "G" in str(test.loc[i,"Cabin"]):
        test.loc[i,"Cabin"] = "G"

training.loc[training["Cabin"]=="T","Cabin"]="A"
copytrain.loc[copytrain["Cabin"]=="T","Cabin"]="A"

test.loc[test["Cabin"]=="T","Cabin"]="A"
copytest.loc[test["Cabin"]=="T","Cabin"]="A"

training.loc[training.Cabin.isnull(),"Cabin"]="U"
test.loc[test.Cabin.isnull(),"Cabin"]="U"

test.Cabin.value_counts()

代码块2输出： #

U    327
C     35
B     18
D     13
E     11
A      7
F      6
G      1
Name: Cabin, dtype: int64

代码块3：副本Cabin列进一步处理 #

copytrain["Cabin"]= copytrain["Cabin"].str.replace("A","")
copytrain["Cabin"]= copytrain["Cabin"].str.replace("B","")
copytrain["Cabin"]= copytrain["Cabin"].str.replace("C","")
copytrain["Cabin"]= copytrain["Cabin"].str.replace("D","")
copytrain["Cabin"]= copytrain["Cabin"].str.replace("E","")
copytrain["Cabin"]= copytrain["Cabin"].str.replace("F","")
copytrain["Cabin"]= copytrain["Cabin"].str.replace("G","")
#-------------------------------------------------------
copytest["Cabin"]= copytest["Cabin"].str.replace("A","")
copytest["Cabin"]= copytest["Cabin"].str.replace("B","")
copytest["Cabin"]= copytest["Cabin"].str.replace("C","")
copytest["Cabin"]= copytest["Cabin"].str.replace("D","")
copytest["Cabin"]= copytest["Cabin"].str.replace("E","")
copytest["Cabin"]= copytest["Cabin"].str.replace("F","")
copytest["Cabin"]= copytest["Cabin"].str.replace("G","")

copytrain.loc[(copytrain["Cabin"].str.len()==5) | (copytrain["Cabin"].str.len()==8),"Cabin"]=copytrain.Cabin.str.slice(stop=2)
copytrain.loc[(copytrain["Cabin"].str.len()==7) | (copytrain["Cabin"].str.len()==11),"Cabin"]=copytrain.Cabin.str.slice(stop=3)
#----------------------------------------------------------------------------------------------------------------------------
copytest.loc[(copytest["Cabin"].str.len()==5) | (copytest["Cabin"].str.len()==8),"Cabin"]=copytest.Cabin.str.slice(stop=2)
copytest.loc[(copytest["Cabin"].str.len()==7) | (copytest["Cabin"].str.len()==11),"Cabin"]=copytest.Cabin.str.slice(stop=3)

copytrain["Cabin"]=pd.to_numeric(copytrain["Cabin"],errors="coerce")
copytest["Cabin"]=pd.to_numeric(copytest["Cabin"],errors="coerce")

test.Cabin.value_counts()

代码块3输出： #

Series([], Name: Cabin, dtype: int64)

数据集样例 #

Training集前5行： #

Surv    Pclass  Sex Age SibSp   Parch   Fare    Cabin   Embarked
0   3   male    22.0    1   0   7.2500  U   S
1   1   female  38.0    1   0   71.2833 C   C
1   3   female  26.0    0   0   7.9250  U   S
1   1   female  35.0    1   0   53.1000 C   S
0   3   male    35.0    0   0   8.0500  U   S

Test集前5行（代码块3执行前）： #

Pclass  Sex Age SibSp   Parch   Fare    Cabin   Embarked
3   male    34.5    0   0   7.8292  U   Q
3   female  47.0    1   0   7.0000  U   S
2   male    62.0    0   0   9.6875  U   Q
3   male    27.0    0   0   8.6625  U   S
3   female  22.0    1   1   12.2875 U   S

问题根源 #

代码块1中，copytrain = training和copytest = test是引用赋值，并非创建独立副本。这意味着copytest和test指向同一个DataFrame对象，对copytest的任何修改都会直接作用在test上。

代码块3中，copytest["Cabin"]被替换掉A-G所有字母后，原有的"A""B"等类别变为空字符串，填充的"U"虽未被替换，但后续执行pd.to_numeric(copytest["Cabin"], errors="coerce")时，"U"无法转为数值，会被强制转为NaN，最终导致test的Cabin列全部变为NaN。

解决方案 #

1. 创建独立副本：使用.copy()方法生成独立的DataFrame对象，避免引用关联：

# 替换代码块1中的赋值语句
copytrain = training.copy()
copytest = test.copy()

2. 优化Cabin处理逻辑：替换低效的循环遍历，改用pandas内置字符串操作：

# 处理training的Cabin列
training['Cabin'] = training['Cabin'].str.extract('([A-G])', expand=False)
training['Cabin'] = training['Cabin'].replace('T', 'A').fillna('U')
# 处理test的Cabin列
test['Cabin'] = test['Cabin'].str.extract('([A-G])', expand=False)
test['Cabin'] = test['Cabin'].replace('T', 'A').fillna('U')

3. 完成上述修改后，对副本copytrain、copytest的操作将不会影响原数据集training、test。

内容的提问来源于stack exchange，提问作者onat dicleli