数据划分与交叉验证报错：无法将字符串转为float的解决方法

我正在执行数据划分与交叉验证操作，需求是先提取测试数据集，剩余数据用于交叉验证。但在交叉验证阶段末尾出现报错：ValueError: could not convert string to float: 'Curtis RIngraham Directge'。我已尝试检查x_train、y_train、x_valid、y_valid中的字符串值，但未发现问题，请问该如何修复？

数据划分代码 #

from sklearn.model_selection import train_test_split
from sklearn.model_selection import KFold

# 先提取测试集，存入x_test、y_test
features = features_df.to_numpy()
labels = labels_df.to_numpy()
_x, x_test, _y, y_test = train_test_split(features, labels, test_size=0.10, random_state=42)

# 设置k=5
k = 5

kfold_spliiter = KFold(n_splits=k)

folds_data = [] # 这种方式效率不高，但暂时这么写

fold = 1
for train_index, validation_index in kfold_spliiter.split(_x):
    x_train , x_valid = _x[train_index,:],_x[validation_index,:]
    y_train , y_valid = _y[train_index,:] , _y[validation_index,:]
    print (f"Fold {fold} training data shape = {(x_train.shape,y_train.shape)}")
    print (f"Fold {fold} validation data shape = {(x_valid.shape,y_valid.shape)}")
    fold+=1
    folds_data.append((x_train,y_train,x_valid,y_valid))

交叉验证代码 #

best_validation_accuracy = 0
best_model_name = ""
best_model = None

# 遍历所有模型
for model_name in all_models.keys():

    print (f"Evaluating {model_name} ...")
    model = all_models[model_name]

    # 存储所有折的训练和验证准确率
    train_acc_for_all_folds = []
    valid_acc_for_all_folds = []

    # 遍历所有折
    for i, fold in enumerate(folds_data):
        x_train, y_train, x_valid, y_valid = fold

        # 训练模型
        _ = model.fit(x_train,y_train.flatten())

        # 在训练集上评估模型
        y_pred_train = model.predict(x_train)

        # 在验证集上评估模型
        y_pred_valid = model.predict(x_valid)

        # 计算训练准确率
        train_acc = accuracy_score(y_pred_train , y_train)

        # 存储每折的训练准确率
        train_acc_for_all_folds.append(train_acc)

        # 计算验证准确率
        valid_acc = accuracy_score(y_pred_valid , y_valid.flatten())

        # 存储每折的验证准确率
        valid_acc_for_all_folds.append(valid_acc)

    # 计算k折的平均训练准确率
    avg_training_acc = sum(train_acc_for_all_folds)/k

    print (f"Average training accuracy for model {model_name} = {avg_training_acc}")

    # 计算k折的平均验证准确率
    avg_validation_acc = sum(valid_acc_for_all_folds)/k

    print (f"Average validation accuracy for model {model_name} = {avg_validation_acc}")

    # 根据平均验证准确率选择最优模型
    if avg_validation_acc > best_validation_accuracy:
        best_validation_accuracy = avg_validation_acc
        best_model_name = model_name
        best_model = model
    print ("-----------------------------------")

print (f"Best model for the task is {best_model_name} which offers the validation accuracy of {best_validation_accuracy}")

修复方案 #

1. 准确定位字符串特征位置 #

报错中的字符串Curtis RIngraham Directge必然存在于特征矩阵features中，可通过以下代码快速定位：

import numpy as np

# 遍历特征列，查找目标字符串的位置
for col_idx in range(features.shape[1]):
    col_data = features[:, col_idx]
    # 检查列中是否包含字符串类型
    if any(isinstance(val, str) for val in col_data):
        matches = np.where(col_data == 'Curtis RIngraham Directge')[0]
        if len(matches) > 0:
            print(f"目标字符串位于第{col_idx}列，第{matches[0]}行")

2. 将字符串特征转换为数值型 #

Scikit-learn的模型仅支持数值型输入，需根据特征类型选择对应转换方式：

• 分类类字符串特征（如姓名、类别标签）：使用OneHotEncoder（无顺序分类）或LabelEncoder（有顺序分类）
  示例代码：

  from sklearn.preprocessing import OneHotEncoder
  import numpy as np
  
  # 假设已定位到目标列索引为col_idx
  encoder = OneHotEncoder(sparse_output=False, handle_unknown='ignore')
  encoded_col = encoder.fit_transform(features[:, col_idx].reshape(-1, 1))
  # 替换原特征列
  features = np.hstack([features[:, :col_idx], encoded_col, features[:, col_idx+1:]])
  

• 文本类字符串特征：使用TF-IDF、词袋模型等方式将文本转换为数值向量。

3. 规范预处理流程避免数据泄露 #

必须在划分测试集之前完成所有特征预处理，且交叉验证时仅在训练折上拟合预处理工具，再应用到验证折，正确流程：

1. 对全部特征执行预处理（如编码字符串）
2. 划分测试集与剩余数据
3. 对剩余数据执行交叉验证

4. 检查模型输入兼容性 #

确认使用的模型（如SVM、线性回归等）仅接受数值型输入，无遗漏的未处理字符串特征。

内容的提问来源于stack exchange，提问作者FRGMT