将Series转为2D数组解决scikit-learn线性回归输入维度报错问题

数据来源为 copper-new.txt

我正在分析铜热膨胀系数随温度变化的数据集，计划构建线性回归（Linear Regression）模型拟合二者的关联关系，实现任意温度下热膨胀系数的预测。

# Need to split into columns since Pandas did not do it for us
copperdata['X'] = copperdata.apply(lambda x: x.str.split()[0][1],axis=1)
copperdata['y'] = copperdata.apply(lambda x: x.str.split()[0][0],axis=1)
copperdata = copperdata[['X','y']].astype(float)
copperdata.head()

我需要完成lin_model()函数，该函数接收copperdata DataFrame作为输入，执行以下操作：

• 为输入特征X（温度）和目标值y（热膨胀系数）创建NumPy数组，需将X数组重塑为第二维为1的2D数组以适配scikit-learn模型输入要求
• 拆分数据集：总数据10%为测试集，剩余90%中80%为训练集、20%为验证集，拆分时设置random_state=0
• 训练线性回归模型，计算验证集MAE
  函数需返回训练好的模型和验证集MAE两个结果。

原有错误代码如下：

def lin_model(df):
    # YOUR CODE HERE
    X = copperdata['X']
    np.asarray('X')
    y = copperdata['y']
    np.asarray('y')
    X.values.reshape(-1, 1)
    y.values.reshape(-1, 1)
    X_train_full,X_test,_y_train_full,y_test = train_test_split(X,y,random_state=0,test_size = .10)
    X_train,X_val,y_train,y_val = train_test_split(X_train_full,y_train_full,random_state = 0, test_size = .20)
    model = LinearRegression()
    model.fit(X_train,y_train)
    val_preds = model.predict(X_val)
    mae = metrics.mean_absolute_error(y_test,y_pred)
    return model, mae

    raise NotImplementedError()

运行测试单元时执行以下代码：

model,mae = lin_model(copperdata)

抛出如下错误：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-112-8931ac554101> in <module>
      1 # Test cell
----> 2 model,mae = lin_model(copperdata)
      3 
      4 # Print model coefficients and intercept
      5 m = model.coef_

<ipython-input-111-ba2035c8e718> in lin_model(df)
      6     model.fit(X_train,y_train)
      7     #val_preds = model.predict(X_val)
----> 8     y_preds = model.predict(X_test)
      9     mae = metrics.mean_absolute_error(y_test,y_pred)
     10     return model, mae

/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sklearn/linear_model/_base.py in predict(self, X)
    234             Returns predicted values.
    235         """
--> 236         return self._decision_function(X)
    237 
    238     _preprocess_data = staticmethod(_preprocess_data)

/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sklearn/linear_model/_base.py in _decision_function(self, X)
    216         check_is_fitted(self)
    217 
--> 218         X = check_array(X, accept_sparse=['csr', 'csc', 'coo'])
    219         return safe_sparse_dot(X, self.coef_.T,
    220                                dense_output=True) + self.intercept_

/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sklearn/utils/validation.py in inner_f(*args, **kwargs)
     70                           FutureWarning)
     71         kwargs.update({k: arg for k, arg in zip(sig.parameters, args)})
--- 72         return f(**kwargs)
     73     return inner_f
     74 
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sklearn/utils/validation.py in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, estimator)
    617             # If input is 1D raise error
    618             if array.ndim == 1:
--> 619                 raise ValueError(
    620                     "Expected 2D array, got 1D array instead:\narray={}.\n"
    621                     "Reshape your data either using array.reshape(-1, 1) if "

ValueError: Expected 2D array, got 1D array instead:
array=[656.2  544.47 524.7   60.41 447.41  89.57].
Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.

修复方案 #

错误原因 #

原有代码存在以下问题：

1. 数组重塑操作没有赋值回原变量，X.values.reshape(-1, 1)执行后结果直接丢弃，X始终是1D数组
2. 没有使用函数入参df，硬编码了全局变量copperdata，函数复用性差
3. 第一次拆分时变量名写错，将y_train_full误写为_y_train_full，导致后续拆分找不到变量
4. MAE计算逻辑错误：要求计算验证集MAE，原有代码误用了测试集y_test和不存在的变量y_pred
5. np.asarray('X')写法错误，传入的是字符串而不是变量X，数组转换操作完全无效
6. 目标变量y不需要重塑为2D数组，scikit-learn中目标变量接受1D数组输入

修正后代码 #

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn import metrics

def lin_model(df):
    # 提取特征和目标值，将X转换为符合要求的2D数组
    X = np.asarray(df['X']).reshape(-1, 1)
    y = np.asarray(df['y'])
    
    # 第一次拆分：10%测试集，90%剩余集
    X_train_full, X_test, y_train_full, y_test = train_test_split(X, y, random_state=0, test_size=0.1)
    # 第二次拆分：剩余90%中20%为验证集，80%为训练集
    X_train, X_val, y_train, y_val = train_test_split(X_train_full, y_train_full, random_state=0, test_size=0.2)
    
    # 训练模型
    model = LinearRegression()
    model.fit(X_train, y_train)
    
    # 计算验证集MAE
    val_preds = model.predict(X_val)
    mae = metrics.mean_absolute_error(y_val, val_preds)
    
    return model, mae

内容的提问来源于stack exchange，提问作者diesmiling