如何在Databricks AutoML中指定特征列？关联特征表报错求助

核心问题分析 #

1. AutoML的feature_store_lookups中feature_names参数若格式错误（比如用字符串而非列表），会导致无法筛选特征，加载特征表所有列。
2. exclude_cols仅能作用于原始输入数据集（即automl_data）的列，无法直接排除特征表引入的列。

解决方法 #

方法1：手动预关联特征表并筛选列（最可靠） #

通过Feature Store Client提前将所需特征关联到原始数据，严格控制加载的列，再将处理后的数据集传给AutoML，彻底避免多余列问题。

from databricks.feature_store import FeatureStoreClient

# 初始化Feature Store客户端
fs = FeatureStoreClient()

# 第一步：关联油价特征表，仅保留指定列
data_with_oil = fs.create_training_set(
    automl_data,
    feature_lookups=[{
        "table_name":"lakehouse_in_action.favorita_forecasting.oil_10d_lag_ft",
        "lookup_key":"date",
        "feature_names":["lag10_oil_price"]
    }],
    label=label_name,
    exclude_columns=[]
).load_df()

# 第二步：关联门店节假日特征表，明确指定需要的列（替换为实际列名）
data_with_holidays = fs.create_training_set(
    data_with_oil,
    feature_lookups=[{
        "table_name":"lakehouse_in_action.favorita_forecasting.store_holidays_ft",
        "lookup_key":["date","store_nbr"],
        "feature_names":["holiday_type", "is_holiday"]  # 仅加载需要的特征
    }],
    label=label_name,
    exclude_columns=[]
).load_df()

# 第三步：关联门店基础特征表
final_data = fs.create_training_set(
    data_with_holidays,
    feature_lookups=[{
        "table_name":"lakehouse_in_action.favorita_forecasting.stores_ft",
        "lookup_key":"store_nbr",
        "feature_names":["cluster","store_type"]
    }],
    label=label_name,
    exclude_columns=[]
).load_df()

# 若仍有多余列，直接删除
final_data = final_data.drop("id", "city", "state", "price_date")

# 将处理好的数据集传给AutoML，无需再用feature_store_lookups参数
summary = databricks.automl.regress(final_data, 
                                    target_col=label_name,
                                    time_col="date", 
                                    timeout_minutes=60)

方法2：修正feature_store_lookups参数格式 #

确保feature_names始终为列表格式（即使单个特征也要用列表包裹），避免AutoML加载特征表所有列：

import databricks.automl

automl_feature_lookups = [
    {
      "table_name":"lakehouse_in_action.favorita_forecasting.oil_10d_lag_ft",
      "lookup_key":"date",
      "feature_names":["lag10_oil_price"]  # 改为列表，而非字符串
    },
    {
      "table_name":"lakehouse_in_action.favorita_forecasting.store_holidays_ft",
      "lookup_key":["date","store_nbr"],
      "feature_names":["需要的列1", "需要的列2"]  # 必须明确指定列，不能留空
    },
    {
      "table_name":"lakehouse_in_action.favorita_forecasting.stores_ft",
      "lookup_key":"store_nbr",
      "feature_names":["cluster","store_type"]
    }
]

automl_data = raw_data.filter("date > '2016-12-31'")

summary = databricks.automl.regress(automl_data, 
                                    target_col=label_name,
                                    time_col="date", 
                                    timeout_minutes=60, 
                                    feature_store_lookups=automl_feature_lookups)

方法3：处理exclude_cols的限制 #

exclude_cols只能排除原始数据集的列，若要移除特征表引入的列，需在关联完成后手动删除：

# 假设关联后的数据集中有多余列
final_data = final_data.drop("city", "state", "price_date")

内容的提问来源于stack exchange，提问作者Climbs_lika_Spyder