如何在Python中为LightGBM模型绘制学习曲线？附代码咨询

Nice question! I get it, Keras makes grabbing training history super straightforward with that history object, but LightGBM does things a bit differently—don't worry, it's just as easy once you know where to look. Here's how you can adapt your existing code to plot learning curves just like you do in Keras:

Key Background #

When you train a LightGBM model with eval_set and eval_metric specified, the model automatically saves every round's training/validation metrics in its evals_result_ property. This is LightGBM's equivalent of Keras' history.history dictionary.

Modified Code with Learning Curve Plotting #

Here's your updated function, with added logic to extract metrics and generate the learning curve:

def f_lgboost(data, params):
    import matplotlib.pyplot as plt
    from sklearn.metrics import roc_auc_score, precision_recall_fscore_support
    import lightgbm as lgb

    model = lgb.LGBMClassifier(**params)
    X_train = data['X_train']
    y_train = data['y_train']
    X_dev = data['X_dev']
    y_dev = data['y_dev']
    X_test = data['X_test']
    categorical_feature= ['Ticker_code', 'Category_code']
    
    # Convert categorical features
    X_train[categorical_feature] = X_train[categorical_feature].astype('category')
    X_dev[categorical_feature] = X_dev[categorical_feature].astype('category')
    X_test[categorical_feature] = X_test[categorical_feature].astype('category')
    
    feature_name = X_train.columns.to_list()
    # Train the model - we'll add the training set to eval_set to capture its metrics
    model.fit(
        X_train, y_train,
        eval_set = [(X_train, y_train), (X_dev, y_dev)],
        eval_metric = 'auc',
        early_stopping_rounds = 20,
        categorical_feature = categorical_feature,
        feature_name = feature_name,
        verbose=10  # Optional: Print metrics every 10 rounds for visibility
    )
    
    # Extract all training/validation metrics from the model
    evals_result = model.evals_result_
    
    # Plot the AUC learning curve
    plt.figure(figsize=(10, 6))
    plt.plot(evals_result['training']['auc'], label='Train AUC')
    plt.plot(evals_result['valid_1']['auc'], label='Validation AUC')
    plt.title('LightGBM Learning Curve (AUC)')
    plt.xlabel('Training Rounds')
    plt.ylabel('AUC Score')
    plt.legend()
    plt.grid(alpha=0.3)
    plt.show()
    
    # Calculate your existing metrics (unchanged from your original code)
    y_pred_train = model.predict_proba(X_train)[:, 1].ravel()
    y_pred_dev = model.predict_proba(X_dev)[:, 1].ravel()
    
    auc_train = roc_auc_score(y_train, y_pred_train)
    auc_dev = roc_auc_score(y_dev, y_pred_dev)
    
    precision, recall, fscore, support = precision_recall_fscore_support(
        y_dev, 
        (y_pred_dev > 0.5).astype(int), 
        beta=0.5
    )
    
    y_pred_test = model.predict_proba(X_test)[:, 1].ravel()
    print(f'auc_train: {auc_train:.4f}, auc_dev : {auc_dev:.4f}, precision : {precision[1]:.4f}, recall: {recall[1]:.4f}, fscore : {fscore[1]:.4f}')
    
    # Fixed a small bug here: replaced undefined 'bst' with 'model'
    Results = {
        'params' : params,
        'data' : data,
        'lg_boost_model' : model,
        'y_pred_train' : y_pred_train,
        'y_pred_dev' : y_pred_dev,
        'y_pred_test' : y_pred_test,
        'auc_train' : auc_train,
        'auc_dev' : auc_dev,
        'precision_dev': precision,
        'recall_dev' : recall,
        'fscore_dev' : fscore,
        'support_dev' : support,
        'evals_result': evals_result  # Optional: Save metrics for later use
    }
    return Results

Important Notes #

• I added (X_train, y_train) to eval_set so we can track both training and validation AUC across rounds. Without this, you'd only get validation metrics.
• The evals_result dictionary is structured like:

  {'training': {'auc': [0.85, 0.87, ...]}, 'valid_1': {'auc': [0.83, 0.86, ...]}}
  

  Where valid_0 corresponds to the first entry in eval_set (training set) and valid_1 to the second (validation set).
• If you use multiple evaluation metrics (e.g., eval_metric=['auc', 'logloss']), evals_result will include sequences for each metric—you can plot them separately the same way.
• I fixed a minor bug in your original code: you were trying to save bst to the results dict, but your model variable is named model. That would have thrown an error otherwise.

Extra Tips #

• If you don't want training logs cluttering your output, set verbose=0 in the fit() call.
• To save the plot to a file instead of just showing it, add plt.savefig('lightgbm_learning_curve.png', dpi=300) before plt.show().

内容的提问来源于stack exchange，提问作者user8270077