Python 3.6.5中KFold拆分数据遇TypeError问题求助

Hey there! Let's get to the bottom of that frustrating TypeError you're seeing when splitting your data for k-fold cross validation.

What's Causing the Error? #

The line throwing the error:

Z_train, Z_test, label_train, label_test = Z[train], Z[test], label[train], label[test]

is failing because Z and label are regular Python lists, but train and test returned by KFold.split() are numpy integer arrays. Python lists don't support indexing with numpy arrays—they only accept single integers, slices, or standard Python list indices. That's exactly why you're getting the message: TypeError: only integer scalar arrays can be converted to a scalar index.

Two Easy Fixes #

Option 1: Convert Z and label to Numpy Arrays #

Numpy arrays natively support indexing with other numpy arrays, so this is the simplest fix. Right after you finish populating Z and label, add these lines:

Z = np.array(Z)
label = np.array(label)

Your original splitting line will work perfectly after this change.

Option 2: Use List Comprehensions (No Numpy Conversion Needed) #

If you prefer to keep Z and label as Python lists, use list comprehensions to extract the training/testing elements manually:

Z_train = [Z[i] for i in train]
Z_test = [Z[i] for i in test]
label_train = [label[i] for i in train]
label_test = [label[i] for i in test]

This loops through the indices in train/test and pulls the corresponding elements from your lists.

Bonus: Fixing Other Potential Issues in Your Code #

While we're at it, let's address a few other things that might cause problems down the line:

• Missing Import: You're using SVC but haven't imported it. Add this at the top of your code:

  from sklearn.svm import SVC
  

• Redundant Imports: You've imported modules like itemfreq, listdir, and join multiple times—clean those up to avoid clutter.
• Unused xrange: The astropy.extern.ply.cpp.xrange import is unnecessary. In Python 3, range behaves like Python 2's xrange, so replace all xrange calls with range.
• Empty Training/Testing Data: You're resetting X_train, Y_train, etc., to empty lists but never populating them with data from your file paths. Your SVC.fit() call will fail because it needs non-empty training data. Make sure to add code that reads files from mypath and appends data to X_train/Y_train.
• Path Handling: The ~/ in your file paths won't be resolved automatically. Use os.path.expanduser() to handle home directories correctly:

  mypath = os.path.expanduser('~/Users/awindmon/Documents/DISEASE_Example/')
  

Modified Working Code Snippet #

Here's a cleaned-up version of your code with the first fix applied:

from scipy.stats import itemfreq
from os import listdir
from os.path import isfile, join, expanduser
import numpy as np
import pickle
from csv import reader
from sklearn.model_selection import KFold
from sklearn.svm import SVC

seed = 0
np.random.seed(seed)

X_train = []
Y_train = []
X_test = []
Y_test = []
Z = []
label = []
eps=1e-7
set_probs = []
correct = 0
DISEASE = 1
NORMAL = 1

# Populate Z and label
for i in range(1,10):
    Z.append(DISEASE)
    DISEASE += 1
    label.append(1)

for i in range(1,10):
    Z.append(NORMAL)
    NORMAL += 1
    label.append(2)

# Convert to numpy arrays to fix indexing issue
Z = np.array(Z)
label = np.array(label)

print(len(Z))

kf = KFold(n_splits = 10, shuffle = True)
add1 = 0  # Initialize add1 to avoid UnboundLocalError
for train, test in kf.split(Z):
    X_train = []
    Y_train = []
    X_test = []
    Y_test = []
    set_probs = []
    
    # Now this line works!
    Z_train, Z_test, label_train, label_test = Z[train], Z[test], label[train], label[test]

    # Training set - add code to load data into X_train/Y_train here
    for z in range(len(Z_train)):
        if label_train[z] == 1:
            mypath = expanduser('~/Users/awindmon/Documents/DISEASE_Example/')
        else: # Use else to avoid missing cases
            mypath = expanduser('~/Users/awindmon/Documents/NORMAL_Example/')
        # Example: Load CSV files from mypath and append to X_train
        # files = [f for f in listdir(mypath) if isfile(join(mypath, f))]
        # for f in files:
        #     with open(join(mypath, f), 'r') as csvfile:
        #         csv_reader = reader(csvfile)
        #         data = list(csv_reader)
        #         X_train.append(data)
        #         Y_train.append(label_train[z])

    # Testing set - similarly load data into X_test/Y_test
    for z in range(len(Z_test)):
        if label_test[z] == 1:
            mypath = expanduser('~/Users/awindmon/Documents/DISEASE_Example/')
        else:
            mypath = expanduser('~/Users/awindmon/Documents/NORMAL_Example/')
        # Add data loading code here too

    # Only fit if X_train/Y_train are not empty!
    if X_train and Y_train and X_test and Y_test:
        clf = SVC(kernel='linear', random_state=0, gamma=1, C=1, probability=True)
        clf.fit(X_train, Y_train)
        filename = 'LinearSVM_Model.sav'
        pickle.dump(clf, open(filename, 'wb'))

        # Rest of your prediction code...
        count = 0
        probability_list = clf.predict_proba(X_test)
        p0=p1=p2=p3=p4=p5=p6=0
        set_probs = []
        for l in range(len(probability_list)):
            if l != 0 and l % 3 == 0:
                set_probs.append([p0,p1,p2,p3,p4,p5,p6])
                p0=p1=p2=p3=p4=p5=p6=0
            p0 += probability_list[l][0]
            p1 += probability_list[l][1]
            p2 += probability_list[l][2]
            p3 += probability_list[l][3]
            p4 += probability_list[l][4]
            p5 += probability_list[l][5]
            p6 += probability_list[l][6]
            if l == len(probability_list)-1:
                set_probs.append([p0,p1,p2,p3,p4,p5,p6])
        print(set_probs, Y_test)
        add1 += clf.score(X_test, Y_test)
print(add1/10)

内容的提问来源于stack exchange，提问作者public profile