Pandas apply将矩阵转为nan/None的问题排查求助

我在数据集上运行代码，统计符合多条件的行数，使用Pandas apply函数将统计结果存入矩阵，并调用辅助函数修改矩阵单个值。但矩阵列有时会变为含NaN的float类型，或在代码迭代中变为NoneType，导致**“类型不可索引”**错误（前91次循环成功，第92次报错）。

报错相关函数代码 #

#######################################
# takes in year and month ex: ("12", "01")
# loads creates time series data with matrix of shared rides
#######################################
def processMonth(year, month):
    print("Processing " + year + "-" + month)
    
    #read in correct month data
    currentMonth = pd.read_parquet("blocked/20" + year + "/data_wBlocks_20" + year + "-" + month + ".parquet")

    #Cleaning Data:
    #remove whitespace from columns
    currentMonth.columns = currentMonth.columns.str.replace(' ', '')
    #drop rows with no match in either side
    currentMonth.dropna(inplace=True)
    #convert datetime to date
    currentMonth["pickup_datetime"] = currentMonth["pickup_datetime"].dt.date
    
    #create new df as time series on business days for daily ride vols
    MonthTS = pd.DataFrame({'date' : pd.Series(pd.date_range(datetime.date(2000 + int(year), int(month), 1), end=datetime.date(2000 + int(year), int(month), 1) + pd.DateOffset(months=1) - pd.DateOffset(days=1), freq='D'))})
    #for each day of the year, create empty matrix to initialize with ride volumes
    MonthTS["matrix"] = [np.zeros((len(firms.index), len(brokers.index))) for x in range(len(MonthTS))]
    MonthTS['date'] = MonthTS['date'].dt.date
    MonthTS = MonthTS.set_index(['date'])

    #iterate over combonations of firm and brokerage, and append to matrix
    for idx1, firm in firms.iterrows():
        #MonthTS["matrix"] = MonthTS["matrix"].apply(lambda x : x.append([]))
        for idx2, broker in brokers.iterrows():
            #tallying taxi rides on the given day in either direction between firms and brokers
            tmp = currentMonth[(currentMonth["pu_block"] == str(firm.bctcb2010)) & (currentMonth["do_block"] == (broker.bctcb2010)) | ((currentMonth["do_block"] == str(firm.bctcb2010)) & (currentMonth["pu_block"] == str(broker.bctcb2010)))]
            tmp = tmp.groupby("pickup_datetime").count()
            MonthTS = pd.concat([MonthTS, tmp], axis = 1)
            #print(MonthTS.head(5))
            MonthTS["matrix"] = MonthTS.apply(lambda x: replaceVal(x, idx1, idx2), axis=1)
            MonthTS = MonthTS["matrix"]

    #at this point, for the index on a given day, matrix[i][j] represents the ride between firm i and brokerage j on the given day
    return MonthTS

调用的辅助函数代码 #

def replaceVal(x, idx1, idx2):
    if (x.pu_block == x.pu_block):
        (x.matrix)[idx1][idx2] = x.pu_block
    return x.matrix

问题现象 #

• 尝试用单组行组合复现错误失败，报错时x.matrix显示为NaN。
• apply前DataFrame状态：

matrix
date
2012-02-01 [[0, 1, 2, ...
2012-02-02 [[0, 1, 2, ...
2012-02-03 [[0, 1, 2, ...
...
2012-02-27 [[0, 1, 2, ...
2012-02-28 [[0, 1, 2, ...
2012-02-29 [[0, 1, 2, ...

• apply后状态：

matrix
date
2012-02-01 [[0, 1, 2, ...
2012-02-02 [[0, 1, 2, ...
2012-02-03 nan
...
2012-02-27 nan
2012-02-28 nan
2012-02-29 nan

内容的提问来源于stack exchange，提问作者Shand Seiffert