Pandas DataFrame切片性能受前期子集赋值方式影响的问题求助

最近我碰到了一个搞不懂的Pandas性能问题，在对DataFrame做切片操作时，之前给额外列赋值的方式居然会影响后续循环的执行速度！

我写了一段没有实际业务意义但能凸显问题的代码，逻辑如下：

• 创建一个包含两类列的DataFrame：extra_columns和columns
• 耗时的核心部分是循环里对columns切片的赋值操作

最让我百思不得其解的是，循环前给extra_columns赋值的不同方式，居然会让循环的性能产生天壤之别！

Python代码 #

import timeit

setup_stmt ="""
import pandas as pd
num_cols = 500
n_iter = 100
extra_column = [ "product"]
columns = [chr(i+65) for i in range(num_cols)]
index= range(n_iter)
"""

stmt1 ="""
df = pd.DataFrame(index = index, columns=extra_column + columns)
df["product"] = "x"
for i in index:
    df.loc[i,columns] = 0
"""

stmt2 ="""
df = pd.DataFrame(index = index, columns=extra_column + columns)
df.product = "x"            
for i in index:
    df.loc[i,columns] = 0
"""

stmt3 ="""
df = pd.DataFrame(index= index, columns=extra_column + columns)
df.loc[index,"product"] = "x"
for i in index:
    df.loc[i,columns] = 0
"""

stmt4 ="""
df = pd.DataFrame(index = index, columns=extra_column + columns)
for i in index:
    df.loc[i,columns] = 0
df["product"] = "x"
"""

print(f" stmt1 takes { timeit.timeit(setup= setup_stmt, stmt= stmt1,  number=10):2.2f} seconds" )
print(f" stmt2 takes { timeit.timeit(setup= setup_stmt, stmt= stmt2,  number=10):2.2f} seconds" )
print(f" stmt3 takes { timeit.timeit(setup= setup_stmt, stmt= stmt3,  number=10):2.2f} seconds" )
print(f" stmt4 takes { timeit.timeit(setup= setup_stmt, stmt= stmt4,  number=10):2.2f} seconds" )

输出结果 #

stmt1 takes 20.60 seconds
 stmt2 takes 0.46 seconds
 stmt3 takes 0.46 seconds
 stmt4 takes 0.46 seconds

备注：内容来源于stack exchange，提问作者user1573820