Dask DataFrame列max()方法无法返回具体值的解决咨询

在使用Dask DataFrame时，调用列的max()方法不会直接返回具体的最大值，而是返回dd.Scalar<series-..., dtype=int32>这类对象，以下是相关代码及运行结果：

import numpy as np
import pandas as pd

import dask.dataframe as dd
import dask.array as da
import dask.bag as db

pandas_df = pd.DataFrame({'Name': ['Tom', 'Joseph', 'Krish', 'John'], 'Last Name': ['Johnson', 'Cameron', 'Biden', 'Washington'], 'Age': [20, 21, 19, 18], 'Weight': [60.0, 61.0, 62.0, 63.0]})  
print('pandas_df is')
print(pandas_df)  

print('max Age from Pandas Dataframe is')
print(str(pandas_df['Age'].max()))

ddf = dd.from_pandas(pandas_df, npartitions=1)

print('ddf is')
print(ddf.head(10))

print('max Age from Dask Dataframe is')
print(str(ddf['Age'].max()))

运行结果

pandas_df is
     Name   Last Name  Age  Weight
0     Tom     Johnson   20    60.0
1  Joseph     Cameron   21    61.0
2   Krish       Biden   19    62.0
3    John  Washington   18    63.0
max Age from Pandas Dataframe is
21
ddf is
     Name   Last Name  Age  Weight
0     Tom     Johnson   20    60.0
1  Joseph     Cameron   21    61.0
2   Krish       Biden   19    62.0
3    John  Washington   18    63.0
max Age from Dask Dataframe is
dd.Scalar<series-..., dtype=int32>

解决方法 #

Dask采用懒执行机制，调用max()只是生成了计算任务，并没有实际执行计算。要获取具体的最大值，需要在max()后调用.compute()方法触发计算：

修改后的代码片段：

print('max Age from Dask Dataframe is')
print(str(ddf['Age'].max().compute()))

修改后的运行结果

max Age from Dask Dataframe is
21

内容的提问来源于stack exchange，提问作者Henry