如何通过Snowflake Snowpark将Pandas DataFrame写入Snowflake数据库?
问题
使用Snowflake Snowpark将Pandas DataFrame(如描述统计、相关矩阵、模型输出等转换结果)写入Snowflake数据库时,报错:
'DataFrame' object has no attribute 'write'
附原代码:
from snowflake.snowpark import Session from snowflake.snowpark import table from snowflake.snowpark.functions import udf from snowflake.snowpark.functions import col from snowflake.snowpark.types import StringType sess = None print('Connecting...') cnn_params = { "account": "eu-west-1", "user": "user", "password": 'pass', "warehouse": "xs", "database": "demodb", "schema": "demosch", "role": "ACCOUNTADMIN" } try: print('session...') import pandas as pd import numpy as np sess = Session.builder.configs(cnn_params).create() df = pd.DataFrame(np.random.randint(0,100,size=(100, 4)), columns=list('ABCD')) #print(df.describe()) df.describe().write.mode("overwrite").save_as_table("describe_output", table_type="temporary") except Exception as e: print(e) finally: if sess: sess.close() print('connection closed...') print('done.')
错误原因
df.describe()返回的是Pandas原生DataFrame,而write、save_as_table是Snowpark DataFrame专属的方法,Pandas DataFrame并没有这些属性,因此触发报错。
解决方案
需要先将Pandas DataFrame转换为Snowpark DataFrame,再调用写入方法。核心是使用Snowpark Session的create_dataframe()方法完成转换。
修改后的代码:
from snowflake.snowpark import Session import pandas as pd import numpy as np sess = None print('Connecting...') cnn_params = { "account": "eu-west-1", "user": "user", "password": 'pass', "warehouse": "xs", "database": "demodb", "schema": "demosch", "role": "ACCOUNTADMIN" } try: print('session...') sess = Session.builder.configs(cnn_params).create() # 生成Pandas DataFrame并计算描述统计 pd_df = pd.DataFrame(np.random.randint(0,100,size=(100, 4)), columns=list('ABCD')) desc_pd_df = pd_df.describe() # 将Pandas DataFrame转换为Snowpark DataFrame,reset_index把统计指标转为列 snowpark_df = sess.create_dataframe(desc_pd_df.reset_index()) # 写入Snowflake临时表 snowpark_df.write.mode("overwrite").save_as_table("describe_output", table_type="temporary") print("数据写入成功") except Exception as e: print(e) finally: if sess: sess.close() print('connection closed...') print('done.')
关键说明
reset_index():describe()生成的结果中,统计指标(如count、mean)是索引,转换为Snowpark DataFrame时需要将其转为普通列,否则会丢失这部分核心信息。sess.create_dataframe():这是Snowpark将Pandas DataFrame转为Snowpark DataFrame的核心方法,转换后才能使用Snowpark的写入API。
内容的提问来源于stack exchange,提问作者314mip
相关产品推荐
相关产品推荐

