You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

在Dataiku中使用Python Recipe运行BigQuery SQL查询的认证问题求助

在Dataiku中使用Python Recipe运行BigQuery SQL查询的认证问题求助

我现在有个需求,要在Dataiku的Python Recipe里以指定格式运行BigQuery的SQL查询。在本地环境中,我可以通过设置服务账号凭证JSON文件的路径,用它来完成查询的认证,但到了Dataiku里,我就搞不清楚该怎么正确使用这个JSON文件了。

我已经尝试了两种方法,都遇到了错误,具体情况如下:

方法1:直接设置环境变量指定凭证文件

import dataiku
from dataiku import pandasutils as pdu
import pandas as pd
import google.cloud.bigquery as bigquery

from google.cloud import bigquery
import os
import pandas as pd

cred_file = "adc.json"
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]=cred_file
os.environ["GOOGLE_CLOUD_PROJECT"] = "PROJECT_ID"
def query_bq(QUERY):
    client = bigquery.Client(project="PROJECT_ID")
    query_job = client.query(QUERY)  # API request
    rows = query_job.result()  # Waits for query to finish
    df = rows.to_dataframe()
    return df


# Initialize BigQuery client
client = bigquery.Client()

# Step 1: Fetch SQL snippets from BigQuery
fetch_sql_query = """
SELECT DISTINCT Col1, Col2, col_3, Col_4 FROM  `BQ_Table_name`
"""

query_job = client.query(fetch_sql_query)
snippets = query_job.result()

遇到的错误:

DefaultCredentialsError: File adc.json was not found.

方法2:使用Dataiku的托管文件夹存放凭证文件

我在Dataiku里创建了一个托管文件夹,把adc.json上传进去了,然后用下面的代码尝试加载凭证:

#created a managed folder in Dataiku and uploaded the adc.json file into it.
folder_path = dataiku.Folder("folder_name").get_path()

credentials = service_account.Credentials.from_service_account_file(
"folder_path/adc.json"
)
client = bigquery.Client(credentials=credentials, project='PROJECT_ID')

遇到的错误:

MalformedError: Service account info was not in the expected format, missing fields token_uri, client_email.

补充一下,方法1在我本地环境里运行完全没问题,只有在Dataiku的Python Notebook里执行才会出这些错误。

备注:内容来源于stack exchange,提问作者Sreeram Godavarthy

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.04.14 17:54:31