You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

PyAirbyte+Airflow同步Google Sheet至PostgreSQL为空问题求助

问题:PyAirbyte+Airflow同步Google Sheet至PostgreSQL表为空

我用PyAirbyte结合Airflow将Google Sheet数据同步到PostgreSQL表,代码执行无报错,但生成的表一直是空的。已经确认服务账号具备必要权限,怀疑可能漏掉了Airbyte库或source-google-sheets版本相关的额外步骤。

我的代码

from airflow.decorators import task

@task.external_python(python="/venv_airbyte/bin/python")
def extract():
    import airbyte as ab
    def get_credentials():
        with open(PATH_to_KEY) as f:
            return f.read()

    #VARIABLES
    PATH_to_KEY = "my_service_account.json"
    SPREADSHEET_ID = "1pYS24BwqRl5hIhVAjo1QFDsupTx67nYCTCTIF-PcVoY"
    DB_HOST="localhost"
    DB_PORT=5423
    DB_USR=""
    DB_PWD=""
    DB_NAME=""


    credentials = get_credentials() # 获取字符串格式的JSON服务账号凭证
    source = ab.get_source(
        "source-google-sheets",
        pip_url="airbyte-source-google-sheets unidecode",
        install_if_missing=True,
        config={
            "spreadsheet_id": SPREADSHEET_ID,
            "credentials": {
                "auth_type": "Service",
                "service_account_info": credentials,
            },
        },
    )
    source.select_all_streams()

    pg_cache = ab.caches.PostgresCache(
            host=DB_HOST,
            port=5432,
            username=DB_USR,
            password=DB_PWD,
            database=DB_NAME,
            schema_name="raw"
    )
    read_result = source.read(cache=pg_cache)

    # 调试:查看read_result结构和流信息
    print("Read Result Streams:", read_result.streams)

电子表格示例

电子表格示例

使用的版本信息

名称:airbyte 版本:0.24.2
概述:PyAirbyte
依赖:airbyte-api, airbyte-cdk, airbyte-protocol-models-pdv2, click, duckdb, duckdb-engine, google-auth, google-cloud-bigquery, google-cloud-bigquery-storage, google-cloud-secret-manager, jsonschema, orjson, overrides, pandas, psycopg, psycopg2-binary, pyarrow, pydantic, pydantic-core, python-dotenv, python-ulid, pytz, pyyaml, requests, rich, snowflake-connector-python, snowflake-sqlalchemy, sqlalchemy, sqlalchemy-bigquery, structlog, typing-extensions, uuid7


内容的提问来源于stack exchange,提问作者dev_community

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.06.13 10:46:21