You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

Flink CDC同步Postgres至StarRocks时Postgres JAR包缺失问题

我正在基于Flink搭建CDC管道,将PostgreSQL数据库的数据同步至StarRocks数据库。目前JobManager和TaskManager已正常启动,但启动CDC任务时始终报错:

Exception in thread "main" java.lang.RuntimeException: Cannot find factory with identifier "postgres-cdc" in the classpath.

Available factory classes are:

org.apache.flink.cdc.connectors.starrocks.sink.StarRocksDataSinkFactory
...

运行时能找到StarRocks连接器的JAR包,但同一目录(/opt/flink/usrlib/flink-cdc-3.4.0/lib/)下的PostgreSQL相关JAR包无法被识别。我是首次使用Flink,已经花了一天时间尝试配置但没解决,求解决方案。

容器配置

# JobManager
podman run \
  --name=flink_jobmanager \
  --detach \
  --env FLINK_PROPERTIES="jobmanager.rpc.address: 192.168.1.10" \
  --volume "/var/opt/flink/data:/opt/flink/usrlib" \
  --network flink_net \
  --publish 8081:8081 \
  --publish 6123:6123 \
  docker.io/flink:latest \
  jobmanager

# TaskManager
podman run \
  --name=flink_taskmanager \
  --detach \
  --env FLINK_PROPERTIES="jobmanager.rpc.address: 192.168.1.10" \
  --volume "/var/opt/flink/data:/opt/flink/usrlib" \
  --network flink_net \
  docker.io/flink:latest \
  taskmanager

# Start the job
podman run \
  --rm \
  --env FLINK_PROPERTIES="jobmanager.rpc.address: 192.168.1.10" \
  --volume "/var/opt/flink/data:/opt/flink/usrlib:rw" \
  --network flink_net \
  docker.io/flink:latest \
  /opt/flink/usrlib/flink-cdc-3.4.0/bin/flink-cdc.sh \
  /opt/flink/usrlib/postgres-to-starrocks.yaml

目录结构与JAR包

PostgreSQL JAR包参考官方文档下载,本地目录结构如下:

/var/opt/flink/data$ tree
.
├── flink-cdc-3.4.0
│   ├── bin
│   │   └── flink-cdc.sh
│   ├── conf
│   │   ├── flink-cdc.yaml
│   │   └── log4j-cli.properties
│   ├── lib
│   │   ├── flink-cdc-dist-3.4.0.jar
│   │   ├── flink-cdc-pipeline-connector-starrocks-3.4.0.jar
│   │   ├── flink-connector-postgres-cdc-3.4.0.jar
│   │   └── flink-sql-connector-postgres-cdc-3.4.0.jar
│   ├── LICENSE
│   ├── log
│   └── NOTICE
└── postgres-to-starrocks.yaml

同步任务配置文件(postgres-to-starrocks.yaml)

# Job properties
job:
  name:        postgres-to-starrocks
  parallelism: 4
  checkpointing:
    enabled:   true
    interval:  60000 # Checkpoint every 60 seconds (in milliseconds)
    mode:      EXACTLY_ONCE
    timeout:   600000 # Checkpoint timeout 10 minutes
    min-pause: 5000 # Minimum pause between checkpoints

source:
  type:          "postgres-cdc"
  hostname:      192.168.1.10
  port:          5432
  username:      username
  password:      "password"
  database-name: database
  schema-name:   public
  tables:        database.\.*

sink:
  type:          starrocks
  name:          StarRocks Sink
  jdbc-url:      "jdbc:mysql://192.168.1.10:9030"
  load-url:      192.168.1.10:8030
  username:      username
  password:      "password"
  database-name: test

内容的提问来源于stack exchange,提问作者usdn

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.06.12 19:45:18