实时整库同步常见问题--大数据研发治理套件-火山引擎

文档中心

导航

实时整库同步常见问题

最近更新时间：2025.07.17 11:35:25首次发布时间：2025.07.17 11:35:25

MySQL2ByteHouse_CDW 实时整库同步常见问题

表结构中或解决方案配置时带切分键，任务 oom

问题现象：
MySQL2ByteHouse_CDW 实时整库同步方案执行全量同步任务时，源端表数据量比较大时，日志提示 OutOfMemoryError: Java heap space，详细错误日志如下：

Caused by: java.lang.OutOfMemoryError: Java heap space
        at java.math.BigInteger.<init>(BigInteger.java:1123) ~[?:1.8.0_181]
        at java.math.BigInteger.valueOf(BigInteger.java:1107) ~[?:1.8.0_181]
        at com.bytedance.dts.batch.jdbc.split.QuickSplitOneShardCallable.calculateRanges(QuickSplitOneShardCallable.java:47) ~[?:?]
        at com.bytedance.dts.batch.jdbc.split.SplitOneShardCallable.call(SplitOneShardCallable.java:206) ~[?:?]
        at com.bytedance.dts.batch.jdbc.split.SplitOneShardCallable.call(SplitOneShardCallable.java:45) ~[?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_181]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_181]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_181]
        at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]

解决方案：
- 方案一：解决方案配置时，去掉切分键配置
- 方案二：在解决方案配置 > 资源组高级配置 > 离线全量同步高级参数配置中，配置以下高级参数：job.reader.shard_split_mode=nosplit、job.common.is_use_batch_mode=false

批量写入分区限制数问题

问题现象：
MySQL2ByteHouse_CDW 实时整库同步中离线同步任务执行失败，错误关键字提示：errbook_Gateway_InsertToWorker_WaitFail_IEToo many partitions for single INSERT block (more than 100)，详细错误日志如下：

2024-11-07 14:17:18
java.io.IOException: Couldn't write data - {"byteSize":8,"rawData":1800003},{"byteSize":0,"rawData":""},{"byteSize":24,"rawData":"m_O9qaP1s2uci0r_O9qaP1s2"},{"byteSize":93,"rawData":"http://1252524126.vod2.myqcloud.com/522ff1e0vodcq1252524126/5f394e04387702291367856795/f0.aac"},{"byteSize":0,"rawData":""},{"byteSize":32,"rawData":"cb9961e1403552fe3fa0f3df6379f243"},{"byteSize":8,"rawData":1},{"byteSize":8,"rawData":"30749.92"},{"byteSize":8,"rawData":0},{"byteSize":8,"rawData":0},{"byteSize":0},{"byteSize":8,"rawData":0},{"byteSize":0,"subType":"DATETIME"},{"byteSize":8,"rawData":1636772979000,"subType":"DATETIME"},{"byteSize":8,"rawData":1652747646000,"subType":"DATETIME"}
 at com.bytedance.bitsail.core.bytedance.flink.bridge.writer.delegate.DelegateFlinkWriter.processElement(DelegateFlinkWriter.java:302)
 at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:191)
 at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.processElement(StreamTaskNetworkInput.java:188)
 at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.emitNext(StreamTaskNetworkInput.java:163)
 at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:67)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:478)
 at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:191)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:930)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.executeInvoke(StreamTask.java:876)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:887)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:860)
 at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:1036)
 at org.apache.flink.runtime.taskmanager.Task.run(Task.java:807)
 at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: com.bytedance.bitsail.common.BitSailException: Code:[Common-05], Describe:[Internal error found in BitSail. This is usually a bug.] - com.bytedance.bytehouse.exception.ByteHouseSQLException: errbook_Gateway_InsertToWorker_WaitFail_IEToo many partitions for single INSERT block (more than 100). The limit is controlled by 'max_partitions_per_insert_block' setting. Large number of partitions is a common misconception. It will lead to severe negative performance impact, including slow server startup, slow INSERT queries and slow SELECT queries. Recommended total number of partitions for a table is under 1000..10000. Please note, that partitioning is not intended to speed up SELECT queries (ORDER BY key is sufficient to make range queries fast). Partitions are intended for data manipulation (DROP PARTITION, etc). SQLSTATE: HY000. Stack trace:

解决方案：
该错误提示根因是因为目前离线任务默认批量写入的分区数限制为100，如果分区数超过100，则会报错。您可在解决方案离线全量同步中添加集成高级参数： job.writer.bh_connection_properties={"max_partitions_per_insert_block":"10000"}

缺失唯一键：errbook_Gateway_InsertToWorker_WaitError_IENo such column delete_flag in table

问题现象：
MySQL2ByteHouse_CDW 实时整库同步中离线同步任务执行失败，错误关键字提示：errbook_Gateway_InsertToWorker_WaitError_IENo such column delete_flag in table，详细错误日志如下：

2024-11-07 12:17:43,176 ERROR org.apache.flink.runtime.executiongraph.ExecutionGraph       flink-akka.actor.default-dispatcher-17                       [] - bytehouse_writer(1/4) - execution #0 (da959709c09229c65fa17737766cec9a) switched from RUNNING to FAILED on s-1854376938343854082-taskmanager-1-2 @ 33.1.192.126 (dataPort=39739) java.io.IOException: Couldn't write data - {"byteSize":8,"rawData":50000},{"byteSize":15,"rawData":"appyMlpdpb01900"},{"byteSize":36,"rawData":"099aad74-3131-48e5-926e-44b3185ac6db"},{"byteSize":102,"rawData":"http://wechatapppro-1252524126.file.myqcloud.com/appyMlpdpb01900/image/ueditor/93591500_1540728476.jpg"},{"byteSize":0,"rawData":""},{"byteSize":32,"rawData":"8468cafad05789f05bbc2911f2c925dc"},{"byteSize":8,"rawData":1},{"byteSize":5,"rawData":"14.99"},{"byteSize":8,"rawData":0},{"byteSize":8,"rawData":0},{"byteSize":0},{"byteSize":8,"rawData":0},{"byteSize":8,"rawData":1643212527000,"subType":"DATETIME"},{"byteSize":8,"rawData":1579282487000,"subType":"DATETIME"},{"byteSize":8,"rawData":1579282564000,"subType":"DATETIME"}
 at com.bytedance.bitsail.core.bytedance.flink.bridge.writer.delegate.DelegateFlinkWriter.processElement(DelegateFlinkWriter.java:302)
 at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:191)
 at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.processElement(StreamTaskNetworkInput.java:188)
 at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.emitNext(StreamTaskNetworkInput.java:163)
 at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:67)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:478)
 at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:191)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:930)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.executeInvoke(StreamTask.java:876)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:887)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:860)
 at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:1036)
 at org.apache.flink.runtime.taskmanager.Task.run(Task.java:807)
 at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: com.bytedance.bitsail.common.BitSailException: Code:[Common-05], Describe:[Internal error found in BitSail. This is usually a bug.] - java.lang.RuntimeException: com.bytedance.bytehouse.exception.ByteHouseSQLException: errbook_Gateway_InsertToWorker_WaitError_IENo such column _delete_flag_ in table `2100132197.db_material_center`.t_material_extend (3cadfe8c-5ae1-4b59-ba00-28936047fa02) SQLSTATE: 42703. Stack trace:

解决方案：
实时整库同步时以离线+实时的方式，数据写入方式都是以 Upsert 方式写入，因此已创建的目标 ByteHouse CDW 表必须要设置UNIQUE KEY，否则会导致写入失败。表设置操作详见唯一键表。

CDW 没有默认资源组： BytehouseBaseResp(message=null, data=null, error=Error(message=no virtual warehouse was selected), meta=null)

问题现象：
MySQL2ByteHouse_CDW 实时整库同步中自动建表失败，错误关键字提示：Failed to create table, remote response: BytehouseBaseResp(message=null, data=null, error=Error(message=no virtual warehouse was selected), meta=null)，详细错误日志如下：

com.bytedance.dataplatform.dts.solution.exception.SolutionException: Create table error., info: Failed to create table, remote response: BytehouseBaseResp(message=null, data=null, error=Error(message=no virtual warehouse was selected), meta=null)
ErrorStack
com.bytedance.dataplatform.dts.manager.ByteHouseCdwManager.createTable(ByteHouseCdwManager.java:418) com.bytedance.dataplatform.dts.manager.ByteHouseCdwManager$$FastClassBySpringCGLIB$$552b7ae2.invoke(<generated>) org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

解决方案：
ByteHouse CDW 租户管理控制台中，没有设置默认计算组参数，需要手动设置，设置方法详见设置计算组。