ClickHouse安全通道复制失败:不存在指定Interserver IO端点
我正在测试ClickHouse安全通道下的复制功能,参照官方文档配置后始终无法正常运行。插入数据时复制失效,副本节点MY-232报出如下错误:
2023.03.20 16:51:00.757079 [ 32722 ] {} mybd.mytable (b4a7593a-c2e1-4d2d-90ec-f8050c17a19a): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(DB::StorageReplicatedMergeTree::LogEntryPtr &) const: Code: 86. DB::HTTPException: Received error from remote server /?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F01%2Fmydb%2Fmytable%2Freplicas%2F02&part=20230320_66_71_1&client_protocol_version=7&compress=false. HTTP status code: 500 Internal Server Error, body: Code: 221. DB::Exception: No interserver IO endpoint named DataPartsExchange:/clickhouse/tables/01/mydb/mytable/replicas/02. (NO_SUCH_INTERSERVER_IO_ENDPOINT), Stack trace (when copying this message, always include the lines below):
安全通道端口(含跨节点安全端口9010)已正常监听,clickhouse-client、ZK 4lw等功能在安全通道下可正常使用。此前非安全通道环境下,由MY-232、MY-233两个ClickHouse Keeper节点组成的集群复制功能正常。目前向MY-233插入数据时MY-232报上述错误;向MY-232插入数据时MY-233出现DNS相关错误,正在调试该问题。已查阅相关社区讨论,现寻求问题排查思路与建议。
排查思路与建议
- 检查跨节点SSL配置完整性:确保两个节点都正确配置
interserver_https_port,同时确认SSL证书(CA证书、节点证书/密钥)在双节点上部署完整,且权限配置正确(ClickHouse进程需具备证书文件读取权限)。 - 统一
interserver_http_host配置格式:当前节点分别使用IP地址配置,建议统一切换为主机名(如MY-232.ch_test.local、MY-233.ch_test.local),同时确认SSL证书的SAN字段包含对应主机名,避免证书验证失败导致端点注册异常。 - 验证跨节点端点可达性:在MY-232上执行curl命令测试MY-233的9010端口端点:
(curl -k https://MY-233.ch_test.local:9010/?endpoint=DataPartsExchange:/clickhouse/tables/01/mydb/mytable/replicas/02-k仅用于测试阶段跳过证书验证,正式环境需移除并确保证书信任链正常) - 检查ZK复制元数据一致性:登录ClickHouse Keeper,查看
/clickhouse/tables/01/mydb/mytable/replicas路径下的节点信息,确认两个副本的端点注册状态是否正常,是否存在元数据不一致情况。 - 核对
macros与表定义匹配性:确保ReplicatedMergeTree表路径'/clickhouse/tables/{shard}/mydb/mytable'与节点macros中的shard值完全匹配,且双节点的replica宏值(01、02)唯一且对应正确。 - 补全MY-233的Keeper配置:当前提供的MY-233配置中
keeper_server部分不完整,需确认其zookeeper客户端配置、raft_configuration等是否与MY-232对称,避免Keeper集群通信异常影响复制元数据同步。
相关配置信息
--- from MY-232 <interserver_https_port>9010</interserver_https_port> <interserver_http_host>10.5.106.232</interserver_http_host> <!-- 曾尝试配置为“MY-232.ch_test.local” --> <zookeeper> <node> <host>MY-232.ch_test.local</host> <port>9281</port> <secure>1</secure> </node> <node> <host>MY-233.ch_test.local</host> <port>9281</port> <secure>1</secure> </node> </zookeeper> <keeper_server> <tcp_port>9281</tcp_port> <server_id>1</server_id> <log_storage_path>/var/lib/clickhouse/coordination/log</log_storage_path> <snapshot_storage_path>/var/lib/clickhouse/coordination/snapshots</snapshot_storage_path> <coordination_settings> <operation_timeout_ms>10000</operation_timeout_ms> <session_timeout_ms>30000</session_timeout_ms> <raft_logs_level>trace</raft_logs_level> </coordination_settings> <raft_configuration> <server> <id>1</id> <hostname>MY-232.ch_test.local</hostname> <port>9444</port> <secure>1</secure> </server> <server> <id>2</id> <hostname>MY-233.ch_test.local</hostname> <port>9444</port> <secure>1</secure> </server> </raft_configuration> </keeper_server> <remote_servers> <cluster_1s_2r> <shard> <internal_replication>true</internal_replication> <replica> <host>MY-232.ch_test.local</host> <port>9440</port> <user>default</user> <password></password> <secure>1</secure> </replica> <replica> <host>MY-233.ch_test.local</host> <port>9440</port> <user>default</user> <password></password> <secure>1</secure> </replica> </shard> </cluster_1s_2r> </remote_servers> <macros> <shard>01</shard> <replica>01</replica> </macros> --- MY-233 <interserver_https_port>9010</interserver_https_port> <interserver_http_host>10.5.106.233</interserver_http_host> <keeper_server> <tcp_port_secure>9281</tcp_port_secure> <server_id>2</server_id> <macros> <shard>01</shard> <replica>02</replica> </macros> --- /etc/hosts 10.5.106.232 MY-232.ch_test.local 10.5.106.233 MY-233.ch_test.local --- 表结构 CREATE TABLE mydb.mytable ON CLUSTER cluster_1s_2r ( `field1` String, ….. `fieldX` Int32 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/mydb/mytable', '{replica}')
内容的提问来源于stack exchange,提问作者Jae

