WSO2 API Manager 3.2.0高可用端点故障问题
问题场景
已配置负载均衡类型端点,正常请求可正常路由至后端节点;但停止任一端点节点后,仍有2个请求被路由至故障节点,之后才切换至活跃节点;故障节点仅被标记为SUSPENDED而非inactive/down,该问题反复出现。
错误响应示例:
{"fault":{"code":101503,"type":"Status report","message":"Runtime Error","description":"Error connecting to the back end"}}
Carbon日志核心内容:
TID: [-1234] [] [2022-12-14 09:51:23,309] WARN {org.apache.synapse.endpoints.EndpointContext} - Endpoint : NewMCMInboundChannel-RESTAPIService--vv2_APIproductionEndpoint_1 with address http://100.66.2.32:7010/mcm-provider will be marked SUSPENDED as it failed
TID: [-1234] [] [2022-12-14 09:51:23,309] WARN {org.apache.synapse.endpoints.EndpointContext} - Suspending endpoint : NewMCMInboundChannel-RESTAPIService--vv2_APIproductionEndpoint_1 with address http://100.66.2.32:7010/mcm-provider - current suspend duration is : 30000ms - Next retry after : Wed Dec 14 09:51:53 UTC 2022
解决方案
1. 调整端点故障触发阈值
修改负载均衡端点的故障检测规则,让单次失败就触发端点挂起:
- 进入APIM Publisher,编辑目标API的生产/沙箱端点
- 展开负载均衡端点的高级配置,设置
Failover Threshold为1(默认值大于1,需多次失败才会触发挂起) - 根据业务需求调整
Suspend Duration(挂起时长),避免短时间内重试故障节点
2. 优化Passthru传输连接池配置
修改repository/conf/deployment.toml,避免连接池复用无效连接:
[transport.passthru_http] sender.connections.max_per_host = 10 sender.connection.timeout = 3000 sender.socket.timeout = 3000 sender.max_retries = 0
sender.max_retries设为0,禁止重试故障连接- 缩短连接/套接字超时,快速识别故障节点
3. 启用端点主动健康检查
通过主动探测替代被动故障检测,提前排除失效节点:
- 在端点高级配置中开启Health Check
- 设置健康检查的
Interval(探测间隔)、Retries(重试次数)和URL(后端健康检查接口) - 健康检查失败时,端点会被直接标记为不可用,避免请求路由
4. 切换负载均衡算法
将默认轮询算法切换为最少连接数算法,优先选择活跃节点:
- 在负载均衡端点配置中,修改
Load Balance Algorithm为leastconn,减少故障节点被选中的概率
内容的提问来源于stack exchange,提问作者PradeepKumar

