JMeter测试计划本地正常但K8s分布式模式无执行无报错

我在K8s环境部署了单Master、多Slave的JMeter架构。本地单节点跑测试计划完全正常，Slave节点本地执行也没问题，但用-R参数调用远程Slave执行时，任务直接完成，没执行任何请求，也没有报错信息。

执行命令 #

jmeter -Jserver.rmi.ssl.disable=true -n -t /tests/jmeter/GGSPerf_testplan.jmx -j jmeter_tests.log
jmeter -Jserver.rmi.ssl.disable=true -n -t /tests/jmeter/GGSPerf_testplan.jmx -j jmeter_tests.log -R XX.XX.XX.XX

对比情况 #

一个简单的示例测试计划可以正常通过分布式执行，但我的测试计划出现异常，测试计划代码如下：

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.1.1 r1855137">
<hashTree>
<TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan" enabled="true">
<stringProp name="TestPlan.comments"></stringProp>
<boolProp name="TestPlan.functional_mode">false</boolProp>
<boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp>
<boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
<elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
<collectionProp name="Arguments.arguments"/>
</elementProp>
<stringProp name="TestPlan.user_define_classpath"></stringProp>
</TestPlan>
<hashTree>
<HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
<collectionProp name="HeaderManager.headers">
<elementProp name="" elementType="Header">
<stringProp name="Header.name">Content-Type</stringProp>
<stringProp name="Header.value">text/plain</stringProp>
</elementProp>
</collectionProp>
</HeaderManager>
<hashTree/>
<Arguments guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
<collectionProp name="Arguments.arguments"/>
</Arguments>
<hashTree/>
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">100</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">8</stringProp>
<stringProp name="ThreadGroup.ramp_time">4</stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
</ThreadGroup>
<hashTree>
<HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
<collectionProp name="HeaderManager.headers">
<elementProp name="" elementType="Header">
<stringProp name="Header.name">Content-Type</stringProp>
<stringProp name="Header.value">application/json</stringProp>
</elementProp>
</collectionProp>
</HeaderManager>
<hashTree/>
<CSVDataSet guiclass="TestBeanGUI" testclass="CSVDataSet" testname="CSV Data Set Config" enabled="true">
<stringProp name="TestPlan.comments">Test configuration</stringProp>
<stringProp name="filename">${CSVFileName}</stringProp>
<stringProp name="fileEncoding">${CSVFileEncoding}</stringProp>
<stringProp name="variableNames">${CSVHeaderVariables}</stringProp>
<stringProp name="ignoreFirstLine">${CSVIgnoreHeader}</stringProp>
<stringProp name="delimiter">${CSVDelimeter}</stringProp>
<stringProp name="quotedData">${CSVAllowQuotedData}</stringProp>
<stringProp name="recycle">${CSVRecycleOnEOF}</stringProp>
<stringProp name="stopThread">${CSVStopThreadOnEOF}</stringProp>
<stringProp name="shareMode">${CSVSharingMode}</stringProp>
</CSVDataSet>
<hashTree/>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="AddressGeocodeRequest" enabled="true">
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">I have JSON body here</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp name="HTTPSampler.domain">${GGS_ServerName}</stringProp>
<stringProp name="HTTPSampler.port">${GGS_ServerPort}</stringProp>
<stringProp name="HTTPSampler.protocol"></stringProp>
<stringProp name="HTTPSampler.contentEncoding"></stringProp>
<stringProp name="HTTPSampler.path">${GGS_RequestPath}</stringProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<boolProp name="HTTPSampler.auto_redirects">false</boolProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
<stringProp name="HTTPSampler.embedded_url_re"></stringProp>
<stringProp name="HTTPSampler.connect_timeout"></stringProp>
<stringProp name="HTTPSampler.response_timeout"></stringProp>
</HTTPSamplerProxy>
<hashTree/>
<BackendListener guiclass="BackendListenerGui" testclass="BackendListener" testname="Backend Listener" enabled="true">
<elementProp name="arguments" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" enabled="true">
<collectionProp name="Arguments.arguments">
<elementProp name="graphiteMetricsSender" elementType="Argument">
<stringProp name="Argument.name">graphiteMetricsSender</stringProp>
<stringProp name="Argument.value">org.apache.jmeter.visualizers.backend.graphite.TextGraphiteMetricsSender</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
<elementProp name="graphiteHost" elementType="Argument">
<stringProp name="Argument.name">graphiteHost</stringProp>
<stringProp name="Argument.value">test-influxdb</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
<elementProp name="graphitePort" elementType="Argument">
<stringProp name="Argument.name">graphitePort</stringProp>
<stringProp name="Argument.value">2003</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
<elementProp name="rootMetricsPrefix" elementType="Argument">
<stringProp name="Argument.name">rootMetricsPrefix</stringProp>
<stringProp name="Argument.value">jmeter.</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
<elementProp name="summaryOnly" elementType="Argument">
<stringProp name="Argument.name">summaryOnly</stringProp>
<stringProp name="Argument.value">true</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
<elementProp name="samplersList" elementType="Argument">
<stringProp name="Argument.name">samplersList</stringProp>
<stringProp name="Argument.value"></stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
<elementProp name="percentiles" elementType="Argument">
<stringProp name="Argument.name">percentiles</stringProp>
<stringProp name="Argument.value">90;95;99</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp name="classname">org.apache.jmeter.visualizers.backend.graphite.GraphiteBackendListenerClient</stringProp>
</BackendListener>
<hashTree/>
</hashTree>
</hashTree>

日志内容（无有效报错） #

2019-09-30 10:37:36,873 INFO o.a.j.r.RmiUtils: Disabling SSL for RMI as server.rmi.ssl.disable is set to 'true'
2019-09-30 10:37:37,294 INFO o.a.j.e.ClientJMeterEngine: sent test to 10.56.0.27 basedir='tmp'
2019-09-30 10:37:37,294 INFO o.a.j.e.ClientJMeterEngine: Sending properties {}
2019-09-30 10:37:37,315 INFO o.a.j.e.ClientJMeterEngine: sent run command to 10.56.0.27
2019-09-30 10:37:37,315 INFO o.a.j.e.DistributedRunner: Remote engines have been started
2019-09-30 10:37:38,055 INFO o.a.j.JMeter: Started remote host: 10.56.0.26 (1569839858055)
2019-09-30 10:37:38,254 INFO o.a.j.r.Summariser: summary = 0 in 00:00:00 = ******/s Avg: 0 Min: 9223372036854775807 Max: -9223372036854775808 Err: 0 (0.00%)
2019-09-30 10:37:38,255 INFO o.a.j.JMeter: Finished remote host: 10.56.0.26 (1569839858255)
2019-09-30 10:37:38,613 INFO o.a.j.JMeter: Started remote host: 10.56.0.27 (1569839858613)
2019-09-30 10:37:38,706 INFO o.a.j.r.Summariser: summary = 0 in 00:00:00 = ******/s Avg: 0 Min: 9223372036854775807 Max: -9223372036854775808 Err: 0 (0.00%)
2019-09-30 10:37:38,707 INFO o.a.j.JMeter: Finished remote host: 10.56.0.27 (1569839858707)
2019-09-30 10:37:43,260 INFO o.a.j.JMeter: Interrupting RMI Reaper

问题分析与解决思路 #

从日志里的summary = 0能看出来，Slave节点根本没执行任何请求，结合你的测试计划和环境，大概率是这几个原因：

• 自定义变量未传递给Slave：你的测试计划里用了${CSVFileName}、${GGS_ServerName}等一堆变量，本地执行时你可能在配置里指定了这些值，但分布式执行时Master不会自动把本地变量同步给Slave。解决办法是在启动命令里明确传递这些变量，比如：

  jmeter -Jserver.rmi.ssl.disable=true -n -t /tests/jmeter/GGSPerf_testplan.jmx -j jmeter_tests.log -R XX.XX.XX.XX -JCSVFileName=/path/to/your.csv -JGGS_ServerName=your-server -JGGS_ServerPort=8080
  

  或者把变量配置到user.properties里，同步所有Slave节点的该文件。

• CSV文件未在Slave节点存在：如果CSV文件只在Master节点有，Slave找不到文件的话，会直接终止线程（尤其是CSVStopThreadOnEOF设为true时）。你可以把文件同步到所有Slave的相同路径，或者用K8s的PersistentVolume让所有Pod共享存储。

• 临时排除Backend Listener影响：虽然日志里没显示连接失败，但你可以先禁用Backend Listener试试，如果测试能正常执行，再排查Graphite的连接问题。

另外，一定要去Slave节点看jmeter-server.log，那里会有更详细的错误信息，比如找不到文件、变量未定义这些，Master日志通常不会同步Slave的详细报错。你也可以先简化测试计划：去掉CSV配置，把HTTP请求参数硬编码，再试分布式执行，定位问题根源。

内容的提问来源于stack exchange，提问作者Ajeet