JDK8下用Byteman插桩ArrayList.size()引发死锁的解决方法问询

问题背景 #

在JDK8环境下结合Maven测试使用Byteman，给java.util.ArrayList.size()方法插了一段随机休眠代码后，触发了Java级死锁，疑似Byteman实现层面的问题。

插桩代码 #

java.util.Random rand = new java.util.Random();
try{
    int time = rand.nextInt(10);
    Thread.sleep(time);
    // System.out.println("sleep "+time+" ms");
} catch (InterruptedException e) {
    e.printStackTrace();
}

检测到的死锁信息 #

Found one Java-level deadlock:
=============================
"pool-1-thread-1":
  waiting to lock monitor 0x00007fee50003308 (object 0x00000000aad29098, a sun.misc.URLClassPath),
  which is held by "surefire-forkedjvm-command-thread"
"surefire-forkedjvm-command-thread":
  waiting to lock monitor 0x00007fee80005fe8 (object 0x00000000a342fc48, a org.jboss.byteman.modules.ClassbyteClassLoader),
  which is held by "pool-1-thread-1"

Java stack information for the threads listed above:
===================================================
"pool-1-thread-1":
        at sun.misc.URLClassPath.knownToNotExist(URLClassPath.java:379)
        - waiting to lock <0x00000000aad29098> (a sun.misc.URLClassPath)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:336)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:405)
        - locked <0x00000000a342fc48> (a org.jboss.byteman.modules.ClassbyteClassLoader)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        at sun.misc.Unsafe.defineClass(Native Method)
        at sun.reflect.ClassDefiner.defineClass(ClassDefiner.java:63)
        at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:399)
        at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:394)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.reflect.MethodAccessorGenerator.generate(MethodAccessorGenerator.java:393)
        at sun.reflect.MethodAccessorGenerator.generateConstructor(MethodAccessorGenerator.java:92)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:55)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.jboss.byteman.rule.Rule.execute(Rule.java:817)
        at org.jboss.byteman.rule.Rule.execute(Rule.java:789)
        at java.util.ArrayList.size(ArrayList.java:284)
        at sun.misc.URLClassPath.getLoader(URLClassPath.java:510)
        - locked <0x00000000aaf16ad8> (a sun.misc.URLClassPath)
        at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:495)
        - locked <0x00000000aaf16ad8> (a sun.misc.URLClassPath)
        at sun.misc.URLClassPath.getResource(URLClassPath.java:249)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        - locked <0x00000000a57ffbc8> (a java.lang.Object)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:405)
        - locked <0x00000000a57ffb18> (a java.lang.Object)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        at org.junit.runner.notification.RunNotifier.fireTestStarted(RunNotifier.java:153)
        at org.apache.maven.surefire.common.junit4.Notifier.fireTestStarted(Notifier.java:100)
        at org.junit.internal.runners.model.EachTestNotifier.fireTestStarted(EachTestNotifier.java:42)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:364)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
        at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
        at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
"surefire-forkedjvm-command-thread":
        at sun.misc.Unsafe.defineClass(Native Method)
        at sun.reflect.ClassDefiner.defineClass(ClassDefiner.java:63)
        at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:399)
        at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:394)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.reflect.MethodAccessorGenerator.generate(MethodAccessorGenerator.java:393)
        at sun.reflect.MethodAccessorGenerator.generateConstructor(MethodAccessorGenerator.java:92)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:55)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.jboss.byteman.rule.Rule.execute(Rule.java:817)
        at org.jboss.byteman.rule.Rule.execute(Rule.java:789)
        at java.util.ArrayList.size(ArrayList.java:284)
        at sun.misc.URLClassPath.getLoader(URLClassPath.java:510)
        - locked <0x00000000aad29098> (a sun.misc.URLClassPath)
        at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:495)
        - locked <0x00000000aad29098> (a sun.misc.URLClassPath)
        at sun.misc.URLClassPath.getResource(URLClassPath.java:249)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        - locked <0x00000000a55f3b68> (a java.lang.Object)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        at org.apache.maven.surefire.booter.MasterProcessCommand.decode(MasterProcessCommand.java:139)
        at org.apache.maven.surefire.booter.CommandReader$CommandRunnable.run(CommandReader.java:391)
        at java.lang.Thread.run(Thread.java:750)

Found 1 deadlock.

死锁原因 #

从栈信息可看出，死锁源于类加载器相关锁的交叉持有：

1. pool-1-thread-1持有ClassbyteClassLoader锁，等待URLClassPath锁
2. surefire-forkedjvm-command-thread持有URLClassPath锁，等待ClassbyteClassLoader锁

本质是Byteman执行插桩代码时触发反射方法生成，该过程又调用了被插桩的ArrayList.size()，形成递归触发+锁交叉。

解决方法 #

1. 更换插桩目标方法 #

ArrayList.size()属于JDK核心基础方法，被类加载器等底层代码调用，插桩后极易引发递归和锁冲突。建议改为插桩业务代码中的方法，避开JDK核心集合类的基础方法。

2. 给插桩规则添加线程过滤条件 #

在Byteman规则中加入判断，避免在类加载相关线程中执行插桩逻辑：

RULE avoid deadlock in ArrayList.size()
CLASS java.util.ArrayList
METHOD size()
CONDITION !Thread.currentThread().getName().contains("surefire-forkedjvm-command-thread")
AT ENTRY
DO
    java.util.Random rand = new java.util.Random();
    try{
        int time = rand.nextInt(10);
        Thread.sleep(time);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
ENDRULE

3. 预加载插桩依赖类 #

在测试启动前提前加载插桩代码用到的类，避免插桩执行时触发类加载：
在测试类的@BeforeClass方法中添加：

Class.forName("java.util.Random");

4. 升级Byteman版本 #

部分旧版本Byteman在核心类插桩的锁机制上存在缺陷，尝试升级到最新稳定版（如4.0.20+），官方可能已修复此类问题。

5. 调整Surefire插件配置 #

修改Maven Surefire插件配置，关闭并行测试或限制线程数，减少并发锁冲突：

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.22.2</version>
    <configuration>
        <parallel>none</parallel>
        <forkCount>1</forkCount>
        <reuseForks>true</reuseForks>
    </configuration>
</plugin>

内容的提问来源于stack exchange，提问作者user8682225