执行sbt clean update遇scala-xml版本冲突及sbt run报错求助

一、Scala-XML版本冲突解决 #

执行sbt clean update遇到scala-xml版本冲突时，直接在build.sbt中强制统一版本：

// 替换为与你的Scala版本兼容的scala-xml版本，Scala 2.12推荐1.3.x系列
dependencyOverrides += "org.scala-lang.modules" %% "scala-xml" % "1.3.0"

若要定位冲突来源，可执行sbt dependencyTree查看依赖树，找到引入不同版本scala-xml的依赖项。

二、Hadoop-Client-API Jar找不到异常解决 #

程序执行成功后抛出java.nio.file.NoSuchFileException，原因是Hadoop的ShutdownHook线程在程序结束后尝试读取临时目录中的jar，但sbt已清理该目录。可通过以下方法解决：

方法1：阻止sbt清理临时目录 #

在build.sbt中添加配置，保留bg-jobs目录：

cleanKeepFiles += baseDirectory.value / "target" / "bg-jobs"

方法2：显式引入依赖 #

将hadoop-client-api添加到依赖列表，确保jar被下载到本地仓库而非临时目录：

libraryDependencies += "org.apache.hadoop" % "hadoop-client-api" % "3.3.4" % "compile"

方法3：禁用Hadoop ShutdownHook #

如果不影响业务功能，可关闭Hadoop的 shutdown hook：

• 在代码中添加：

import org.apache.hadoop.util.ShutdownHookManager;
ShutdownHookManager.get().clearShutdownHooks();

• 或在build.sbt中设置JVM参数：

javaOptions += "-Dhadoop.shutdown.hook=false"

报错日志（中文翻译版） #

sathvika@sathvika-vm:~/geotrellis-spark-hdfs$ sbt run
警告：发生了非法的反射访问操作
警告：org.jline.terminal.impl.exec.ExecTerminalProvider$ReflectionRedirectPipeCreator（文件:/home/sathvika/.sbt/boot/scala-2.12.19/org.scala-sbt/sbt/1.10.1/jline-terminal-3.24.1.jar）对构造函数java.lang.ProcessBuilder$RedirectPipeImpl()进行了非法反射访问
警告：请考虑向org.jline.terminal.impl.exec.ExecTerminalProvider$ReflectionRedirectPipeCreator的维护人员报告此问题
警告：使用--illegal-access=warn启用进一步非法反射访问操作的警告
警告：未来版本将拒绝所有非法访问操作
[info] 欢迎使用sbt 1.10.1（Oracle Corporation Java 11.0.23）
[info] 从/home/sathvika/geotrellis-spark-hdfs/project加载项目定义
[info] 从build.sbt加载geotrellis-spark-hdfs项目的设置...
[info] 将当前项目设置为geotrellis-spark-hdfs（构建文件:/home/sathvika/geotrellis-spark-hdfs/）
[info] 运行HDFSFileChecker2
Raster(ArrayMultibandTile(1,1,4,uint8raw),Extent(-4007.50166856125, 1.2501004675868899E-9, -1.2501004675868899E-9, 4007.50166856125))
[success] 总时间: 18 s, 完成于2024年8月5日 上午7:22:02
线程"Thread-5"中的异常: java.lang.RuntimeException: java.nio.file.NoSuchFileException: /home/sathvika/geotrellis-spark-hdfs/target/bg-jobs/sbt_641180ba/target/5dca9fb7/359669fc/hadoop-client-api-3.3.4.jar
    at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3089)
    at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:3036)
    at org.apache.hadoop.conf.Configuration.loadProps(Configuration.java:2914)
    at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2896)
    at org.apache.hadoop.conf.Configuration.get(Configuration.java:1246)
    at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1863)
    at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1840)
    at org.apache.hadoop.util.ShutdownHookManager.getShutdownTimeout(ShutdownHookManager.java:183)
    at org.apache.hadoop.util.ShutdownHookManager.shutdownExecutor(ShutdownHookManager.java:145)
    at org.apache.hadoop.util.ShutdownHookManager.access$300(ShutdownHookManager.java:65)
    at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:102)
Caused by: java.nio.file.NoSuchFileException: /home/sathvika/geotrellis-spark-hdfs/target/bg-jobs/sbt_641180ba/target/5dca9fb7/359669fc/hadoop-client-api-3.3.4.jar
    at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
    at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
    at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
    at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
    at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
    at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
    at java.base/java.nio.file.Files.readAttributes(Files.java:1764)
    at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1275)
    at java.base/java.util.zip.ZipFile$CleanableResource.<init>(ZipFile.java:741)
    at java.base/java.util.zip.ZipFile$CleanableResource$FinalizableResource.<init>(ZipFile.java:857)
    at java.base/java.util.zip.ZipFile$CleanableResource.get(ZipFile.java:846)
    at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:256)
    at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:185)
    at java.base/java.util.jar.JarFile.<init>(JarFile.java:348)
    at java.base/sun.net.www.protocol.jar.URLJarFile.<init>(URLJarFile.java:103)
    at java.base/sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:72)
    at java.base/sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:168)
    at java.base/sun.net.www.protocol.jar.JarFileFactory.getOrCreate(JarFileFactory.java:91)
    at java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:132)
    at java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:175)
    at org.apache.hadoop.conf.Configuration.parse(Configuration.java:3009)
    at org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:3105)
    at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3063)
    ... 10个更多的异常栈帧

内容的提问来源于stack exchange，提问作者Sathvika M