完成前置配置后spark-shell无法初始化，报常量池索引错误

问题场景 #

已下载并配置以下组件：

• spark-3.4.4-bin-hadoop3压缩包
• hadoop-3 winutils
• Java JDK 23.0.1

完成环境变量配置后，执行spark-shell命令时无法正常初始化，抛出bad constant pool index: 0 at pos: 49842错误。

错误日志 #

>>spark-shell
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
ReplGlobal.abort: bad constant pool index: 0 at pos: 49842
[init] error:
  bad constant pool index: 0 at pos: 49842
     while compiling: <no file>
        during phase: globalPhase=<no phase>, enteringPhase=<some phase>
     library version: version 2.12.17
    compiler version: version 2.12.17
  reconstructed args: -classpath  -Yrepl-class-based -Yrepl-outdir C:\TEMP\spark-8c7c7071-0844-45a1-92e7-e97996bb80b8\repl-c344ae8d-0919-4569-97d2-0cc91ad23a37

  last tree to typer: EmptyTree
       tree position: <unknown>
            tree tpe: <notype>
              symbol: null
           call site: <none> in <none>

== Source file context for tree position ==


Exception in thread "main" scala.reflect.internal.FatalError:
  bad constant pool index: 0 at pos: 49842
     while compiling: <no file>
        during phase: globalPhase=<no phase>, enteringPhase=<some phase>
     library version: version 2.12.17
    compiler version: version 2.12.17
  reconstructed args: -classpath  -Yrepl-class-based -Yrepl-outdir C:\TEMP\spark-8c7c7071-0844-45a1-92e7-e97996bb80b8\repl-c344ae8d-0919-4569-97d2-0cc91ad23a37

  last tree to typer: EmptyTree
       tree position: <unknown>
            tree tpe: <notype>
              symbol: null
           call site: <none> in <none>

== Source file context for tree position ==


        at scala.reflect.internal.Reporting.abort(Reporting.scala:69)
        at scala.reflect.internal.Reporting.abort$(Reporting.scala:65)
        at scala.tools.nsc.interpreter.IMain$$anon$2.scala$tools$nsc$interpreter$ReplGlobal$$super$abort(IMain.scala:239)
        at scala.tools.nsc.interpreter.ReplGlobal.abort(ReplGlobal.scala:31)
        at scala.tools.nsc.interpreter.ReplGlobal.abort$(ReplGlobal.scala:29)
        at scala.tools.nsc.interpreter.IMain$$anon$2.abort(IMain.scala:239)
        at scala.tools.nsc.symtab.classfile.ClassfileParser$ConstantPool.errorBadIndex(ClassfileParser.scala:385)
        at scala.tools.nsc.symtab.classfile.ClassfileParser$ConstantPool.getExternalName(ClassfileParser.scala:249)
        at scala.tools.nsc.symtab.classfile.ClassfileParser.readParamNames$1(ClassfileParser.scala:828)
        at scala.tools.nsc.symtab.classfile.ClassfileParser.parseAttribute$1(ClassfileParser.scala:834)
        at scala.tools.nsc.symtab.classfile.ClassfileParser.$anonfun$parseAttributes$7(ClassfileParser.scala:908)
        at scala.tools.nsc.symtab.classfile.ClassfileParser.parseAttributes(ClassfileParser.scala:908)
        at scala.tools.nsc.symtab.classfile.ClassfileParser.parseMethod(ClassfileParser.scala:611)
        at scala.tools.nsc.symtab.classfile.ClassfileParser.$anonfun$parseClass$4(ClassfileParser.scala:534)
        at scala.tools.nsc.symtab.classfile.ClassfileParser.parseClass(ClassfileParser.scala:534)
        at scala.tools.nsc.symtab.classfile.ClassfileParser.$anonfun$parse$2(ClassfileParser.scala:160)
        at scala.tools.nsc.symtab.classfile.ClassfileParser.$anonfun$parse$1(ClassfileParser.scala:146)
        at scala.tools.nsc.symtab.classfile.ClassfileParser.parse(ClassfileParser.scala:129)
        at scala.tools.nsc.symtab.SymbolLoaders$ClassfileLoader.doComplete(SymbolLoaders.scala:343)
        at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoaders.scala:250)
        at scala.reflect.internal.Symbols$Symbol.completeInfo(Symbols.scala:1542)
        at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
        at scala.reflect.internal.Definitions.scala$reflect$internal$Definitions$$enterNewMethod(Definitions.scala:49)
        at scala.reflect.internal.Definitions$DefinitionsClass.String_$plus$lzycompute(Definitions.scala:1134)
        at scala.reflect.internal.Definitions$DefinitionsClass.String_$plus(Definitions.scala:1134)
        at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreMethods$lzycompute(Definitions.scala:1438)
        at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreMethods(Definitions.scala:1420)
        at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode$lzycompute(Definitions.scala:1450)
        at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode(Definitions.scala:1450)
        at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1506)
        at scala.tools.nsc.Global$Run.<init>(Global.scala:1214)
        at scala.tools.nsc.interpreter.IMain._initialize(IMain.scala:124)
        at scala.tools.nsc.interpreter.IMain.initializeSynchronous(IMain.scala:146)
        at org.apache.spark.repl.SparkILoop.$anonfun$process$10(SparkILoop.scala:211)
        at org.apache.spark.repl.SparkILoop.withSuppressedSettings$1(SparkILoop.scala:189)
        at org.apache.spark.repl.SparkILoop.startup$1(SparkILoop.scala:201)
        at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:236)
        at org.apache.spark.repl.Main$.doMain(Main.scala:78)
        at org.apache.spark.repl.Main$.main(Main.scala:58)
        at org.apache.spark.repl.Main.main(Main.scala)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1020)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:192)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:215)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1111)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1120)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

解决方案 #

1. 降级JDK版本 #

Spark 3.4.4官方仅支持JDK 8、11、17，JDK 23属于最新预览版，存在兼容性问题。建议更换为JDK 17（LTS版本，稳定性最佳）：

• 卸载当前JDK 23
• 下载并安装OpenJDK 17或Oracle JDK 17
• 更新环境变量JAVA_HOME和PATH指向新的JDK目录

2. 验证Winutils配置 #

确保hadoop-3 winutils的路径已添加到HADOOP_HOME环境变量，且%HADOOP_HOME%\bin已加入PATH。执行winutils.exe ls C:\测试是否正常运行。

3. 清理临时文件 #

删除Spark生成的临时目录（日志中显示为C:\TEMP\spark-xxxxxx），避免缓存文件损坏导致的解析错误。

4. 重启终端 #

修改环境变量后，必须关闭并重新打开命令提示符/PowerShell，确保新的环境变量生效。

内容的提问来源于stack exchange，提问作者Nagendra Babu N