首次初始化Hive时hive-site.xml非法字符解析错误求助

问题 #

使用Hadoop 3.1.1、Hive 3.1.1，以MySQL作为Hive元数据库，首次执行schematool -initSchema -dbType mysql命令初始化时，触发hive-site.xml文件的非法字符实体（代码0x8）解析错误，附上错误堆栈信息及hive-site.xml关键配置片段，请求排查问题原因。

错误堆栈信息 #

adminn@master:~$ schematool -initSchema -dbType mysql

Exception in thread "main" java.lang.RuntimeException: com.ctc.wstx.exc.WstxParsingException: Illegal character entity: expansion character (code 0x8
 at [row,col,system-id]: [3210,96,"file:/home/adminn/apache-hive-3.1.1-bin/conf/hive-site.xml"]
    at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3003)
    at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2931)
    at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2806)
    at org.apache.hadoop.conf.Configuration.get(Configuration.java:1460)
    at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:4990)
    at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5063)
    at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5150)
    at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:5098)
    at org.apache.hive.beeline.HiveSchemaTool.<init>(HiveSchemaTool.java:96)
    at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1473)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
Caused by: com.ctc.wstx.exc.WstxParsingException: Illegal character entity: expansion character (code 0x8
 at [row,col,system-id]: [3210,96,"file:/home/adminn/apache-hive-3.1.1-bin/conf/hive-site.xml"]
    at com.ctc.wstx.sr.StreamScanner.constructWfcException(StreamScanner.java:621)
    at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:491)
    at com.ctc.wstx.sr.StreamScanner.reportIllegalChar(StreamScanner.java:2456)
    at com.ctc.wstx.sr.StreamScanner.validateChar(StreamScanner.java:2403)
    at com.ctc.wstx.sr.StreamScanner.resolveCharEnt(StreamScanner.java:2369)
    at com.ctc.wstx.sr.StreamScanner.fullyResolveEntity(StreamScanner.java:1515)
    at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2828)
    at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1123)
    at org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3257)
    at org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3063)
    at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2986)
    ... 15 more

hive-site.xml关键配置片段 #

<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>Hivehadoop@123$</value>
    <description>password to use against metastore database</description>
</property>

<property>
    <name>hive.metastore.ds.connection.url.hook</name>
    <value/>
    <description>Name of the hook to use for retrieving the JDO connection URL. If empty, the value in javax.jdo.option.ConnectionURL is used</description>
</property>

<property>
    <name>javax.jdo.option.Multithreaded</name>
    <value>true</value>
    <description>Set this to true if multiple threads access metastore through JDO concurrently.</description>
</property>

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost/metastore_db?createDatabaseIfNotExist=true</value>
    <description>
      JDBC connect string for a JDBC metastore.
      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
    </description>
</property>

<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hiveuser</value>
    <description>Username to use against metastore database</description>
</property>

<property>
    <name>hive.exec.local.scratchdir</name>
    <value>/tmp/${user.name}</value>
    <description>Local scratch space for Hive jobs</description>
</property>

<property>
    <name>hive.downloaded.resources.dir</name>
    <value>/tmp/${user.name}_resources</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
</property>

问题排查与解决 #

• 根因分析：错误日志明确指向hive-site.xml的第3210行第96列存在0x8字符（ASCII退格符），这是XML规范禁止的控制字符，解析器无法正常识别。这类字符通常是复制粘贴配置时从外部文档、网页带入的不可见隐藏字符。
• 排查方法：
  • 命令行定位：在Linux环境执行cat -A /home/adminn/apache-hive-3.1.1-bin/conf/hive-site.xml | sed -n '3200,3220p'，该命令会显示所有不可见字符，0x8会以^H形式呈现。
  • 编辑器排查：用Vim、Notepad++等支持显示控制字符的编辑器打开文件，开启显示所有字符的功能，直接定位到3210行附近查找异常字符。
• 解决步骤：
  1. 找到该位置的0x8字符并删除；
  2. 保存修改后的hive-site.xml；
  3. 重新执行schematool -initSchema -dbType mysql命令验证问题是否解决。

内容的提问来源于stack exchange，提问作者Salva