MacOS M2芯片Ventura系统编译Hadoop源码报错求助

问题根源 #

M2芯片为arm64架构，当前Hadoop编译流程默认以x86_64架构执行，导致链接器拒绝使用arm64版本的libjvm.dylib，最终出现JNI符号未定义的链接失败错误。错误信息如下：

ld: warning: ignoring file '/Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home/jre/lib/server/libjvm.dylib': found architecture 'arm64', required architecture 'x86_64'
[WARNING] ld: warning: ignoring file '/Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home/jre/lib/server/libjvm.dylib': found architecture 'arm64', required architecture 'x86_64'
[WARNING] ld: warning: ignoring file '/Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home/jre/lib/server/libjvm.dylib': found architecture 'arm64', required architecture 'x86_64'
[WARNING] ld: Undefined symbols:
[WARNING] _JNI_CreateJavaVM, referenced from:
[WARNING] _getGlobalJNIEnv in libhdfs.a3
[WARNING] _JNI_GetCreatedJavaVMs, referenced from:
[WARNING] _getGlobalJNIEnv in libhdfs.a3
[WARNING] ld: Undefined symbols:
[WARNING] _JNI_CreateJavaVM, referenced from:
[WARNING] _getGlobalJNIEnv in libhdfs.a3
[WARNING] _JNI_GetCreatedJavaVMs, referenced from:
[WARNING] _getGlobalJNIEnv in libhdfs.a3
[WARNING] clang: error: linker command failed with exit code 1 (use -v to see invocation)

解决方案 #

方案1：通过Rosetta 2转译以x86_64架构编译 #

• 打开终端应用，右键选择「显示简介」，勾选「使用Rosetta打开」后重启终端。
• 执行arch命令验证，输出应为i386（代表x86_64转译模式）。
• 安装x86_64版本的Zulu 8 JDK，设置环境变量指向该JDK：

  export JAVA_HOME=/Library/Java/JavaVirtualMachines/zulu-8-x86_64.jdk/Contents/Home
  export PATH=$JAVA_HOME/bin:$PATH
  

• 重新执行Hadoop编译命令：

  mvn clean install -DskipTests
  

方案2：强制以arm64架构编译Hadoop #

• 在原生arm64终端中，执行编译命令时添加架构指定参数：

  mvn clean install -DskipTests -Darch=arm64 -Dplatform.name=darwin
  

• 若参数不生效，修改Hadoop原生编译配置：进入hadoop-common-project/hadoop-common/src/main/native/Makefile，将所有x86_64替换为arm64，保存后重新编译。

方案3：安装通用架构JDK #

• 下载支持双架构（arm64+x86_64）的Zulu 8 JDK，安装后设置JAVA_HOME指向该JDK。
• 编译时通过maven参数指定目标架构，确保链接器匹配对应版本的libjvm.dylib：

  mvn clean install -DskipTests -Dmaven.compiler.target=1.8 -Dmaven.compiler.source=1.8 -Darch=arm64
  

内容的提问来源于stack exchange，提问作者mark