解决Scala Native测试阶段test.nativeLink的BuildException链接错误

错误输出 #

[build.mill-57/61] compile
[build.mill-57] [info] compiling 1 Scala source to /workspaces/test/saturday/mill/example/scalalib/native/2-interop/out/mill-build/compile.dest/classes ...
[build.mill-57] [info] done compiling
[134/147] compile
[134] [info] compiling 1 Scala source to /workspaces/test/saturday/mill/example/scalalib/native/2-interop/out/compile.dest/classes ...
[134] [info] done compiling
[141/147] test.compile
[141] [info] compiling 1 Scala source to /workspaces/test/saturday/mill/example/scalalib/native/2-interop/out/test/compile.dest/classes ...
[141] [info] done compiling
[build.mill-57] [info] Linking (multithreadingEnabled=true, disable if not used) (3274 ms)
[build.mill-57] [info] Discovered 1944 classes and 14162 methods after classloading
[build.mill-57] [info] Checking intermediate code (quick) (175 ms)
[build.mill-57] [info] Discovered 1904 classes and 10865 methods after optimization
[build.mill-57] [info] Optimizing (debug mode) (7396 ms)
[build.mill-57] [info] Produced 3 LLVM IR files
[build.mill-57] [info] Generating intermediate code (8943 ms)
[build.mill-57] [info] Compiling to native code (9752 ms)
[build.mill-57] [info] Linking with [pthread, dl, HelloWorld]
[build.mill-57] [error] /usr/bin/ld: cannot find -lHelloWorld: No such file or directory
[build.mill-57] [error] clang: error: linker command failed with exit code 1 (use -v to see invocation)
[build.mill-57] [info] Total (27675 ms)
[147/147] ==================================================== test ====================================================== 51s
1 tasks failed
test.nativeLink scala.scalanative.build.BuildException: Failed to link /workspaces/test/saturday/mill/example/scalalib/native/2-interop/out/test/nativeWorkdir.dest/native/scala.scalanative.testinterface.TestMain

项目结构 #

build.mill
src/foo/
  HelloWorld.scala
  native-src/
HelloWorld.c
test/
  src/
    foo/
      HelloWorldTests.scala

代码文件内容 #

HelloWorld.scala #

package foo
import scala.scalanative.libc._
import scala.scalanative.unsafe._

object Main {
  def main(args: Array[String]): Unit = {
    println("Running HelloWorld function")
    stdio.printf(c"Reversed: %s\n", HelloWorld.reverseString(c"Hello, World!"))
    println("Done...")
  }
}

// Define the external module, the C library containing our function "reverseString"
@extern
@link("HelloWorld")
// Arbitrary object name
object HelloWorld {
  // Name and signature of C function
  def reverseString(str: CString): CString = extern
}

HelloWorld.c #

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

char* reverseString(const char *str) {
  int length = strlen(str);
  char *reversed = (char*) malloc((length + 1) * sizeof(char));

  for (int i = 0; i < length; i++) {
    reversed[i] = str[length - i - 1];
  }
  reversed[length] = '\0'; // Null-terminate the string

  return reversed;
}

HelloWorldTests.scala #

package foo

import utest._
import scala.scalanative.unsafe._

object HelloWorldTest extends TestSuite {
  val tests = Tests {
    test("reverseString should reverse a C string correctly") {
      val expected = "!dlrow olleH"

      val result = HelloWorld.reverseString(c"Hello World!")

      // Check if the reversed string matches the expected result
      assert(fromCString(result) == expected)
      fromCString(result)
    }
  }
}

原始build.mill #

package build
import mill._, scalalib._, scalanativelib._

object `package` extends RootModule with ScalaNativeModule {
  def scalaVersion = "2.13.11"
  def scalaNativeVersion = "0.5.5"

  object test extends ScalaNativeTests {
    def ivyDeps = Agg(ivy"com.lihaoyi::utest::0.8.4")
    def testFramework = "utest.runner.Framework"
  }

  def nativeLinkingOptions = Seq("-L" + millSourcePath.toString + "/target")

  // Compiled C
  def nativeCompiled = T {
    os.makeDir.all(millSourcePath / "target")

    os.proc("gcc", "-m64", "-shared",
      "-c", millSourcePath.toString + "/native-src/HelloWorld.c",
      "-o", millSourcePath.toString + "/target/libHelloWorld.so"
    ).call(stdout = os.Inherit)

    PathRef(T.dest / "target/libHelloWorld.so")
  }
}

问题说明 #

使用Mill构建工具，mill run可正常编译运行，但执行mill test时出现上述链接错误。已尝试修改测试框架版本、在test模块中配置nativeLinkingOptions，问题仍未解决。

解决方案 #

核心问题 #

1. 测试模块未继承主模块的native配置，也未依赖编译好的C库，导致链接阶段找不到libHelloWorld.so
2. build.mill中C文件路径错误：HelloWorld.c在根目录，但配置里写的是native-src/HelloWorld.c，导致编译出的库无效或不存在
3. 编译共享库缺少-fPIC参数，不符合共享库编译要求

修改后的build.mill #

package build
import mill._, scalalib._, scalanativelib._

object `package` extends RootModule with ScalaNativeModule {
  def scalaVersion = "2.13.11"
  def scalaNativeVersion = "0.5.5"

  object test extends ScalaNativeTests {
    def ivyDeps = Agg(ivy"com.lihaoyi::utest::0.8.4")
    def testFramework = "utest.runner.Framework"
    
    // 让测试模块依赖主模块的nativeCompiled任务，确保C库先编译完成
    def nativeDeps = Agg(super.nativeDeps, `package`.nativeCompiled)
    // 测试模块同步主模块的库搜索路径
    def nativeLinkingOptions = `package`.nativeLinkingOptions
  }

  def nativeLinkingOptions = Seq("-L" + millSourcePath.toString + "/target")

  // 修正C文件路径，添加-fPIC参数编译共享库
  def nativeCompiled = T {
    os.makeDir.all(millSourcePath / "target")

    os.proc("gcc", "-m64", "-shared", "-fPIC",
      millSourcePath / "HelloWorld.c",
      "-o", millSourcePath / "target/libHelloWorld.so"
    ).call(stdout = os.Inherit)

    PathRef(millSourcePath / "target/libHelloWorld.so")
  }
}

验证步骤 #

1. 清理之前的编译产物：mill clean
2. 重新执行测试：mill test

内容的提问来源于stack exchange，提问作者Jeffrey Whewhetu