如何为已有Nix Derivation添加Clang Sanitizers编译支持

问题背景 #

我正在编写一个函数，输入是可通过Clang编译的任意Nix Derivation，输出是添加了指定-fsanitize=编译标志的修改版Derivation。该函数在libtasn1等简单场景下能正常工作，但在openssl这类复杂场景下，会因链接器无法找到Clang运行时共享库符号而失败——链接器似乎只会查找GCC的sanitizer符号（和Clang的不兼容）。

我必须用Clang，因为它有GCC没有的sanitizer（比如fuzzer sanitizer）。推测问题出在ld无法识别LLVM资源目录中的libclang_rt.asan-x86_64.so这类库的符号，试过用lld但NixOS对它的支持不好。

注意：我不是要问如何给openssl单独添加sanitizer编译支持，而是要一个适用于任意Derivation的通用方案，且方案要和底层构建系统无关（类似Nix通过包装编译器注入加固标志的方式）。

链接错误示例 #

/nix/store/3r87a2wq1w4l66wnsm7rqvy608mx23h6-binutils-2.40/bin/ld: /build/openssl-3.0.11/test/p_test.c:181: undefined reference to `__asan_report_store4'
/nix/store/3r87a2wq1w4l66wnsm7rqvy608mx23h6-binutils-2.40/bin/ld: /build/openssl-3.0.11/test/p_test.c:83: undefined reference to `__asan_report_load8'
/nix/store/3r87a2wq1w4l66wnsm7rqvy608mx23h6-binutils-2.40/bin/ld: test/p_test-dso-p_test.o: in function `p_teardown':
/build/openssl-3.0.11/test/p_test.c:311: undefined reference to `__sancov_lowest_stack'
/nix/store/3r87a2wq1w4l66wnsm7rqvy608mx23h6-binutils-2.40/bin/ld: /build/openssl-3.0.11/test/p_test.c:317: undefined reference to `__asan_report_load8'
/nix/store/3r87a2wq1w4l66wnsm7rqvy608mx23h6-binutils-2.40/bin/ld: /build/openssl-3.0.11/test/p_test.c:318: undefined reference to `__asan_report_load8'
/nix/store/3r87a2wq1w4l66wnsm7rqvy608mx23h6-binutils-2.40/bin/ld: test/p_test-dso-p_test.o: in function `sancov.module_ctor_8bit_counters':
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)

当前使用的代码 #

# test with
# nix-build -E 'with import <nixpkgs> {}; callPackage ./test_instrument_derivation.nix {}'

{ nixpkgs ? import <nixpkgs> {} }:  let 

# tried adding overlay here to replace stdenv with clangStdenv
# but it often got stuck compiling huge amounts of stuff and breaking 
# before even getting to the package I want to build 
pkgs = import <nixpkgs> {};

# make a clangStdenv derivation out of another derivation and
# add some extra configuration. use clang because not all of the functionality of the clang
# sanitizers is present in gcc, for example, -fsanitize=fuzzer
instrumentLibraryDerivation = orig : (pkgs.clangStdenv.mkDerivation orig.drvAttrs).overrideAttrs (oldAttrs: {

      preBuild = ''

        ${oldAttrs.preBuild or ""}

        # setting the clang sanitizer compile flags
        # doing this after the configure stage because the sanitizers often
        # confuse configure scripts when they try to autodetect compiler features

        # sigaltstack=0 seems to be required on nixos or else asan/ubsan binaries don't work
        # suppress asan detections that might trigger if binaries are run during the build/test phases
        
        export ASAN_OPTIONS=use_sigaltstack=0,detect_leaks=0
        export UBSAN_OPTIONS=use_sigaltstack=false

        # these are sufficient to compile a simple hello world binary with the right instrumentation
        export NIX_CFLAGS_COMPILE="  $NIX_CFLAGS_COMPILE -fsanitize=fuzzer-no-link,address"

        # tried things like:
        #   -fuse-ld=lld
        #   export LD=lld
        #   -lasan
        #   -Wl,-lasan
        #   -L$(clang -print-file-name="libclang_rt.asan-x86_64.so")
        #   -Wl,$(clang -print-file-name="libclang_rt.asan-x86_64.so")
      '';
      
      # trying to avoid including anything with GCC here
      depsBuildBuild =  [ pkgs.buildPackages.clangStdenv pkgs.buildPackages.clangStdenv.cc ]; 

      nativeBuildInputs = (oldAttrs.nativeBuildInputs or []) ++  [
        # trying to make the clang runtime libraries available
        # not working :(
        pkgs.llvmPackages.clang
        pkgs.llvmPackages.compiler-rt
      ];

      # ASAN sometimes changes the behavior of unit tests and breaks them, so skip check
      doCheck = false;

      outputs = [ "out" ];
    }
  );
in

# despite working with simpler derivations, this instrument derivation function causes linker errors
# in things like openSSL. this seems to be because it uses bintools ld to link, which is not aware of
# the clang runtime libraries. this is sometimes easier to see with -fsanitize=fuzzer-no-link, because
# that doesn't exist in gcc. sometimes it can find asan symbols, but they're incompatible since they're
# for GCC asan instead of clang asan.

instrumentLibraryDerivation pkgs.openssl

# this works fine with simple things like libtasn1
#
# instrumentLibraryDerivation  pkgs.libtasn1
#
# $ nm result/lib/libtasn1.so | grep sancov
#
# 0000000000034010 d __sancov_gen_cov_switch_values
# 0000000000034530 d __sancov_gen_cov_switch_values
# 00000000000346e0 d __sancov_gen_cov_switch_values
# 0000000000034880 d __sancov_gen_cov_switch_values
# 0000000000034b00 d __sancov_gen_cov_switch_values
#
# ...
#

通用解决方案 #

核心问题定位 #

链接器找不到Clang sanitizer符号的根本原因：

• 即便用了clangStdenv，复杂构建系统（如openssl的Makefile）仍可能显式调用ld而非Clang驱动，导致无法自动链接Clang runtime库
• 手动加-lasan等参数会优先匹配GCC的sanitizer库，引发符号不兼容

修复方案与修改后代码 #

# test with
# nix-build -E 'with import <nixpkgs> {}; callPackage ./test_instrument_derivation.nix {}'

{ nixpkgs ? import <nixpkgs> {} }:  let 
pkgs = import <nixpkgs> {};

instrumentLibraryDerivation = orig: orig.overrideStdenv (_: pkgs.clangStdenv).overrideAttrs (oldAttrs: {
  preBuild = ''
    ${oldAttrs.preBuild or ""}

    # 强制所有编译、链接操作通过Clang驱动执行
    export CC=clang
    export CXX=clang++
    export LD="$CC"

    # 设置sanitizer运行时参数
    export ASAN_OPTIONS=use_sigaltstack=0,detect_leaks=0
    export UBSAN_OPTIONS=use_sigaltstack=false

    # 同时在编译和链接阶段注入sanitizer标志
    export SANITIZERS="fuzzer-no-link,address"
    export NIX_CFLAGS_COMPILE="$NIX_CFLAGS_COMPILE -fsanitize=$SANITIZERS"
    export NIX_LDFLAGS="$NIX_LDFLAGS -fsanitize=$SANITIZERS"

    # 显式指定Clang Runtime库路径
    export CLANG_RT_DIR=$(clang -print-runtime-dir)
    export NIX_LDFLAGS="$NIX_LDFLAGS -L$CLANG_RT_DIR"
  '';

  nativeBuildInputs = (oldAttrs.nativeBuildInputs or []) ++ [
    pkgs.llvmPackages.clang
    pkgs.llvmPackages.compiler-rt
  ];

  doCheck = false;
  outputs = [ "out" ];
});
in

instrumentLibraryDerivation pkgs.openssl

关键说明 #

1. 强制用Clang驱动链接：Clang会自动处理sanitizer runtime库的链接逻辑，无需手动指定-lasan等参数，彻底避免和GCC库的冲突
2. 同步设置编译与链接标志：确保所有编译单元和最终链接步骤都应用sanitizer配置
3. 使用overrideStdenv替换编译环境：保留原Derivation的所有属性，仅替换编译环境，避免重新编译无关依赖，解决之前全局替换stdenv编译慢的问题

内容的提问来源于stack exchange，提问作者ggg