基于S2I的.NET Core自定义Builder镜像OpenShift运行时C++库依赖缺失

问题背景 #

我用S2I构建OpenShift Linux容器的.NET Core自定义Builder镜像，已添加复制dll和.so文件的步骤，但运行容器时出现共享库加载失败的错误。

错误信息 #

初始错误 #

"unable to load shared library 'CustomCppWrapper' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libWrapperName: cannot open shared object file: No such file or directory"

设置LD_DEBUG后的错误输出 #

/lib64/libstdc++.so.6: error: version lookup error: version `CXXABI_1.3.8' not found (required by /opt/app-root/app/libCWrappeNamer.so) (fatal)
/lib64/libstdc++.so.6: error: version lookup error: version `CXXABI_1.3.8' not found (required by ./libCWrappeNamer.so) (fatal)

执行ldd libCWrappeNamer.so的结果 #

./libCWrappeNamer.so: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by ./libCWrappeNamer.so)
./libCWrappeNamer.so: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /ab/sdk/customlib/gcc540/lib/libabc.so)
./libCWrappeNamer.so: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /ab/sdk/customlib/gcc540/lib/libxmlc.so)

自定义Dockerfile内容 #

FROM dotnet/dotnet-31-runtime-rhel7
# This image provides a .NET Core 3.1 environment you can use to run your .NET
# applications.

ENV PATH=/opt/app-root/src/.local/bin:/opt/app-root/src/bin:/opt/app-root/node_modules/.bin:${PATH} \
    STI_SCRIPTS_PATH=/usr/libexec/s2i

LABEL io.k8s.description="Platform for building and running .NET Core 3.1 applications" \
      io.openshift.tags="builder,.net,dotnet,dotnetcore,rh-dotnet31"

# Labels consumed by Red Hat build service
LABEL name="dotnet/dotnet-31-rhel7" \
      com.redhat.component="rh-dotnet31-container" \
      version="3.1" \
      release="1" \
      architecture="x86_64"

#-------------------------- COPY CPP LIBS

COPY CustomCppWrapper.lib /opt/app-root/app
COPY libCWrappeNamer.so /opt/app-root/app


#----------------------------------


# Labels consumed by Eclipse JBoss OpenShift plugin
LABEL com.redhat.dev-mode="DEV_MODE:false" \
      com.redhat.deployments-dir="/opt/app-root/src"

# Switch to root for package installs
USER 0

# Copy the S2I scripts from the specific language image to $STI_SCRIPTS_PATH.
COPY ./s2i/bin/ /usr/libexec/s2i

RUN INSTALL_PKGS="rh-nodejs10-npm rh-nodejs10-nodejs-nodemon rh-dotnet31-dotnet-sdk-3.1 rsync" && \
    yum install -y --setopt=tsflags=nodocs --disablerepo=\* \
      --enablerepo=rhel-7-server-rpms,rhel-server-rhscl-7-rpms,rhel-7-server-dotnet-rpms \
      $INSTALL_PKGS && \
    rpm -V $INSTALL_PKGS && \
    yum clean all -y && \
# yum cache files may still exist (and quite large in size)
    rm -rf /var/cache/yum/*

# Directory with the sources is set as the working directory.
RUN mkdir /opt/app-root/src
WORKDIR /opt/app-root/src

# Trigger first time actions.
RUN scl enable rh-dotnet31 'dotnet help'

# Build the container tool.
RUN /usr/libexec/s2i/container-tool build-tool

# Since $HOME is set to /opt/app-root, the yum install may have created config
# directories (such as ~/.pki/nssdb) there. These will be owned by root and can
# cause actions that work on all of /opt/app-root to fail. So we need to fix
# the permissions on those too.
RUN chown -R 1001:0 /opt/app-root && fix-permissions /opt/app-root

ENV ENABLED_COLLECTIONS="$ENABLED_COLLECTIONS rh-nodejs10" \
# Needed for the `dotnet watch` to detect changes in a container.
    DOTNET_USE_POLLING_FILE_WATCHER=true

# Run container by default as user with id 1001 (default)
USER 1001

# Set the default CMD to print the usage of the language image.
CMD /usr/libexec/s2i/usage

解决方案 #

核心原因 #

RHEL7默认的libstdc++.so.6版本偏低，缺少CXXABI_1.3.8和GLIBCXX_3.4.20符号，而你的C共享库是用GCC 5.4.0编译的，需要更高版本的libstdc支持；同时部分依赖库未复制到镜像中，路径配置也有问题。

修复步骤 #

1. 安装高版本GCC工具集
   修改Dockerfile中的安装命令，加入devtoolset-4-gcc-c++（对应GCC 5.4.0版本）：

   RUN INSTALL_PKGS="rh-nodejs10-npm rh-nodejs10-nodejs-nodemon rh-dotnet31-dotnet-sdk-3.1 rsync devtoolset-4-gcc-c++" && \
       yum install -y --setopt=tsflags=nodocs --disablerepo=\* \
         --enablerepo=rhel-7-server-rpms,rhel-server-rhscl-7-rpms,rhel-7-server-dotnet-rpms \
         $INSTALL_PKGS && \
       rpm -V $INSTALL_PKGS && \
       yum clean all -y && \
       rm -rf /var/cache/yum/*
   

2. 配置环境变量加载高版本libstdc++
   两种可选方式：

   • 方式一：用scl enable包裹启动命令，确保加载devtoolset的环境：

     CMD scl enable devtoolset-4 rh-nodejs10 rh-dotnet31 '/usr/libexec/s2i/usage'
     

   • 方式二：直接将devtoolset的lib路径加入LD_LIBRARY_PATH：

     ENV LD_LIBRARY_PATH=/opt/rh/devtoolset-4/root/usr/lib64:$LD_LIBRARY_PATH
     

3. 复制所有依赖库并配置路径
   把/ab/sdk/customlib/gcc540/lib/下的libabc.so、libxmlc.so等依赖库复制到镜像中，并添加到库搜索路径：

   # 复制所有C++依赖库
   COPY ./customlib/gcc540/lib/ /opt/app-root/app/customlib/
   # 添加库路径到环境变量
   ENV LD_LIBRARY_PATH=/opt/app-root/app:/opt/app-root/app/customlib:$LD_LIBRARY_PATH
   

4. 验证修复效果
   构建镜像后，进入容器执行ldd libCWrappeNamer.so，确认版本错误消失，所有依赖库都能正常找到。

内容的提问来源于stack exchange，提问作者user804401