如何在ivy-settings.xml配置GitLab Maven仓库解决Spark依赖解析失败？

当前ivy-settings.xml配置如下：

<ivysettings>

  <settings defaultResolver="default"/>

  <include url="${ivy.default.settings.dir}/ivysettings-public.xml"/>
  <include url="${ivy.default.settings.dir}/ivysettings-shared.xml"/>
  <include url="${ivy.default.settings.dir}/ivysettings-local.xml"/>
  <include url="${ivy.default.settings.dir}/ivysettings-main-chain.xml"/>
  <include url="${ivy.default.settings.dir}/ivysettings-default-chain.xml"/>

  <credentials host="my.company.com" realm="gitlab-maven" username="Private-Token" passwd="XXXXXXX..."/>

  <resolvers>
      <ibiblio name="repo0"
               m2compatible="true"
               useMavenMetadata="true"
               usepoms="true"
               root="https://my.company.com/api/v4/projects/1337/packages/maven"
               pattern="[organisation]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]"/>
    <chain name="default" returnFirst="true" checkmodified="true">
      <resolver ref="local"/>
      <resolver ref="shared"/>
      <resolver ref="public"/>
      <resolver ref="repo0"/>
    </chain>
  </resolvers>
</ivysettings>

执行以下spark-shell命令：

spark-shell --conf spark.jars.ivySettings=file:///opt/spark/conf/ivy-settings.xml\
            --conf spark.jars.packages=com.company.my:some-artifact:1.2.3

得到报错：

:: loading settings :: file = /opt/spark/conf/ivy-settings.xml
:: loading settings :: url = jar:file:/opt/spark/jars/ivy-2.5.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
Ivy Default Cache set to: /home/spark/.ivy2/cache
The jars for the packages stored in: /home/spark/.ivy2/jars
com.company.my#some-artifact added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-3bb84be4-4bc7-41c7-8a74-3d16ffaa584b;1.0
        confs: [default]
:: resolution report :: resolve 520ms :: artifacts dl 0ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   1   |   0   |   0   |   0   ||   0   |   0   |
        ---------------------------------------------------------------------

:: problems summary ::
:::: WARNINGS
                module not found: com.company.my#some-artifact;1.2.3

        ==== local: tried

          /home/spark/.ivy2/local/com.company.my/some-artifact/1.2.3/ivys/ivy.xml

          -- artifact com.company.my#some-artifact;1.2.3!some-artifact.jar:

          /home/spark/.ivy2/local/com.company.my/some-artifact/1.2.3/jars/some-artifact.jar

        ==== shared: tried

          /home/spark/.ivy2/shared/com.company.my/some-artifact/1.2.3/ivys/ivy.xml

          -- artifact com.company.my#some-artifact;1.2.3!some-artifact.jar:

          /home/spark/.ivy2/shared/com.company.my/some-artifact/1.2.3/jars/some-artifact.jar

        ==== public: tried

          https://repo1.maven.org/maven2/de/fraunhofer/iwes/some-artifact/1.2.3/some-artifact-1.2.3.pom

          -- artifact com.company.my#some-artifact;1.2.3!some-artifact.jar:

          https://repo1.maven.org/maven2/de/fraunhofer/iwes/some-artifact/1.2.3/some-artifact-1.2.3.jar

        ==== repo0: tried

          https://my.company.com/api/v4/projects/1337/packages/maven/de/fraunhofer/iwes/some-artifact/1.2.3/some-artifact-1.2.3.pom

          -- artifact com.company.my#some-artifact;1.2.3!some-artifact.jar:

          https://my.company.com/api/v4/projects/1337/packages/maven/de/fraunhofer/iwes/some-artifact/1.2.3/some-artifact-1.2.3.jar

                ::::::::::::::::::::::::::::::::::::::::::::::

                ::          UNRESOLVED DEPENDENCIES         ::

                ::::::::::::::::::::::::::::::::::::::::::::::

                :: com.company.my#some-artifact;1.2.3: not found

                ::::::::::::::::::::::::::::::::::::::::::::::

发现浏览器能正常打开报错中打印的URL并下载pom和jar包，但使用curl命令：

curl -vvv https://my.company.com/api/v4/projects/1337/packages/maven/de/fraunhofer/iwes/some-artifact/1.2.3/some-artifact-1.2.3.pom

或直接访问带认证的URL：

https://Private-Token:XXXXX...@my.company.com/api/v4/projects/1337/packages/maven/de/fraunhofer/iwes/some-artifact/1.2.3/some-artifact-1.2.3.pom

会返回404错误。请问忽略了什么？

• GitLab认证方式错误
  GitLab Maven仓库要求通过Private-Token请求头传递认证信息，而非HTTP Basic Auth。当前的<credentials>配置是按Basic Auth设计的，Ivy会将username=Private-Token和密码转成Basic Auth凭证，这不符合GitLab的认证逻辑。
  修改方式：移除<credentials>，在ibiblio resolver中添加请求头：

<property name="gitlab.token" value="XXXXXXX..."/>
<resolvers>
  <ibiblio name="repo0"
           m2compatible="true"
           useMavenMetadata="true"
           usepoms="true"
           root="https://my.company.com/api/v4/projects/1337/packages/maven"
           pattern="[organisation]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]">
    <extraHeaders>
      <extraHeader name="Private-Token" value="${gitlab.token}"/>
    </extraHeaders>
  </ibiblio>
</resolvers>

• 组织名（groupId）路径转换异常
  从报错可见，Ivy将指定的com.company.my转换成了de/fraunhofer/iwes，原因可能是：
  1. 指定的依赖坐标com.company.my:some-artifact:1.2.3与GitLab中实际包的groupId不符，需核对包的真实Maven坐标；
  2. Ivy启用了默认的组织名转换规则，需在ibiblio resolver中添加organisationPattern="[organisation]"以禁用自动转换：

<ibiblio name="repo0"
         ...
         organisationPattern="[organisation]"/>

• 浏览器访问依赖会话认证
  浏览器能正常下载是因为已通过公司SSO或其他方式登录GitLab，会话中携带了认证信息；而curl和Ivy没有该会话，必须通过正确的请求头传递Token才能访问。

内容的提问来源于stack exchange，提问作者rabejens