Solutions to Skip the Download #

1. Mark the Dependency as Intransitive & Optional

Add intransitive() and optional() to the Protegrity dependency. This tells SBT two things: don't bother resolving any dependencies of this library, and don't fail if it can't find the library (since you know it's present in your Spark environment):

name := "Protegrity UDF"
version := "1.0"
scalaVersion := "2.11.8"
libraryDependencies ++= Seq(
  // Add intransitive() and optional() here
  "com.protegrity.spark" % "udf" % "2.3.2" % "provided" intransitive() optional(),
  "org.apache.spark" %% "spark-core" % "2.3.2" % "provided",
  "org.apache.spark" %% "spark-sql" % "2.3.2" % "provided"
)

2. Use pomOnly() to Skip Full Resolution

If the first approach doesn't work, use pomOnly() to tell SBT you only care about the POM metadata (and since it's not available, combined with provided, SBT will skip strict resolution):

libraryDependencies ++= Seq(
  "com.protegrity.spark" % "udf" % "2.3.2" % "provided" pomOnly(),
  "org.apache.spark" %% "spark-core" % "2.3.2" % "provided",
  "org.apache.spark" %% "spark-sql" % "2.3.2" % "provided"
)

3. Point to a Local Repository (If You Have the JAR)

If you have the udf-2.3.2.jar saved locally, you can add a local directory as a resolver so SBT finds it without hitting public repos. For example, if it's in your local Maven repo:

name := "Protegrity UDF"
version := "1.0"
scalaVersion := "2.11.8"

// Add local Maven repo as a resolver
resolvers += "Local Maven Repo" at "file:///" + System.getProperty("user.home") + "/.m2/repository"
// Or point to a specific folder with the JAR:
// resolvers += "Local Protegrity JARs" at "file:///C:/path/to/your/protegrity/jars"

libraryDependencies ++= Seq(
  "com.protegrity.spark" % "udf" % "2.3.2" % "provided",
  "org.apache.spark" %% "spark-core" % "2.3.2" % "provided",
  "org.apache.spark" %% "spark-sql" % "2.3.2" % "provided"
)

Why This Works #

• provided tells SBT not to package the dependency into your final JAR, but it still tries to resolve it for compilation.
• intransitive() skips resolving any nested dependencies of the Protegrity library.
• optional() relaxes SBT's requirement to find the dependency—since you know it's present in your Spark cluster, this is safe.
• pomOnly() tells SBT to only look for the POM file (instead of the full JAR), which avoids the download attempt when combined with provided.

内容的提问来源于stack exchange，提问作者haritrichy