Scala Frameless库报错No implicits found for parameters i0: TypedColumn.Exists求助

Let's break down what's causing this error and how to fix it quickly.

Why You're Seeing This Error #

The TypedColumn.Exists implicit Frameless is complaining about is a type-level proof that confirms 'dept is a valid field on your Emp case class, and that Frameless can work with its type (String in this case).

This proof relies on Frameless's TypedEncoder for your Emp class, which wasn't being properly generated because the necessary implicits weren't in scope when you created the TypedDataset.

Step-by-Step Fix #

Here's how to adjust your code to resolve the issue:

1. Import Frameless's TypedEncoder implicits before creating the TypedDataset. This lets Frameless generate the type-level metadata it needs for your Emp case class.
2. Double-check the order of imports (Frameless's syntax should be available when accessing columns, but the TypedEncoder needs to be in scope earlier).

Here's your modified code:

import org.apache.spark.sql.SparkSession
import frameless.{TypedDataset, TypedEncoder} // Add TypedEncoder import

object TestSpark {
  def main(args: Array[String]): Unit = {
    val spark = SparkSession.builder()
      .master("local[*]")
      .appName("Spark Test")
      .getOrCreate

    import spark.implicits._
    import TypedEncoder._ // Critical: Import this to generate Emp's TypedEncoder

    val empDS = spark.read
      .option("header", true)
      .option("delimiter", ",")
      .csv("emp.csv")
      .as[Emp]

    val empTyDS = TypedDataset.create(empDS)
    import frameless.syntax._ // Enables Frameless's column access syntax

    empTyDS.show(10, false).run
    val deptCol = empTyDS('dept) // This should now work without errors
  }
}

case class Emp(name: String, dept: String, manager: String, salary: String)

Additional Checks #

• Verify Dependencies: Make sure your build file (e.g., build.sbt) has the correct Frameless dependency for Spark 2.4:

  libraryDependencies += "org.typelevel" %% "frameless-dataset" % "0.8.0"
  

• Field Names Match: Ensure the 'dept symbol exactly matches the case class field name (case-sensitive! Frameless relies on exact name matching for type safety).

内容的提问来源于stack exchange，提问作者floating_hammer