咨询:基于含嵌套样例类的样例类创建DataFrame的实现方法
从嵌套样例类创建Spark DataFrame的测试实现
最近我在做Spark相关开发时,需要实现从包含嵌套样例类的样例类创建DataFrame的功能,目前已经写了一段Scala测试代码,内容如下:
import com.holdenkarau.spark.testing.DatasetSuiteBase import org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema import org.apache.spark.sql.{Encoders, Row} import org.scalatest.FlatSpec case class Inner(i: Int) case class Outer(in: Inner) class MyTest extends FlatSpec with DatasetSuiteBase { behavior of "Engine" it should "work" in { import spark.implicits._ val input = Seq("alice", "bob").toDF("name") val schema = Encoders.product[Outer].schema implicit val... } }
内容的提问来源于stack exchange,提问作者pcejrowski
相关产品推荐
相关产品推荐

