我正在嘗試實施
使用 spark 2.4.8 和 sbt 版本 1.4.3 使用 intellij
代碼：
val sqlContext = new org.apache.spark.sql.SQLContext(sc) 匯入 sqlContext.implicits._
case class Person(id:Int,Name:String,cityId:Long)
case class City(id:Long,Name:String)
val family=Seq(Person(1,"john",11),(2,"MAR",12),(3,"Iweta",10)).toDF
val cities=Seq(City(11,"boston"),(12,"dallas")).toDF
error:
Exception in thread "main" java.lang.NoClassDefFoundError: no Java class corresponding to Product with Serializable found
at scala.reflect.runtime.JavaMirrors$JavaMirror.typeToJavaClass(JavaMirrors.scala:1300)
at scala.reflect.runtime.JavaMirrors$JavaMirror.runtimeClass(JavaMirrors.scala:192)
at scala.reflect.runtime.JavaMirrors$JavaMirror.runtimeClass(JavaMirrors.scala:54)
at org.apache.spark.sql.catalyst.encoders.ExpressionEncoder$.apply(ExpressionEncoder.scala:60)
at org.apache.spark.sql.Encoders$.product(Encoders.scala:275)
at org.apache.spark.sql.LowPrioritySQLImplicits$class.newProductEncoder(SQLImplicits.scala:248)
at org.apache.spark.sql.SQLImplicits.newProductEncoder(SQLImplicits.scala:34)
at usingcaseclass$.main(usingcaseclass.scala:26)
at usingcaseclass.main(usingcaseclass.scala)
case class Salary(depName: String, empNo: Long, salary: Long)
val empsalary = Seq(Salary("sales", 1, 5000), Salary("personnel", 2, 3900)).toDS
empsalary.show(false)
value toDS is not a member of Seq[Salary]
val empsalary = Seq(Salary("sales", 1, 5000), Salary("personnel", 2, 3900)).toDS
知道如何防止此錯誤
uj5u.com熱心網友回復：
您以錯誤的方式定義了 Seq，這將導致 Seq[Product with Serializable]not Seq[T]on which toDFworks。
以下修改后的行應該適合您。
val family=Seq(Person(1,"john",11),Person(2,"MAR",12),Person(3,"Iweta",10))
family.toDF().show()
--- ----- ------
| id| Name|cityId|
--- ----- ------
| 1| john| 11|
| 2| MAR| 12|
| 3|Iweta| 10|
--- ----- ------
轉載請註明出處，本文鏈接：https://www.uj5u.com/ruanti/406421.html
標籤：
上一篇：Databricks上的ApacheSpark資料生成器功能不起作用
下一篇：跨域請求之間的燒瓶會話不持久