我不斷收到錯誤,我傳遞了太多引數但不知道為什么,因為我正在遵循以下確切示例:
command-629173529675356:9: error: too many arguments for method apply: (colName: String)org.apache.spark.sql.Column in class Dataset
val df_2 = date_format.withColumn("day_of_week", date_format(col("date"), "EEEE"))
我的代碼:
val date_format = df_filter.withColumn("date", to_date(col("pickup_datetime")))
val df_2 = date_format.withColumn("day_of_week", date_format(col("date"), "EEEE"))
感謝您的幫助!
uj5u.com熱心網友回復:
dayofweek 是你正在尋找的功能,所以像這樣
import org.apache.spark.sql.functions.dayofweek
date_format.withColumn("day_of_week", dayofweek(col("date")))
uj5u.com熱心網友回復:
您收到錯誤是因為您將第一個 dataframedate_format命名為與您要使用的 Spark 內置函式同名。因此,當您呼叫 時date_format,您正在檢索資料幀而不是date_format內置函式。
要解決這個問題,您應該重命名您的第一個資料框:
val df_1 = df_filter.withColumn("date", to_date(col("pickup_datetime")))
val df_2 = df_1.withColumn("day_of_week", date_format(col("date"), "EEEE"))
或者確保您date_format通過匯入正確呼叫functions,然后functions.date_format在提取星期幾時呼叫:
import org.apache.spark.sql.functions
val date_format = df_filter.withColumn("date", to_date(col("pickup_datetime")))
val df_2 = date_format.withColumn("day_of_week", functions.date_format(col("date"), "EEEE"))
轉載請註明出處,本文鏈接:https://www.uj5u.com/qianduan/333872.html
