我正在使用 Spark 資料幀。我有一個用例，我需要將日期加一。我的增量日期恰好是周末，然后我需要將其增加到下周/作業日。
val df = Seq(
("50312", "2021-12-01", "0.9992019"),
("50312", "2021-12-02", "0.20171201"),
("50312", "2021-12-03", "2.9992019")
).toDF("id","some_date","item_value")
.withColumn("nextworking_day", date_add(col("some_date"),1))
下一個作業日應該是下一個作業日而不是周末。怎么做？
uj5u.com熱心網友回復：
您可以使用dayofweek獲取作業日的編號，如果當天是星期六則加 2，如果是星期五則加 3。
val day = dayofweek(col("some_date"))
val nextworkday = col("some_date") when(day > 5, -day 9).otherwise(1)
val df = Seq(
("50312", "2021-12-01", "0.9992019"),
("50312", "2021-12-02", "0.20171201"),
("50312", "2021-12-03", "2.9992019")
).toDF("id","some_date","item_value")
.withColumn("some_date", col("some_date").cast("date"))
.withColumn("nextworking_day", nextworkday)
df.show()
----- ---------- ---------- ---------------
| id| some_date|item_value|nextworking_day|
----- ---------- ---------- ---------------
|50312|2021-12-01| 0.9992019| 2021-12-02|
|50312|2021-12-02|0.20171201| 2021-12-03|
|50312|2021-12-03| 2.9992019| 2021-12-06|
----- ---------- ---------- ---------------
uj5u.com熱心網友回復：
寫一個檢查日期的 udf 應該可以解決問題下面是在 pyspark 中運行的示例代碼，不包含假期的代碼，但您可以創建一個串列或列舉并根據您的地區添加條件
import pyspark.sql.functions as f
from pyspark.sql.types import TimestampType
from datetime import datetime, timedelta
@f.udf(returnType=TimestampType())
def get_convert_date_udf(date_column):
datetime_object = datetime.strptime(date_column, "%Y-%m-%d")
new_datetime_object = datetime_object timedelta(days=1)
day = new_datetime_object.strftime("%A")
if day == "Sunday":
new_datetime_object = timedelta(days=1)
elif day == "Saturday":
new_datetime_object = timedelta(days=2)
return new_datetime_object
df = df.withColumn("next_working_date",
get_convert_date_udf(f.col("some_date")))
轉載請註明出處，本文鏈接：https://www.uj5u.com/yidong/377872.html