我的資料框中有以下 JSON 結構作為body屬性。我想根據提供的串列從內容中洗掉多個列/屬性，我該如何在 Scala 中執行此操作？
請注意，屬性串列本質上是可變的。
讓說，
要洗掉的列串列： List(alias, firstName, lastName)
輸入
"Content":{
"alias":"Jon",
"firstName":"Jonathan",
"lastName":"Mathew",
"displayName":"Jonathan Mathew",
"createdDate":"2021-08-10T13:06:35.866Z",
"updatedDate":"2021-08-10T13:06:35.866Z",
"isDeleted":false,
"address":"xx street",
"phone":"xxx90"
}
輸出 ：
"Content":{
"displayName":"Jonathan Mathew",
"createdDate":"2021-08-10T13:06:35.866Z",
"updatedDate":"2021-08-10T13:06:35.866Z",
"isDeleted":false,
"address":"xx street",
"phone":"xxx90"
}
uj5u.com熱心網友回復：
您可以從資料框架構中獲取屬性串列，然后Content通過創建一個包含所有屬性的結構來更新列，但要洗掉列串列中的屬性。
這是一個完整的作業示例：
val jsonStr = """{"id": 1,"Content":{"alias":"Jon","firstName":"Jonathan","lastName":"Mathew","displayName":"Jonathan Mathew","createdDate":"2021-08-10T13:06:35.866Z","updatedDate":"2021-08-10T13:06:35.866Z","isDeleted":false,"address":"xx street","phone":"xxx90"}}"""
val df = spark.read.json(Seq(jsonStr).toDS)
val attrToDrop = Seq("alias", "firstName", "lastName")
val contentAttrList = df.select("Content.*").columns
val df2 = df.withColumn(
"Content",
struct(
contentAttrList
.filter(!attrToDrop.contains(_))
.map(c => col(s"Content.$c")): _*
)
)
df2.printSchema
//root
// |-- Content: struct (nullable = false)
// | |-- address: string (nullable = true)
// | |-- createdDate: string (nullable = true)
// | |-- displayName: string (nullable = true)
// | |-- isDeleted: boolean (nullable = true)
// | |-- phone: string (nullable = true)
// | |-- updatedDate: string (nullable = true)
// |-- id: long (nullable = true)
uj5u.com熱心網友回復：
您可以使用drop一次洗掉多個列：
val newDataframe = oldDataframe.drop("alias", "firstName", "lastName")
檔案：
/**
* Returns a new Dataset with columns dropped.
* This is a no-op if schema doesn't contain column name(s).
*
* This method can only be used to drop top level columns. the colName string is treated literally
* without further interpretation.
*
* @group untypedrel
* @since 2.0.0
*/
@scala.annotation.varargs
def drop(colNames: String*): DataFrame
轉載請註明出處，本文鏈接：https://www.uj5u.com/yidong/381435.html