我的URL資料框中的一列中有資料，我需要從查詢字串中決議出引數并為其創建新列。
有時引數會存在，有時不會，而且它們沒有特定的保證順序，所以我需要能夠按名稱找到它們。我正在用 Qcala 寫這個，但語法不正確，希望得到一些幫助。
我的代碼：
val df = Seq(
(1, "https://www.mywebsite.com/dummyurl/single?originlatitude=35.0133612060147&originlongitude=-116.156211232302&origincountrycode=us&originstateprovincecode=ca&origincity=boston&originradiusmiles=250&datestart=2021-12-23t00:00:00"),
(2, "https://www.mywebsite.com/dummyurl/single?originlatitude=19.9141319141121&originlongitude=-56.1241881401291&origincountrycode=us&originstateprovincecode=pa&origincity=york&originradiusmiles=100&destinationlatitude=40.7811012268066&destinationlon")
).toDF("key", "URL")
val result = df
// .withColumn("param_name", $"URL")
.withColumn("parsed_url", explode(split(expr("parse_url(URL, 'QUERY')"), "&")))
.withColumn("parsed_url2", split($"parsed_url", "="))
// .withColumn("exampletest",$"URL".map(kv: String => (kv.split("=")(0), kv.split("=")(1))) )
.withColumn("Search_OriginLongitude", split($"URL","\\?"))
.withColumn("Search_OriginLongitude2", split($"Search_OriginLongitude"(1),"&"))
// .map(kv: Any => (kv.split("=")(0), kv.split("=")(1)))
// .toMap
// .get("originlongitude"))
display(result)
預期結果：
--- -------------------- -------------------- --------------------
|KEY| URL| originlatitude | originlongitude |
--- -------------------- -------------------- --------------------
| 1|https://www.myweb...| 35.0133612060147 | -116.156211232302 |
| 2|https://www.myweb...| 19.9141319141121 | -56.1241881401291 |
--- -------------------- -------------------- --------------------
uj5u.com熱心網友回復：
parse_url函式實際上可以key為要提取的查詢引數名稱采用第三個引數，如下所示：
val result = df
.withColumn("Search_OriginLongitude", expr("parse_url(URL, 'QUERY', 'originlatitude')"))
.withColumn("Search_OriginLongitude2", expr("parse_url(URL, 'QUERY', 'originlongitude')"))
result.show
// --- -------------------- ---------------------- -----------------------
//|key| URL|Search_OriginLongitude|Search_OriginLongitude2|
// --- -------------------- ---------------------- -----------------------
//| 1|https://www.myweb...| 35.0133612060147| -116.156211232302|
//| 2|https://www.myweb...| 19.9141319141121| -56.1241881401291|
// --- -------------------- ---------------------- -----------------------
或者您可以使用str_to_map函式來創建這樣的地圖parameter->value：
val result = df
.withColumn("URL", expr("str_to_map(split(URL,'[?]')[1],'&','=')"))
.withColumn("Search_OriginLongitude", col("URL").getItem("originlatitude"))
.withColumn("Search_OriginLongitude2", col("URL").getItem("originlongitude"))
轉載請註明出處，本文鏈接：https://www.uj5u.com/caozuo/396452.html