在 PySpark 中，我有一個資料框，我試圖用陣列決議多個列。資料框中的最后兩行包含多個我想決議為單獨行的值。
------------- --------------- ------------- -------------------- -------------- ------------- ---------------------- --------------
| WB-API-CNTY | WB-API-UNIQUE | WB-OIL-CODE | WB-OIL-LSE-NBR | WB-OIL-DIST | WB-GAS-CODE | WB-GAS-RRC-ID | WB-GAS-DIS |
------------- --------------- ------------- -------------------- -------------- ------------- ---------------------- --------------
| 449 | 80212 | [] | [] | [] | [] | [] | [] |
------------- --------------- ------------- -------------------- -------------- ------------- ---------------------- --------------
| 449 | 80214 | ["O"] | ["05361"] | ["06"] | ["O"] | ["060536"] | ["00"] |
------------- --------------- ------------- -------------------- -------------- ------------- ---------------------- --------------
| 449 | 80222 | ["O", "O"] | ["01718", "05492"] | ["06", "06"] | ["O", "O"] | ["060171", "060549"] | ["00", "00"] |
------------- --------------- ------------- -------------------- -------------- ------------- ---------------------- --------------
| 451 | 00005 | ["G", "O"] | ["5568", "04351"] | ["10", "09"] | ["G", "O"] | ["105568", "090435"] | ["09", "00"] |
------------- --------------- ------------- -------------------- -------------- ------------- ---------------------- --------------
結果：
------------- --------------- ------------- ---------------- ------------- ------------- --------------- ------------
| WB-API-CNTY | WB-API-UNIQUE | WB-OIL-CODE | WB-OIL-LSE-NBR | WB-OIL-DIST | WB-GAS-CODE | WB-GAS-RRC-ID | WB-GAS-DIS |
------------- --------------- ------------- ---------------- ------------- ------------- --------------- ------------
| 449 | 80212 | | | | | | |
------------- --------------- ------------- ---------------- ------------- ------------- --------------- ------------
| 449 | 80214 | O | 05361 | 06 | O | 060536 | 00 |
------------- --------------- ------------- ---------------- ------------- ------------- --------------- ------------
| 449 | 80222 | O | 01718 | 06 | O | 060171 | 00 |
------------- --------------- ------------- ---------------- ------------- ------------- --------------- ------------
| 449 | 80222 | O | 05492 | 06 | O | 060549 | 00 |
------------- --------------- ------------- ---------------- ------------- ------------- --------------- ------------
| 451 | 00005 | G | 5568 | 10 | G | 105568 | 09 |
------------- --------------- ------------- ---------------- ------------- ------------- --------------- ------------
| 451 | 00005 | O | 04351 | 09 | O | 090435 | 00 |
------------- --------------- ------------- ---------------- ------------- ------------- --------------- ------------
uj5u.com熱心網友回復：
array_cols = ['WB-OIL-CODE', 'WB-OIL-LSE-NBR', 'WB-OIL-DIST', 'WB-GAS-CODE', 'WB-GAS-RRC-ID', 'WB-GAS-DIS']
other_cols = [c for c in df.columns if c not in array_cols]
df = df.select(
*other_cols,
F.expr(f"inline(arrays_zip({'`' '`,`'.join(array_cols) '`'}))")
)
轉載請註明出處，本文鏈接：https://www.uj5u.com/gongcheng/497520.html
標籤：数组 数据框 阿帕奇火花 pyspark apache-spark-sql
上一篇：計算視窗中過濾值的出現次數