我想在字串中查找重復項，我想知道有什么方法可以使用 spark sql 找到它。下面是我寫的查詢。
spark.sql("""select case when lower(value) like '%,code,%' or lower(value) like '%,code%' or lower(value) like '%code,%' then 'Y' else 'N' end as value from input""").show(false)
輸入 ：
val sample = Seq(("code,code")).toDF("value")
問題是如果值“代碼”重復兩次怎么辦？在這種情況下，我的要求是回傳標志“N”。有沒有辦法使用 Spark SQL 做到這一點？請分享您的建議。TIA
uj5u.com熱心網友回復：
您可以使用長度確定出現次數并替換
set @a = 'code,aaa';
set @b = 'code,code';
set @c = 'aaa,bbb,ccc';
select @a, length(@a) - length(replace(@a,'code','')) lengthdiff,
case when length(@a) - length(replace(@a,'code','')) = 4 then 'y'
when length(@a) - length(replace(@a,'code','')) >= 4 then 'n'
else 'n'
end occurances
;
---------- ------------ ------------
| @a | lengthdiff | occurances |
---------- ------------ ------------
| code,aaa | 4 | y |
---------- ------------ ------------
1 row in set (0.001 sec)
select @b, length(@b) - length(replace(@b,'code','')) lengthdiff,
case when length(@b) - length(replace(@b,'code','')) = 4 then 'y'
when length(@b) - length(replace(@b,'code','')) > 4 then 'n'
else 'n'
end occurances
;
----------- ------------ ------------
| @b | lengthdiff | occurances |
----------- ------------ ------------
| code,code | 8 | n |
----------- ------------ ------------
1 row in set (0.000 sec)
select @c, length(@c) - length(replace(@c,'code','')) lengthdiff,
case when length(@c) - length(replace(@c,'code','')) = 4 then 'y'
when length(@c) - length(replace(@c,'code','')) > 4 then 'n'
else 'n'
end occurances
;
------------- ------------ ------------
| @c | lengthdiff | occurances |
------------- ------------ ------------
| aaa,bbb,ccc | 0 | n |
------------- ------------ ------------
1 row in set (0.001 sec)
轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/447328.html
標籤：mysql 数据框 阿帕奇火花 apache-spark-sql