如何遍歷Scala串列中的每個字典？

我有一個JSON字串，它基本上是一個字典串列。我正在嘗試使用for回圈來遍歷它。基本上在每次迭代中，我應該能夠獲得諸如index、name、source、s3之類的鍵。有人可以幫忙嗎？所以在第一次迭代中我應該能夠得到index=1等source=a。

import scala.util.parsing.json._
class CC[T] { def unapply(a:Any):Option[T] = Some(a.asInstanceOf[T]) }

object M extends CC[Map[String, Any]]
object A extends CC[List[Any]] //for s3
object I extends CC[Double]
object S extends CC[String]
object E extends CC[String]
object F extends CC[String]
object G extends CC[Map[String, Any]]

val jsonString =
    """
    [{
        "index": 1,
      "source": "a",
      "name": "v",
      "s3": [{
        "path": "s3://1",
        "bucket": "p",
        "key": "r"
      }]
     },
     {
        "index": 2,
      "source": "b",
      "name": "b",
      "s3": [{
        "path": "s3://1",
        "bucket": "p",
        "key": "r"
      }]
     }]
     
    """.stripMargin
    
println(List(JSON.parseFull(jsonString)) )

uj5u.com熱心網友回復：

在您的示例中，您正在嘗試使用提取器。基于 unapply 方法的提取器。通常它需要物件并回傳引數。要實作這一點，您需要為要使用的每種型別定義提取器，但您不需要像示例S,E,F中那樣復制它們。將物件提取到特定型別后，您可以對其進行操作。

  import scala.collection.Map
  import scala.util.parsing.json._

  class CC[T] {
    def unapply(a: Any): Option[T] = Some(a.asInstanceOf[T])
  }

  object M extends CC[Map[String, Any]]
  object L extends CC[List[Any]]
  object S extends CC[String]
  object D extends CC[Double]

  val jsonString2 =
    """
    [{
        "index": 1,
      "source": "a",
      "name": "v",
      "s3": [{
        "path": "s3://1",
        "bucket": "p",
        "key": "r"
      }]
     },
     {
        "index": 2,
      "source": "b",
      "name": "b",
      "s3": [{
        "path": "s3://1",
        "bucket": "p",
        "key": "r"
      }]
     }]

    """.stripMargin

  val result = for {
    Some(L(list)) <- List(JSON.parseFull(jsonString2))
    M(map) <- list
    D(index) = map("index")
    S(source) = map("source")
    S(name) = map("name")
    L(s3Connections) = map("s3")
    M(s3) <- s3Connections
    S(path) = s3("path")
    S(bucket) = s3("bucket")
    S(key) = s3("key")
  } yield (index, source, name, path, bucket, key)

  println(result)
  
//  List((1.0,a,v,s3://1,p,r), (2.0,b,b,s3://1,p,r))

此外，您可以使用任何庫來決議 json 檔案。例如，我在下面提供了使用 circe 庫的示例。

  import io.circe.parser.parse
  import io.circe.{Decoder, Json}
  
  case class S3Config(path: String, bucket: String, key: String)

  object S3Config {
    implicit val decoder: Decoder[S3Config] = deriveDecoder[S3Config]
  }

  case class Source(index: Int, source: String, name: String, s3: Vector[S3Config])

  object Source {
    implicit val decoder: Decoder[Source] = deriveDecoder[Source]
  }

  val jsonString =
    """
    [{
        "index": 1,
      "source": "a",
      "name": "v",
      "s3": [{
        "path": "s3://1",
        "bucket": "p",
        "key": "r"
      }]
     },
     {
        "index": 2,
      "source": "b",
      "name": "b",
      "s3": [{
        "path": "s3://1",
        "bucket": "p",
        "key": "r"
      }]
     }]

    """.stripMargin

  val sources = parse(jsonString).getOrElse(Json.Null).as[Vector[Source]].getOrElse(Vector.empty[Source])
  for (source <- sources) {
    println(source.source)
    println(source.index)
    println(source.s3)
  }

  //  a
  //  1
  //  Vector(S3Config(s3://1,p,r))
  //      b
  //  2
  //  Vector(S3Config(s3://1,p,r))

uj5u.com熱心網友回復：

如果你想在沒有 的情況下這樣做spark，那么你可以嘗試這種方式：

JSON.parseFull(jsonString) match {
    case Some(x) => {
        val json_list = x.asInstanceOf[List[Map[String, Any]]]
        json_list.foreach { json_map =>
            // To extract any base level fields

            // You can do any action on the extracted field. This is just to illustrate the extraction method
            println("index: "   json_map("index"))
            println("source: "   json_map("source"))

            // To extract fields from inside a list: example element at index 0 of list s3

            // You can do any action on the extracted field. This is just to illustrate the extraction method
            val s3_list = json_map("s3").asInstanceOf[List[Any]]
            println("s3: "   s3_list(0))

            // To extract fields from inside a map: example path from 0th element of list s3

            // You can do any action on the extracted field. This is just to illustrate the extraction method
            println("s3.path: "   s3_list(0).asInstanceOf[Map[String, Any]]("path"))

        }
    }
    case None => {
        println("Improper JSON")
    }
}

// Output of the print statements

// Iteration #1:

index: 1.0
source: a
s3: Map(path -> s3://1, bucket -> p, key -> r)
s3.path: s3://1

// Iteration #2:

index: 2.0
source: b
s3: Map(path -> s3://1, bucket -> p, key -> r)
s3.path: s3://1

標籤：斯卡拉 阿帕奇火花 pyspark apache-spark-sql

上一篇：如何遍歷資料框列中的陣列，并在另一個資料框中進行查找-Scala，Spark

下一篇：在Spark上讀取CSV檔案