我正在嘗試設定一個簡單的 kafka 消費者應用程式，它使用來自安全 HTTPS kafka 集群的訊息。
這是我的 sbt 構建檔案。
version := "0.1"
//libraryDependencies = "org.apache.spark" % "spark-core_2.11" % "2.0.0"
// https://mvnrepository.com/artifact/org.apache.spark/spark-streaming
libraryDependencies = "org.apache.spark" %% "spark-streaming" % "3.2.0" % "provided"
// https://mvnrepository.com/artifact/org.apache.kafka/kafka
libraryDependencies = "org.apache.kafka" %% "kafka" % "6.1.0-ccs"
d
scalaVersion := "2.13.6"
我的實際消費者代碼是..
package main.scala.kafka
import java.util
import java.util.Properties
import org.apache.kafka.clients.consumer.KafkaConsumer
object consumer extends App {
val TOPIC="amg-dev-time"
val props = new Properties()
props.put("bootstrap.servers", "kafka-localhost.net:9093")
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer")
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer")
props.put("group.id", "amlng-dev-realtime")
val consumer = new KafkaConsumer[String, String](props)
consumer.subscribe(util.Collections.singletonList(TOPIC))
while(true){
val records=consumer.poll(100)
for (record<-records.asScala){
println(record)
}
}
}
當我運行上述設定時，我得到 物件 apache is not a member of package org import org.apache.kafka.clients.consumer.KafkaConsumer
幫我解決這個問題，并幫助我舉例說明如何連接到安全的 kafka 集群以在 Scala 中使用訊息。
重繪 sbt 構建檔案后.. 我得到..
[error] Not found
[error] Not found
[error] not found: /Users/h0j020h/.ivy2/localorg.apache.kafka/kafka_2.13/6.1.0-ccs/ivys/ivy.xml
[error] download error: Caught javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target (PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target) while downloading https://repo1.maven.org/maven2/org/apache/kafka/kafka_2.13/6.1.0-ccs/kafka_2.13-6.1.0-ccs.pom
[error] not found: https://repository.com/content/repositories/pangaea_releases/org/apache/kafka/kafka_2.13/6.1.0-ccs/kafka_2.13-6.1.0-ccs.pom
[error] (ssExtractDependencies) sbt.librarymanagement.ResolveException: Error downloading org.apache.kafka:kafka_2.13:6.1.0-ccs
[error] Not found
[error] Not found
[error] not found: /Users/h0j020h/.ivy2/localorg.apache.kafka/kafka_2.13/6.1.0-ccs/ivys/ivy.xml
[error] download error: Caught javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target (PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target) while downloading https://repo1.maven.org/maven2/org/apache/kafka/kafka_2.13/6.1.0-ccs/kafka_2.13-6.1.0-ccs.pom
[error] not found: https://repository.com/content/repositories/pangaea_releases/org/apache/kafka/kafka_2.13/6.1.0-ccs/kafka_2.13-6.1.0-ccs.pom
[error] Total time: 30 s, completed 19-Oct-2021, 6:21:47 pm
[info] shutting down sbt server```
uj5u.com熱心網友回復：
Kafka 6.1.0-ccs 托管在 Confluent 上。
嘗試添加：
resolvers = "confluent" at "https://packages.confluent.io/maven/"
它應該可以解決問題（例如scastie）
轉載請註明出處，本文鏈接：https://www.uj5u.com/net/327813.html
標籤：斯卡拉