大家好,本人現在需要使用spark streaming來匯入資料,清洗后寫入oracle資料庫,本人在streaming與資料庫連接問題上遇到問題,希望大家幫我檢查下錯誤,問題出現在下面的代碼中,本人想從dstream 得到的資料插入到oracle資料庫中,但 wordCounts.foreachRDD(rdd =>{ 出現編譯錯誤,希望各位幫我檢查錯誤并指正,多謝啦
wordCounts.foreachRDD(rdd =>{
ps = conn.prepareStatement("insert into testtables values(?,?)")
ps.setString(1, rdd._1)
ps.setInt(2, rdd._2)
ps.executeUpdate()
})
所有的代碼如下
val sparkConf = new SparkConf().setAppName("TextStream")
val sc = new SparkContext(sparkConf)
Class.forName("oracle.jdbc.driver.OracleDriver").newInstance()
val conn = DriverManager.getConnection("jdbc:oracle:thin:@host/oracledb", "user", "password")
var ps: PreparedStatement = null
val ssc = new StreamingContext(sparkConf, Seconds(6))
// Create the FileInputDStream on the directory and use the
val lines = ssc.textFileStream(hdfsHost + "/user/input/")
val words = lines.flatMap(_.split(","))
val wordCounts = words.map(x => (x, 1))
wordCounts.print()
wordCounts.foreachRDD(rdd =>{
ps = conn.prepareStatement("insert into testtables values(?,?)")
ps.setString(1, rdd._1)
ps.setInt(2, rdd._2)
ps.executeUpdate()
})
uj5u.com熱心網友回復：
你至少把什么編譯錯誤貼出來吧uj5u.com熱心網友回復：
uj5u.com熱心網友回復：
你至少把什么編譯錯誤貼出來吧uj5u.com熱心網友回復：
謝謝 樓主分享 辛苦了uj5u.com熱心網友回復：
你試試，看這樣可以不！object Test {
case class Person(words: String,number:Int)
def main(args: Array[String]) {
val sparkConf = new SparkConf().setAppName("TextStream")
val sc = new SparkContext(sparkConf)
Class.forName("oracle.jdbc.driver.OracleDriver").newInstance()
val conn = DriverManager.getConnection("jdbc:oracle:thin:@host/oracledb", "user", "password")
var ps: PreparedStatement = null
val ssc = new StreamingContext(sparkConf, Seconds(6))
// Create the FileInputDStream on the directory and use the
val lines = ssc.textFileStream("/user/input/")
val words = lines.flatMap(_.split(","))
val wordCounts = words.map(x => (x, 1))
// wordCounts.print()
wordCounts.foreachRDD((rdd:RDD[(String,Int)])=>{
rdd.foreach(lines=>{
ps = conn.prepareStatement("insert into testtables values(?,?)")
ps.setString(1, lines._1)
ps.setInt(2, lines._2)
ps.executeUpdate()
})
})
}
}
轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/94362.html
標籤：Spark
上一篇：求primer 破解版及操作教程