spark運行在yarn上。
jdk : 1.7.0_71
hadoop : 2.6
spark : 1.6
spark配置 : 只配置了spark-env.sh檔案,其他都沒配置,配置如下
export SPARK_HOME=/yarn/spark-1.6.0-bin-hadoop2.6
export HADOOP_HOME=/yarn/hadoop-2.6.0
export YARN_CONF_DIR=/yarn/hadoop-2.6.0/etc/hadoop
export HBASE_CONF_DIR=/yarn/hbase-0.98.14-hadoop2/conf
問題一 :
運行 : 一個簡單的wordcount程式
spark-submit --class com.cyc.spark.JavaWordCount --master yarn --deploy-mode client /yarn/file/sparktest.jar
一直列印:
16/02/23 21:11:30 INFO Client: Application report for application_1456228788292_0009 (state: ACCEPTED)
16/02/23 21:11:31 INFO Client: Application report for application_1456228788292_0009 (state: ACCEPTED)
16/02/23 21:11:32 INFO Client: Application report for application_1456228788292_0009 (state: ACCEPTED)
16/02/23 21:11:33 INFO Client: Application report for application_1456228788292_0009 (state: ACCEPTED)
16/02/23 21:11:34 INFO Client: Application report for application_1456228788292_0009 (state: ACCEPTED)
在yarn監管頁面,顯示

求解,為什么程式不能跑?
問題二:
修改spark-env.sh檔案,將export YARN_CONF_DIR=/yarn/hadoop-2.6.0/etc/hadoop隨便改一個路徑,例如 export YARN_CONF_DIR=/yarn/hadoop
程式也一直Application report for application,在yarn的監管頁面也有顯示對應的application。
求解,為什么路徑不對也能被yarn接收到
uj5u.com熱心網友回復:
樓主,問題一如何解決的呢?uj5u.com熱心網友回復:
樓主,問題一怎么解決的呀uj5u.com熱心網友回復:
看日志 ,這樣看不出什么轉載請註明出處,本文鏈接:https://www.uj5u.com/qita/72677.html
標籤:Spark
