spark1.6.0啟動時,遇到問題,請大家幫忙指教
[root@Master spark-1.6.0-bin-hadoop2.6]# sbin/start-all.sh
starting org.apache.spark.deploy.master.Master, logging to /home/softwares/spark-1.6.0-bin-hadoop2.6/logs/spark-root-org.apache.spark.deploy.master.Master-1-Master.Hadoop.out
failed to launch org.apache.spark.deploy.master.Master:
/bin/java: 沒有那個檔案或目錄-bin-hadoop2.6/bin/spark-class: line 86: /home/softwares/jdk1.7.0_79
full log in /home/softwares/spark-1.6.0-bin-hadoop2.6/logs/spark-root-org.apache.spark.deploy.master.Master-1-Master.Hadoop.out
Slave1.Hadoop: starting org.apache.spark.deploy.worker.Worker, logging to /home/softwares/spark-1.6.0-bin-hadoop2.6/logs/spark-root-org.apache.spark.deploy.worker.Worker-1-Slave1.Hadoop.out
Slave2.Hadoop: starting org.apache.spark.deploy.worker.Worker, logging to /home/softwares/spark-1.6.0-bin-hadoop2.6/logs/spark-root-org.apache.spark.deploy.worker.Worker-1-Slave2.Hadoop.out
Slave2.Hadoop: failed to launch org.apache.spark.deploy.worker.Worker:
Slave1.Hadoop: failed to launch org.apache.spark.deploy.worker.Worker:
/bin/java: 沒有那個檔案或目錄res/spark-1.6.0-bin-hadoop2.6/bin/spark-class: line 86: /home/softwares/jdk1.7.0_79
/bin/java: 沒有那個檔案或目錄res/spark-1.6.0-bin-hadoop2.6/bin/spark-class: line 86: /home/softwares/jdk1.7.0_79
Slave2.Hadoop: full log in /home/softwares/spark-1.6.0-bin-hadoop2.6/logs/spark-root-org.apache.spark.deploy.worker.Worker-1-Slave2.Hadoop.out
Slave1.Hadoop: full log in /home/softwares/spark-1.6.0-bin-hadoop2.6/logs/spark-root-org.apache.spark.deploy.worker.Worker-1-Slave1.Hadoop.out
[root@Master spark-1.6.0-bin-hadoop2.6]
組態檔:
export JAVA_HOME=/home/softwares/jdk1.7.0_79
export SCALA_HOME=/home/softwares/scala-2.10.6
export HADOOP_HOME=/home/softwares/hadoop-2.6.0
export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop
export SPARK_MASTER_IP=59.67.152.31
export SPARK_WORKER_MEMORY=2g
export master=spark://59.67.152.31
export SPARK_EXECUTOR_MEMORY=2g
export SPARK_DRIVER_MEMORY=2g
export SPARK_WORKDER_CORES=2
環境變數:
export JAVA_HOME=/home/softwares/jdk1.7.0_79
export HADOOP_HOME=/home/softwares/hadoop-2.6.0
export SCALA_HOME=/home/softwares/scala-2.10.6
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
export HIVE_HOME=/home/softwares/apache-hive-1.2.1-bin
export SPARK_HOME=/home/softwares/spark-1.6.0-bin-hadoop2.6
export PATH=$PATH:$JAVA_HOME/bin:$HIVE_HOME/bin:$FLUME_HOME/bin:$SCALA_HOME/bin:${SPARK_HOME}/bin:${SPARK_HOME}/sbin:$PATH
export HIVE_CONF_DIR=${HIVE_HOME}/conf
export FLUME_HOME=/home/softwares/apache-flume-1.6.0-bin
export FLUME_CONF_DIR=$FLUME_HOME/conf
export CLASSPATH=.:$CLASSPATH:$HIVE_HOME/lib:$SPARK_HOME/lib
uj5u.com熱心網友回復:
vi spark-env.sh 發現組態檔有不知名的字符,直接打開組態檔沒有發現錯誤,洗掉不知所以然的字符,重新啟動成功。轉載請註明出處,本文鏈接:https://www.uj5u.com/qita/67595.html
標籤:Spark
