我用local是可以正常運行jar包的，但是集群就失敗了
[hadoop@server1 sbin]$ ./start-master.sh
starting org.apache.spark.deploy.master.Master, logging to /home/hadoop/local/spark-3.0.0/spark-3.0.0-bin-hadoop3.2/logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-server1.out
[hadoop@server1 sbin]$ ./start-slaves.sh spark://server1:7077
server2: starting org.apache.spark.deploy.worker.Worker, logging to /home/hadoop/local/spark-3.0.0/spark-3.0.0-bin-hadoop3.2/logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-server2.out
[hadoop@server1 sbin]$ spark-submit --name testSpark --class org.userData.Clean --master spark://server1:7077 --deploy-mode cluster /home/hadoop/test/test_spark-1.0-SNAPSHOT-jar-with-dependencies.jar
2020-07-25 15:34:14,828 ERROR deploy.ClientEndpoint: Exception from cluster was: java.io.IOException: Failed to create directory /home/hadoop/local/spark-3.0.0/spark-3.0.0-bin-hadoop3.2/work/driver-20200725153409-0000
java.io.IOException: Failed to create directory /home/hadoop/local/spark-3.0.0/spark-3.0.0-bin-hadoop3.2/work/driver-20200725153409-0000
at org.apache.spark.deploy.worker.DriverRunner.createWorkingDirectory(DriverRunner.scala:145)
at org.apache.spark.deploy.worker.DriverRunner.prepareAndRunDriver(DriverRunner.scala:176)
at org.apache.spark.deploy.worker.DriverRunner$$anon$2.run(DriverRunner.scala:96)
[hadoop@server1 sbin]$
work檔案夾給的權限是777，用戶也是同樣的用戶
uj5u.com熱心網友回復：
代碼里有指定路徑嗎轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/9471.html
標籤：Spark