我現在很困惑。請檢查這是否正確。
4 種情況命令如下:
# It mean, yarn is cluster mode and deploy cluster mode.
# cluster have YARN Container(have Spark AM, Spark Driver) and YARN node manager.
spark-submit --master yarn --deploy-mode cluster
# It mean, yarn is cluster mode and deploy client mode.
# client have Spark Driver.
# cluster have YARN Container(have Spark AM, Spark Driver) and YARN node manager.
spark-submit --master yarn --deploy-mode client
# It mean, yarn is client mode and deploy cluster mode.
# cluster have YARN Container(have Spark AM) and YARN node manager.
spark-submit --master yarn-client --deploy-mode cluster
# It mean, yarn is client mode and deploy client mode.
# client have Spark Driver.
# cluster have YARN Container(have Spark AM) and YARN node manager.
spark-submit --master yarn-client --deploy-mode client
上面代碼的解釋是否正確?
uj5u.com熱心網友回復:
#使用yarn,將驅動部署到yarn集群中。
spark-submit --master yarn --deploy-mode cluster
#使用yarn,在我的本地機器(正在啟動代碼的機器)上部署驅動程式
spark-submit --master yarn --deploy-mode client # this is the default if you don't specify --deploy-mode
這些不再是實際選項,因此它們真的不值得討論:
spark-submit --master yarn-client --deploy-mode cluster
spark-submit --master yarn-client --deploy-mode client
--master yarn-client也許是早期版本的 spark 的一個選項,但今天沒有使用。(如上述檔案中所述)
轉載請註明出處,本文鏈接:https://www.uj5u.com/gongcheng/394037.html
上一篇:將Parquet檔案從Zeppelinexecutor保存到HDFS時出現的問題
下一篇:EMREKS無法啟動驅動程式艙
