yarn集群運行saprk作業，無法打開spark ui地址-有解無憂

在spark-shell中，spark的ui能打開，但是用yarn來提交作業，yarn的界面能打開，但是打開spark的ui就會報錯，而且在yarn的日志中又顯示成功開啟4040.

SparkUI:54 - Bound SparkUI to 0.0.0.0, and started at http://Spark1:4040

下面時報錯提示



HTTP ERROR 500

Problem accessing /jobs/. Reason:



    Server Error

Caused by:

java.util.NoSuchElementException

	at java.util.Collections$EmptyIterator.next(Collections.java:4189)

	at org.apache.spark.util.kvstore.InMemoryStore$InMemoryIterator.next(InMemoryStore.java:281)

	at org.apache.spark.status.AppStatusStore.applicationInfo(AppStatusStore.scala:38)

	at org.apache.spark.ui.jobs.AllJobsPage.render(AllJobsPage.scala:273)

	at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:82)

	at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:82)

	at org.apache.spark.ui.JettyUtils$$anon$3.doGet(JettyUtils.scala:90)

	at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)

	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)

	at org.spark_project.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)

	at org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584)

	at org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)

	at org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)

	at org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)

	at org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)

	at org.spark_project.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)

	at org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)

	at org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)

	at org.spark_project.jetty.server.Server.handle(Server.java:534)

	at org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:320)

	at org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)

	at org.spark_project.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)

	at org.spark_project.jetty.io.FillInterest.fillable(FillInterest.java:108)

	at org.spark_project.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)

	at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)

	at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)

	at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)

	at org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)

	at org.spark_project.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)

	at java.lang.Thread.run(Thread.java:748)

Powered by Jetty:// 9.3.z-SNAPSHOT

uj5u.com熱心網友回復：

Maven: javax.servlet:javax.servlet-api:3.1.0

delete this depend, maybe can solve your problem.

uj5u.com熱心網友回復：

參考 1 樓 wenxinyu12 的回復:

Maven: javax.servlet:javax.servlet-api:3.1.0

delete this depend, maybe can solve your problem.

now it works,thanks

uj5u.com熱心網友回復：

HTTP ERROR 500
Problem accessing /jobs/. Reason:

Server Error
Caused by:
java.util.NoSuchElementException
at java.util.Collections$EmptyIterator.next(Collections.java:4189)
at org.apache.spark.util.kvstore.InMemoryStore$InMemoryIterator.next(InMemoryStore.java:281)
at org.apache.spark.status.AppStatusStore.applicationInfo(AppStatusStore.scala:38)
at org.apache.spark.ui.jobs.AllJobsPage.render(AllJobsPage.scala:275)
at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:84)
at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:84)
at org.apache.spark.ui.JettyUtils$$anon$3.doGet(JettyUtils.scala:90)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.spark_project.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
at org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584)
at org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.spark_project.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
at org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.spark_project.jetty.server.Server.handle(Server.java:539)
at org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.spark_project.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.spark_project.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.spark_project.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.spark_project.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
Powered by Jetty:// 9.3.z-SNAPSHOT

您好我也遇到了這個報錯，請問您是如何解決的

轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/19035.html

標籤：Spark

上一篇：spark讀取kafka中的不規則json,是canal采集的binlog日志,實時讀取到mysql

下一篇：CLOUD