問題描述：
1. spark-2.1.1-bin-hadoop2.7，CentOS 7，基于Standalone的高可用集群安裝，共有5個節點。
2. 我的問題是，在每個節點的SPARK_HOME/logs下有幾個日志檔案非常大，而且不能按大小或日期滾動：spark-admin-org.apache.spark.deploy.worker.Worker-1-HOST_NAME.out、spark-admin-org.apache.spark.deploy.master.Master-1-HOST_NAME.out、spark-admin-org.apache.spark.deploy.history.HistoryServer-1-HOST_NAME.out，其級別好多是DEBUG級別的，摘錄幾行如下：
20-01-20.10:27:56.704 [WorkerUI-27 ] DEBUG SelectChannelEndPoint - Local interests updating 0 -> 1 for SelectChannelEndPoint@7dedb6f1{/10.221.216.68:43512<->8081,Open,in,OSHUT,R,-,0/30000,HttpConnection}{io=1,kio=0,kro=1}
20-01-20.10:27:56.704 [WorkerUI-27 ] DEBUG SelectorManager - Queued change org.spark_project.jetty.io.SelectChannelEndPoint$1@64e3a4da
20-01-20.10:27:56.704 [WorkerUI-25-selector-ServerConnectorManager@1a0e7661/1] DEBUG SelectorManager - Selector loop woken up from select, 0/1 selected
20-01-20.10:27:56.704 [WorkerUI-25-selector-ServerConnectorManager@1a0e7661/1] DEBUG SelectorManager - Running change org.spark_project.jetty.io.SelectChannelEndPoint$1@64e3a4da
20-01-20.10:27:56.704 [WorkerUI-25-selector-ServerConnectorManager@1a0e7661/1] DEBUG SelectChannelEndPoint - Key interests updated 0 -> 1 on SelectChannelEndPoint@7dedb6f1{/10.221.216.68:43512<->8081,Open,in,OSHUT,R,-,0/30000,HttpConnection}{io=1,kio=1,kro=1}
20-01-20.10:27:56.704 [WorkerUI-25-selector-ServerConnectorManager@1a0e7661/1] DEBUG SelectorManager - Selector loop waiting on select
20-01-20.10:27:56.707 [WorkerUI-25-selector-ServerConnectorManager@1a0e7661/1] DEBUG SelectorManager - Selector loop woken up from select, 1/1 selected
20-01-20.10:27:56.708 [WorkerUI-25-selector-ServerConnectorManager@1a0e7661/1] DEBUG SelectChannelEndPoint - Key interests updated 1 -> 0 on SelectChannelEndPoint@7dedb6f1{/10.221.216.68:43512<->8081,Open,in,OSHUT,R,-,3/30000,HttpConnection}{io=1,kio=0,kro=1}
20-01-20.10:27:56.708 [WorkerUI-25-selector-ServerConnectorManager@1a0e7661/1] DEBUG SelectChannelEndPoint - Local interests updating 1 -> 0 for SelectChannelEndPoint@7dedb6f1{/10.221.216.68:43512<->8081,Open,in,OSHUT,R,-,4/30000,HttpConnection}{io=0,kio=0,kro=1}
20-01-20.10:27:56.708 [WorkerUI-25-selector-ServerConnectorManager@1a0e7661/1] DEBUG SelectorManager - Queued change org.spark_project.jetty.io.SelectChannelEndPoint$1@64e3a4da
20-01-20.10:27:56.708 [WorkerUI-25-selector-ServerConnectorManager@1a0e7661/1] DEBUG AbstractConnection - FILL_INTERESTED-->FILLING HttpConnection@74a9ce80[FILLING,SelectChannelEndPoint@7dedb6f1{/10.221.216.68:43512<->8081,Open,in,OSHUT,-,-,4/30000,HttpConnection}{io=0,kio=0,kro=1}][p=HttpParser{s=CLOSED,0 of 0},g=HttpGenerator{s=START},c=HttpChannelOverHttp@743ee72f{r=1,c=false,a=IDLE,uri=}]
20-01-20.10:27:56.708 [WorkerUI-25-selector-ServerConnectorManager@1a0e7661/1] DEBUG SelectorManager - Running change org.spark_project.jetty.io.SelectChannelEndPoint$1@64e3a4da
20-01-20.10:27:56.708 [WorkerUI-25-selector-ServerConnectorManager@1a0e7661/1] DEBUG SelectorManager - Selector loop waiting on select
20-01-20.10:27:56.708 [WorkerUI-30 ] DEBUG HttpConnection - HttpConnection@74a9ce80[FILLING,SelectChannelEndPoint@7dedb6f1{/10.221.216.68:43512<->8081,Open,in,OSHUT,-,-,4/30000,HttpConnection}{io=0,kio=0,kro=1}][p=HttpParser{s=CLOSED,0 of 0},g=HttpGenerator{s=START},c=HttpChannelOverHttp@743ee72f{r=1,c=false,a=IDLE,uri=}] onFillable HttpChannelState@485a5da2{s=IDLE i=true a=null}
20-01-20.10:27:56.708 [WorkerUI-30 ] DEBUG ChannelEndPoint - filled -1 SelectChannelEndPoint@7dedb6f1{/10.221.216.68:43512<->8081,Open,in,OSHUT,-,-,4/30000,HttpConnection}{io=0,kio=0,kro=1}
20-01-20.10:27:56.708 [WorkerUI-30 ] DEBUG ChannelEndPoint - ishut SelectChannelEndPoint@7dedb6f1{/10.221.216.68:43512<->8081,Open,in,OSHUT,-,-,4/30000,HttpConnection}{io=0,kio=0,kro=1}
20-01-20.10:27:56.708 [WorkerUI-30 ] DEBUG AbstractEndPoint - onClose SelectChannelEndPoint@7dedb6f1{/10.221.216.68:43512<->8081,CLOSED,ISHUT,OSHUT,-,-,4/30000,HttpConnection}{io=0,kio=0,kro=1}
20-01-20.10:27:56.708 [WorkerUI-30 ] DEBUG ChannelEndPoint - close SelectChannelEndPoint@7dedb6f1{/10.221.216.68:43512<->8081,CLOSED,ISHUT,OSHUT,-,-,4/30000,HttpConnection}{io=0,kio=0,kro=1}
20-01-20.10:27:56.708 [WorkerUI-30 ] DEBUG SelectorManager - Destroyed SelectChannelEndPoint@7dedb6f1{/10.221.216.68:43512<->8081,CLOSED,ISHUT,OSHUT,-,-,4/30000,HttpConnection}{io=0,kio=-1,kro=-1}
20-01-20.10:27:56.708 [WorkerUI-30 ] DEBUG AbstractConnection - onClose HttpConnection@74a9ce80[FILLING,SelectChannelEndPoint@7dedb6f1{/10.221.216.68:43512<->8081,CLOSED,ISHUT,OSHUT,-,-,4/30000,HttpConnection}{io=0,kio=-1,kro=-1}][p=HttpParser{s=CLOSED,0 of 0},g=HttpGenerator{s=START},c=HttpChannelOverHttp@743ee72f{r=1,c=false,a=IDLE,uri=}]
20-01-20.10:27:56.708 [WorkerUI-30 ] DEBUG AbstractEndPoint - onClose SelectChannelEndPoint@7dedb6f1{/10.221.216.68:43512<->8081,CLOSED,ISHUT,OSHUT,-,-,4/30000,HttpConnection}{io=0,kio=-1,kro=-1}
20-01-20.10:27:56.708 [WorkerUI-30 ] DEBUG HttpParser - atEOF HttpParser{s=CLOSED,0 of 0}
20-01-20.10:27:56.708 [WorkerUI-30 ] DEBUG HttpParser - parseNext s=CLOSED HeapByteBuffer@76b3285a[p=0,l=0,c=16384,r=0]={<<<>>>HEAD / HTTP/1.0\r\n...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}
3. SPARK_HOME/conf/log4j.properties也修改了，但無效：
# Set everything to be logged to the console
log4j.rootCategory=INFO, file
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n
# Set the default spark-shell log level to WARN. When running the spark-shell, the
# log level for this class is used to overwrite the root logger's log level, so that
# the user can have different defaults for the shell and regular Spark apps.
log4j.logger.org.apache.spark.repl.Main=WARN
# Settings to quiet third party logs that are too verbose
log4j.logger.org.spark_project.jetty=WARN
log4j.logger.org.spark_project.jetty.util.component.AbstractLifeCycle=ERROR
log4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=INFO
log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=INFO
log4j.logger.org.apache.parquet=ERROR
log4j.logger.parquet=ERROR
# SPARK-9183: Settings to avoid annoying messages when looking up nonexistent UDFs in SparkSQL with Hive support
log4j.logger.org.apache.hadoop.hive.metastore.RetryingHMSHandler=FATAL
log4j.logger.org.apache.hadoop.hive.ql.exec.FunctionRegistry=ERROR
log4j.appender.file = org.apache.log4j.DailyRollingFileAppender
log4j.appender.file.File = /export/data/spark/logs/spark.log
log4j.appender.file.Append = true
log4j.appender.file.Threshold = INFO
log4j.appender.file.DatePattern='.'yyyy-MM-dd
log4j.appender.file.layout = org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern = %-d{yyyy-MM-dd HH:mm:ss} [ %t:%r ] - [ %p ] %m%n
log4j.appender.file.encoding=UTF-8
log4j.logger.org.apache.spark.deploy.history.HistoryServer=INFO,historyserver
log4j.appender.historyserver=org.apache.log4j.RollingFileAppender
log4j.appender.historyserver.layout=org.apache.log4j.PatternLayout
log4j.appender.historyserver.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n
log4j.appender.historyserver.MaxFileSize=1GB
log4j.appender.historyserver.MaxBackupIndex=7
log4j.appender.historyserver.File=/export/servers/spark/logs/historyserver.log
log4j.logger.org.mortbay=WARN
log4j.logger.org.apache.hadoop=WARN
謝謝。
uj5u.com熱心網友回復：
終于找到原因了。由于我需要在Spark環境下執行業務代碼，我修改了spark-env.sh，添加了：
export SPARK_CLASSPATH=/export/servers/appJars/*
這里放的是我的業務依賴的第三方jar，有幾十個jar檔案，去掉該行后，SPARK_HOME/conf/log4j.properties，就生效了。
結貼！
轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/17413.html
標籤：Spark
上一篇：服務器的問題