起因
我使用的是 hadoop-3.1.3 版本，在進行基準測驗時，執行以下代碼；
[root@tcloud mapreduce]# hadoop jar ./hadoop-mapreduce-client-jobclient-3.1.3-tests.jar TestDFSIO -write -nrFiles 10 -size 10kb -resFile home/hadoop/tmp/TestDFSIO.log
問題一【Retrying connect to server】
無法連接 ResourceManager：
2021-08-16 14:25:30,253 INFO client.RMProxy: Connecting to ResourceManager at tcloud/172.17.0.8:8032
2021-08-16 14:25:31,339 INFO ipc.Client: Retrying connect to server: tcloud/172.17.0.8:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
解決：
原因是未啟動yarn，啟動即可：
[root@tcloud mapreduce]# start-yarn.sh
問題二【InvalidResourceRequestException】
無效的資源請求，請求的資源 type=[memory-mb] < 0或大于允許的最大分配【這是使用基礎配置云服務器的尷尬，由于服務器是1核心2G記憶體，所以配置memory-mb時配的低，造成請求資源大于允許的最大記憶體】：
2021-08-16 15:22:10,535 INFO mapreduce.JobSubmitter: Cleaning up the staging area
/tmp/hadoop-yarn/staging/root/.staging/job_1629098481992_0001
java.io.IOException: org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException:
Invalid resource request, requested resource type=[memory-mb] < 0 or greater than maximum
allowed allocation. Requested resource=<memory:1536, vCores:1>, maximum allowed
allocation=<memory:256, vCores:4>, please note that maximum allowed allocation is
calculated by scheduler based on maximum resource of registered NodeManagers, which might
be less than configured maximum allocation=<memory:256, vCores:4>
解決：
修改 ${HADOOP_HOME}/etc/hadoop/yarn-site.xml 檔案：
# 停掉yarn
[root@tcloud hadoop]# stop-yarn.sh
# 修改組態檔
[root@tcloud hadoop]# vim ./yarn-site.xml
# 啟動yarn
[root@tcloud hadoop]# start-yarn.sh
<!-- 調整記憶體大小即可 -->
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>256</value>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>1540</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>1540</value>
</property>
未解決的問題 但不影響執行
2021-08-16 15:10:55,499 INFO conf.Configuration: resource-types.xml not found
2021-08-16 15:10:55,500 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/294655.html
標籤：其他