大家按照之前的鏈接進行安裝好之后進入到這里安裝高可用
https://blog.csdn.net/weixin_45955039/article/details/118784204?spm=1001.2014.3001.5502
1.進入到hadoop的組態檔中
cd /usr/local/hadoop/etc/hadoop/
2.配置core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
<!-- hdfs 的資料存盤位置 -->
<property>
<name>hadoop.tmp.dir</name>
<value>file:/usr/local/hadoop/tmp</value>
</property>
<!-- hdfs 的訪問地址,ns 是 NameNode 集群的名字,在 hdfs-site.xml 檔案配置 -->
<property>
<name>fs.defaultFS</name>
<value>hdfs://ns</value>
</property>
<!-- 允許 hive 中的用戶操作操縱 hdfs -->
<property>
<name>hadoop.proxyuser.hadoop.groups</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hadoop.hosts</name>
<value>*</value>
</property>
<!-- 配置 zookeeper 地址,JournalNode 通過 zookeeper 實作功能 -->
<property>
<name>ha.zookeeper.quorum</name>
<value>master:2181,slave1:2181,slave2:2181</value>
</property>
</configuration>
3.配置hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<!-- 設定 NameNode 集群的名字為 ns -->
<property>
<name>dfs.nameservices</name>
<value>ns</value>
</property>
<!-- 為 ns 集群中的每個 NameNode 起一個名字,分別是 nn1, nn2 -->
<property>
<name>dfs.ha.namenodes.ns</name>
<value>nn1,nn2</value>
</property>
<!-- 分別配置每個 NameNode 的請求埠和監控頁面埠 -->
<property>
<name>dfs.namenode.rpc-address.ns.nn1</name>
<value>master:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.ns.nn1</name>
<value>master:50070</value>
</property>
<property>
<name>dfs.namenode.rpc-address.ns.nn2</name>
<value>slave1:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.ns.nn2</name>
<value>slave1:50070</value>
</property>
<!-- 在哪些節點啟動 JournalNode 行程,用于在兩個 NameNode 之間同步 fsimage 和 edits,通常是單數個 -->
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://master:8485;slave1:8485;slave2:8485/ns</value>
</property>
<!-- JournalNode 行程資料的存盤位置 -->
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/usr/local/hadoop/tmp/journal</value>
</property>
<!-- NameNode 行程資料的存盤位置 -->
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/usr/local/hadoop/tmp/dfs/name</value>
</property>
<!-- DataNode 行程資料的存盤位置 -->
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/usr/local/hadoop/tmp/dfs/data</value>
</property>
<!-- 啟用自動切換 -->
<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
<!-- 配置切換 Active 和 StrandBy 狀態的類 -->
<property>
<name>dfs.client.failover.proxy.provider.ns</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<!-- 使用 fence 軟體進行切換 -->
<property>
<name>dfs.ha.fencing.methods</name>
<value>sshfence</value>
</property>
<!-- fence 使用需要指定私鑰的地址 -->
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/home/hadoop/.ssh/id_rsa</value>
</property>
<!-- NameNode 等待 JournalNode 啟動的超時時間 -->
<property>
<name>dfs.qjournal.write-txns.timeout.ms</name>
<value>60000</value>
</property>
</configuration>
4.將修改好的檔案發送給另外兩臺虛擬機
sudo scp -r /usr/local/hadoop/etc/hadoop slave1:`pwd`
sudo scp -r /usr/local/hadoop/etc/hadoop slave2:`pwd`
5.啟動zookeeper
zkServer.sh start
6,查看zookeeper狀態
zkServer.sh status

7. 3個節點都使用以下命令啟動JournalNode
hadoop-daemon.sh start journalnode

8.在master上格式化NameNode,在master使用以下命令:
hdfs namenode -format

9.啟動master上的NameNode,在master使用以下命令:
hadoop-daemon.sh start namenode
10.同步master上NameNode的資料到slave1,在slave1使用以下命令:
hdfs namenode -bootstrapStandby
11.關閉master上的NameNode,在master使用以下命令:
hadoop-daemon.sh stop namenode
12. 在master初始化Zookeeper監控工具,在master使用以下命令:
hdfs zkfc -formatZK
13.在master啟動hdfs,yarn
#關閉hdfs,yarn
sbin/stop-dfs.sh
sbin/stop-yarn.sh
#啟動hdfs,yarn
sbin/start-dfs.sh
sbin/start-yarn.sh
#啟動所有
sbin/start-all.sh
#關閉所有
sbin/stop-all.sh

轉載請註明出處,本文鏈接:https://www.uj5u.com/qita/438076.html
標籤:其他
上一篇:Flink同步Mysql資料到Hive(不開啟Binlog)
下一篇:nacos的一致性協議介紹
