HadoopHA 搭建

1.準備三臺機器 centos7

安裝步驟 略（ 可以看我的 虛擬機安裝 centos7 文章）

2.配置免秘鑰訪問

略（可以看我的免秘鑰配置文章）

3.JKD 安裝1.8

略

4.Hadoop官網下載3.1.2

網址 http://hadoop.apache.org/
https://archive.apache.org/dist/hadoop/common/hadoop-3.1.2/

5.解壓

 tar -zxvf hadoop-3.1.2.tar.gz
 mv hadoop-3.1.2 /opt/test/
 cd /opt/test/hadoop-3.1.2/etc/hadoop/

6.修改環境變數

• vi hadoop-env.sh

export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_111
export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_ZKFC_USER=root
export HDFS_JOURNALNODE_USER=root
export YARN_RESOURCEMANAGER_USER=root
export YARN_NODEMANAGER_USER=root

7.修改組態檔

• vi core-site.xml

<property>
<name>fs.defaultFS</name>
<value>hdfs://hdfs-test</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/var/test/hadoop/ha</value>
</property>
<property>
<name>hadoop.http.staticuser.user</name>
<value>root</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value>10.1.1.201:2181,10.1.1.158:2181,10.1.1.185:2181</value>
</property>

• vi hdfs-site.xml

<property>
<name>dfs.nameservices</name>
<value>hdfs-test</value>
</property>
<property>
<name>dfs.ha.namenodes.hdfs-test</name>
<value>nn1,nn2</value>
</property>
<property>
<name>dfs.namenode.rpc-address.hdfs-test.nn1</name>
<value>10.1.1.58:8020</value>
</property>
<property>
<name>dfs.namenode.rpc-address.hdfs-test.nn2</name>
<value>10.1.1.152:8020</value>
</property>
<property>
<name>dfs.namenode.http-address.hdfs-test.nn1</name>
<value>10.1.1.58:9870</value>
</property>
<property>
<name>dfs.namenode.http-address.hdfs-test.nn2</name>
<value>10.1.1.152:9870</value>
</property>
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://10.1.1.58:8485;10.1.1.152:8485;10.1.1.195:8485/hdfs-test</value>
</property>
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/var/test/hadoop/ha/qjm</value>
</property>
<property>
<name>dfs.client.failover.proxy.provider.hdfs-test</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxy
Provider</value>
</property>
<property>
<name>dfs.ha.fencing.methods</name>
<value>sshfence</value>
<value>shell(true)</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/root/.ssh/id_rsa</value>
</property>
<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>

-vi workers

10.1.1.58
10.1.1.152
10.1.1.195

8.分發

scp -r hadoop-3.1.2 root@10.1.1.152:/opt/test
scp -r hadoop-3.1.2 root@10.1.1.195:/opt/test

9.修改環境變數

• vi /etc/profile

export HADOOP_HOME=/opt/test/hadoop-3.1.2
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH

分發

• scp /etc/profile root@10.1.1.152:/etc/profile

• scp /etc/profile root@10.1.1.195:/etc/profile

• source /etc/profile

10.首先啟動Zookeeper

三節點都啟動

[root@node01 ]#zkServer.sh start
[root@node02 ]#zkServer.sh start
[root@node03 ]#zkServer.sh start
zkServer.sh status

11.啟動JournalNode

三節點都啟動

[root@node01 ]# hdfs --daemon start journalnode
[root@node02 ]# hdfs --daemon start journalnode
[root@node03 ]# hdfs --daemon start journalnode

12.格式化NameNode

[root@node01 ]# hdfs namenode -format
[root@node01 ]# hdfs --daemon start namenode
[root@node02 ]# hdfs namenode -bootstrapStandby
[root@node01 ]# hdfs zkfc -formatZK
[root@node01 ]# start-dfs.sh

13.測驗

http://10.1.1.58:9870
http://10.1.1.152:9870