CentOS7 離線環境使用ansible自動部署CDH5.16

前言

本文介紹如何使用作者開發的自動化腳本，離線部署cdh集群，只需要簡單的配置下yum源和cdh集群節點IP等幾個引數，就可實作一鍵部署cdh集群，省去配置mysql、ntp服務、主機配置、cdh檔案分發等繁雜操作，安裝程序快速便捷，我自己測驗，三個節點的集群，cdh安裝不超過15分鐘，

注意： 主機配置邏輯存盤卷并不在自動化范圍內，如您已配置邏輯卷或不需要邏輯卷，請繼續往下，（建議先做邏輯卷，以后好擴容存盤，以后有時間，我會把配置邏輯卷也加到腳本里，筆者使用的磁盤由ceph的塊存盤服務提供，可自由擴容，）

簡介

整個cdh集群的部署由anible-palybook的劇本完成，其中mysql采用docker的部署方式，ntp服務使用chrony搭建，

下載安裝包

下載地址：https://pan.baidu.com/s/1yosjmPLZHngL1QFbxV095g
提取碼：w4uf
檔案大小：3.74GB
安裝包包含軟體：cdh5.16、ansible2.9.21、docker20.10.7、chrony、mysql5.7、vim等基礎工具包以及作者開發的相關自動化腳本，

安裝并配置ansible

將安裝包發送至目標主機（scm-server節點）/root目錄下，并解壓

[root@cdh-auto-deploy-test-1 ~]# tar -xvf cdh.5.16.tar
[root@cdh-auto-deploy-test-1 ~]# ll cdh5.16
total 2984412
-rw-r--r--. 1 root root 2127506677 Jun  9 10:14 CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel
-rw-r--r--. 1 root root         41 Jun  9 10:13 CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha
-rw-r--r--. 1 root root  841524318 Jun  9 10:14 cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz
drwxr-xr-x. 4 root root        153 Jun 25 15:17 deployfiles
-rw-r--r--. 1 root root       5670 Jun  9 10:14 KAFKA-1.2.0.jar
-rw-r--r--. 1 root root   85897902 Jun  9 10:14 KAFKA-4.0.0-1.4.0.0.p0.1-el7.parcel
-rw-r--r--. 1 root root         41 Jun  9 10:14 KAFKA-4.0.0-1.4.0.0.p0.1-el7.parcel.sha
-rw-r--r--. 1 root root      66538 Jun  9 10:14 manifest.json
-rw-r--r--. 1 root root       5356 Jun  9 10:14 manifestkafka.json
-rw-r--r--. 1 root root    1007502 Jun  9 10:14 mysql-connector-java-5.1.47.jar

修改yum源

[root@cdh-auto-deploy-test-1 ~]# mkdir /etc/yum.repos.d/back
[root@cdh-auto-deploy-test-1 ~]# mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/back/
[root@cdh-auto-deploy-test-1 ~]# cp /root/cdh5.16/deployfiles/yumPackages/software.repo /etc/yum.repos.d/
[root@cdh-auto-deploy-test-1 ~]# vi /etc/yum.repos.d/software.repo

[software]
name=software
## 修改`baseurl`的值，使其指向軟體包，
baseurl=file:///root/cdh5.16/deployfiles/yumPackages/rpmPackages/
enabled=1
gpgcheck=0
[vim]
name=vim
## 修改`baseurl`的值，使其指向解壓出的檔案，
baseurl=file:///root/cdh5.16/deployfiles/yumPackages/vim/
enabled=1
gpgcheck=0

使用yum安裝ansible

[root@cdh-auto-deploy-test-1 ~]# yum install -y ansible vim perl 
[root@cdh-auto-deploy-test-1 ~]# ansible --version
ansible 2.9.21
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Apr  9 2019, 14:30:50) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
[root@cdh-auto-deploy-test-1 ~]#

修改ansible.cfg,關閉ssh密鑰檢查，

[root@cdh-auto-deploy-test-1 ~]# vim /etc/ansible/ansible.cfg

 host_key_checking = False

修改ansible的hosts檔案，配置ansible主機,替換主機IP和密碼，

[root@cdh-auto-deploy-test-1 ~]# vim /etc/ansible/hosts

[scm_server]
10.0.5.77 ansible_host=10.0.5.77 hostname=cdh1 ansible_user=root ansible_ssh_pass=12345 ansible_connection=local

[scm_agent]
10.0.5.74 ansible_host=10.0.5.74 hostname=cdh2 ansible_user=root ansible_ssh_pass=12345

[cdh:children]
scm_server
scm_agent

[db:children]
scm_server

測驗主機通信，ping所有節點：

[root@cdh-auto-deploy-test-1 ~]# ansible all -m ping
cdh1 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": false,
    "ping": "pong"
}
cdh2 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": false,
    "ping": "pong"
}
[root@cdhdeploytest-2 yumPackages]#

安裝cdh5.16

修改/root/cdh5.16/deployfiles/vars.yaml，配置引數：

##變數存盤檔案
#

#vm_type代表ansible節點的虛擬機型別，
#當前版本有效值為“centos7”，
vm_type: "centos7"

#當有可用的ntp服務器時填寫服務器的ip,沒有不填，不填（為空）時，將同步scm-server所在節點的時間同步至其它節點，
ntp_server:

#安裝包(解壓后)所在目錄，
cdh_packages_dir: "/root"

#資料盤目錄，docker的持久卷、cdh的scm等將存放在此目錄下，
cdh_data_dir: "/opt"

使用ansible-playbook自動安裝cdh5.16

[root@cdh-auto-deploy-test-1 ~]# ansible-playbook /root/cdh5.16/deployfiles/deploy-cdh.yaml

注意：
　　/root/cdh5.16/deployfiles/deploy-cdh.yaml是自動化部署cdh的ansible腳本，
　　可以自行修改此腳本，定制部署自己的cdh，
　　mysql用戶密碼、docker卷目錄都在此檔案中修改，

當腳本結束后，執行 tail -200f /opt/cloudera-manager/cm-5.16.1/log/cloudera-scm-server/cloudera-scm-server.log查看scn-server的日志,大約幾分鐘后出現如下內容表示sever啟動完成：

Started SelectChannelConnector@0.0.0.0:7180
Started Jetty server.
ScmActive completed successfully.
Discovered parcel on CM server: CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel
Created torrent file: /opt/cloudera/parcel-repo/CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.torrent
 Creating single-file torrent for CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel...
 Hashing data from CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel with 4 threads (4058 pieces)...
   ... 10% complete
   ... 20% complete
   ... 30% complete
   ... 40% complete
   ... 50% complete
   ... 60% complete
   ... 70% complete
   ... 80% complete
   ... 90% complete
 Hashed 1 file(s) (2127506677 bytes) in 4058 pieces (4058 expected) in 6605.6ms.
 Single-file torrent information:
   Torrent name: CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel
   Announced at: Seems to be trackerless
   Created on..: Fri Jun 25 16:49:59 CST 2021
   Created by..: cm-server
   Pieces......: 4058 piece(s) (524288 byte(s)/piece)
   Total size..: 2,127,506,677 byte(s)
calParcelManagerImpl: Discovered parcel on CM server: KAFKA-4.0.0-1.4.0.0.p0.1-el7.parcel
calParcelManagerImpl: Created torrent file: /opt/cloudera/parcel-repo/KAFKA-4.0.0-1.4.0.0.p0.1-el7.parcel.torrent
 Creating single-file torrent for KAFKA-4.0.0-1.4.0.0.p0.1-el7.parcel...
 Hashing data from KAFKA-4.0.0-1.4.0.0.p0.1-el7.parcel with 4 threads (164 pieces)...
   ... 10% complete
   ... 20% complete
   ... 30% complete
   ... 40% complete
   ... 50% complete
   ... 60% complete
   ... 70% complete
   ... 80% complete
   ... 90% complete
 Hashed 1 file(s) (85897902 bytes) in 164 pieces (164 expected) in 277.9ms.
 Single-file torrent information:
   Torrent name: KAFKA-4.0.0-1.4.0.0.p0.1-el7.parcel
   Announced at: Seems to be trackerless
   Created on..: Fri Jun 25 16:50:06 CST 2021
   Created by..: cm-server
   Pieces......: 164 piece(s) (524288 byte(s)/piece)
   Total size..: 85,897,902 byte(s)

現在可以登錄cdh1:7180部署集群了，用戶：admin 密碼：admin

備注

/opt/cloudera-manager/cm-5.16.1/etc/init.d/cloudera-scm-agent status 查看agent狀態
/opt/cloudera-manager/cm-5.16.1/etc/init.d/cloudera-scm-server status 查看server狀態