一、Elasticsearch 集群

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.0-linux-x86_64.tar.gz

https://artifacts.elastic.co/downloads/kibana/kibana-7.10.2-linux-x86_64.tar.gz

https://artifacts.elastic.co/downloads/logstash/logstash-7.11.1-linux-x86_64.tar.gz

https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.10.2-linux-x86_64.tar.gz

目錄結構

[root@vm1 playbook]# tree elk
elk
├── deploy-elasticsearch.yml
├── elasticsearch.j2
├── elastic-vars.yml
├── elk-pkg
│   ├── elasticsearch-7.10.0-linux-x86_64.tar.gz
│   ├── filebeat-7.10.0-linux-x86_64.tar.gz
│   ├── kibana-7.10.0-linux-x86_64.tar.gz
│   └── logstash-7.10.0-linux-x86_64.tar.gz
├── jvm.options
├── limits.conf
└── sysctl.conf

1 directory, 10 files

elk/deploy-elasticsearch.yml

---
- name: 部署 elasticsearch 集群
  hosts: es
  gather_facts: no
  remote_user: root
  vars_files:
    - elastic-vars.yml
  tasks:
    - name: 創建用戶
      user:
        name: "{{ ela_user }}"
      tags: deploy

    - name: 傳輸本地軟體包到遠程主機并且解壓到指定目錄
      ansible.builtin.unarchive:
        src: "{{ ela }}"
        dest: /usr/local/
        owner: "{{ ela_user }}"
        group: "{{ ela_user }}"
        list_files: yes
      register: ret
      tags: deploy

    - name: 創建軟鏈接
      ansible.builtin.file:
        src: /usr/local/{{ ret.files.0 | regex_replace('/.*') }}
        dest: /usr/local/elasticsearch
        state: link
      tags: deploy

    - name: 傳輸組態檔
      template:
        src: elasticsearch.j2
        dest: /usr/local/elasticsearch/config/elasticsearch.yml
        owner: "{{ ela_user }}"
        group: "{{ ela_user }}"
      tags:
        - deploy
        - update-conf

    - name: 傳輸 jvm 組態檔
      copy:
        src: jvm.options
        dest: /usr/local/elasticsearch/config/jvm.options
        owner: "{{ ela_user }}"
        group: "{{ ela_user }}"
      tags:
        - deploy
        - update-conf

    - name: 傳輸系統組態檔
      copy:
        src: limits.conf
        dest: /etc/security/limits.conf
      tags: deploy

    - name: 傳輸系統組態檔
      copy:
        src: sysctl.conf
        dest: /etc/sysctl.conf
      tags: deploy

    - name: 加載 /etc/sysctl.conf  檔案,使內核引數生效
      shell: sysctl -p
      tags: deploy

    - name: 檢查 行程 PID
      shell:
        cmd: ./jps | grep [E]lasticsearch | cut -d ' ' -f 1
        chdir: /usr/local/elasticsearch/jdk/bin/
      register: ela_pid
      tags:
        - restart
        - update-conf

    #- name: 除錯
    #  debug: var=ela_pid

    - name: 停止 elasticsearch
      when: ela_pid.stdout
      shell: kill -9 "{{ ela_pid.stdout }}"
      tags:
        - restart
        - update-conf

    - name: 啟動服務
      # 使用 elastic 用戶執行此命令
      become: yes
      become_user: "{{ ela_user }}"
      command:
        # argv 是一個串列，存放了需要執行的命令及其引數
        # 一行一個
        argv:
          - nohup
          - /usr/local/elasticsearch/bin/elasticsearch
          - -p
          - /tmp/elasticsearch.pid
          - -d
      tags:
        - deploy
        - restart
        - update-conf
...

playbook 使用的變數檔案內容如下：

[root@vm1 playbook]# cat elk/elastic-vars.yml
# elasticsearch 用戶
ela_user: elastic

# 程式的二進制包名稱
ela: elk-pkg/elasticsearch-7.10.0-linux-x86_64.tar.gz
logstash: elk-pkg/logstash-7.10.0-linux-x86_64.tar.gz
kibana: elk-pkg/kibana-7.10.0-linux-x86_64.tar.gz
filebeat: eli-pkg/filebeat-7.10.0-linux-x86_64.tar.gz

playbook 中使用到的模板檔案內容如下：

[root@vm1 playbook]# cat elk/elasticsearch.j2
cluster.name: elk
node.name: "{{ node_name }}"
node.data: true
network.host: 0.0.0.0
http.port: 9200

discovery.seed_hosts:
   - es01
   - es02:9300
   - es03
cluster.initial_master_nodes: "{{ es_nodes }}"

上面的模板檔案在 資產清單檔案中添加如下變數

[es]
# 集群節點名稱
es01 node_name=ela1
es02 node_name=ela2
es03 node_name=ela3

[es:vars]
# 用于參加選舉的節點名串列
es_nodes=["ela1", "ela2", "ela3"]

ES 集群使用的 JVM 組態檔

[root@vm1 playbook]# cat elk/jvm.options
-Xms1g
-Xmx1g
8-13:-XX:+UseConcMarkSweepGC
8-13:-XX:CMSInitiatingOccupancyFraction=75
8-13:-XX:+UseCMSInitiatingOccupancyOnly
14-:-XX:+UseG1GC
14-:-XX:G1ReservePercent=25
14-:-XX:InitiatingHeapOccupancyPercent=30
-Djava.io.tmpdir=${ES_TMPDIR}
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=data
-XX:ErrorFile=logs/hs_err_pid%p.log
8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:logs/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m
9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m

系統檔案句柄組態檔 elk/limits.conf 內容如下：

root soft nofile 65535
root hard nofile 65535
* soft nofile 65535
* hard nofile 65535
* soft nproc  4096
* hard nproc 4096

系統的內核組態檔 elk/sysctl.conf 內容如下

只有 # elasticsearch 下面的配置是必須的，其他可選

vm.swappiness = 0
kernel.sysrq = 1

net.ipv4.neigh.default.gc_stale_time = 120

# see details in https://help.aliyun.com/knowledge_detail/39428.html
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_announce = 2

# see details in https://help.aliyun.com/knowledge_detail/41334.html
net.ipv4.tcp_max_tw_buckets = 5000
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 1024
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_slow_start_after_idle = 0

# elasticsearch
net.ipv4.tcp_retries2=5
vm.max_map_count=262144