綜合案例：使用Configmap-reload動態熱加載Configmap中的組態檔

1.configmap-reload資源熱更新機制

在k8s集群中，當configmap以volume的形式掛載到pod內時，更新configmap，k8s會自動將更改的組態檔內容同步到pod掛載的檔案中，這個并不是立刻生效的，大約需要1分鐘左右，實際案例中，如果應用程式支持熱更新功能，所謂熱更新就是通過http介面的方式就可以更新程式的配置，如果支持熱更新機制就可以使用configmap-reload監控組態檔的變更，當組態檔變更后呼叫程式介面自動將應用程式進行更新

以prometheus為例配置當promet.yaml更新后立即進行熱更新

注意：只有支持熱更新的程式才能進行configmap-reload配置

整體實作思路：

? 1.首先準備一個configmap資源，用于保存prometheus的組態檔

? 2.準備一套pv、pvc，用于持久化prometheus監控系統的資料

? 3.準備一個statfulset控制器跑prometheus pod，configmap-reload必須依靠固定的主機名，因此需要采用statfulset資源

? 4.準備svc資源，用于在外部訪問prometheus

? 5.在所有k8s節點上部署node_exporter，用于prometheus采集資料

? 6.編輯configmap資源增加k8s節點的配置內容，觀察configmap-reload是否會自動更新

2.使用configmap-reload實作prometheus監控系統配置熱更新

2.1.準備prometheus的configmap資源

將prometheus組態檔以configmap的方式掛載到容器中

1.撰寫yaml
[root@k8s-master ~/k8s_1.19_yaml/configmap]# vim prometheus-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-configmap
  namespace: prometheus
data:
  prometheus.yaml: |
    global:
      scrape_interval:     90s
      evaluation_interval: 90s
    scrape_configs:
    - job_name: prometheus
      static_configs:
      - targets:
        - 127.0.0.1:9090

2.創建yaml
[root@k8s-master ~/k8s_1.19_yaml/configmap]# kubectl create -f prometheus-configmap.yaml
configmap/prometheus-configmap created

3.查看資源狀態
[root@k8s-master ~/k8s_1.19_yaml/configmap]# kubectl get cm -n prometheus
NAME                   DATA   AGE
prometheus-configmap   1      4m56s

4.查看資源的詳細資訊
[root@k8s-master ~/k8s_1.19_yaml/configmap]# kubectl describe cm -n prometheus
Name:         prometheus-configmap
Namespace:    prometheus
Labels:       <none>
Annotations:  <none>

Data
====
prometheus.yaml:
----
global:
  scrape_interval:     90s
  evaluation_interval: 90s
scrape_configs:
- job_name: prometheus
  static_configs:
  - targets:
    - 127.0.0.1:9090

Events:  <none>

2.2.準備prometheus pv、pvc資源

為prometheus資料提供持久化存盤

1.撰寫yaml
[root@k8s-master ~/k8s_1.19_yaml/configmap]# vim prometheus-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-2
  labels:
    pv: pv-2
spec:
  capacity:
    storage: 5Gi
  accessModes:
  - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  nfs:												#以nfs型別存盤
    path: /data/pv_2/prometheus
    server: 192.168.81.210

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-2
  namespace: prometheus
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 5Gi
  selector:
    matchLabels:
      pv: pv-2							#通過標簽方式管理pv-2的pv存盤

2.創建資源
[root@k8s-master ~/k8s_1.19_yaml/configmap]# kubectl create -f prometheus-pv.yaml
persistentvolume/pv-2 created
persistentvolumeclaim/pvc-2 created

2.3.撰寫prometheus statfulset、svc資源

使用statefulset資源控制器部署prometheus容器并使用svc進行暴露，由于configmap-reload必須要配合固定的主機名進行使用，因此采用statefulset控制器

1.撰寫statfulset控制器yaml檔案
[root@k8s-master ~/k8s_1.19_yaml/configmap]# vim prometheus-statfulset.yaml 
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: statefulset-prometheus
  namespace: prometheus
spec:
  serviceName: "prometheus"
  replicas: 1
  podManagementPolicy: "Parallel"
  updateStrategy:
   type: "RollingUpdate"
  selector:
    matchLabels:
      app: prometheus-pod
  template:
    metadata:
      labels:
        app: prometheus-pod
    spec:
      initContainers:									#定義一個初始化容器，主要是給/data目錄賦權限并掛載pvc
      - name: "init-chown-data"
        image: "busybox:1.30"
        command: ["chown", "-R", "65534:65534", "/data"]
        volumeMounts:
        - name: prometheus-data 
          mountPath: /data
      containers: 
      - name: prometheus
        image: prom/prometheus:v2.23.0
        args:								#定義prometheus啟動引數
          - --config.file=/etc/config/prometheus.yaml
          - --storage.tsdb.path=/data
          - --web.console.libraries=/etc/prometheus/console_libraries
          - --web.console.templates=/etc/prometheus/consoles
          - --web.enable-lifecycle					#啟動prometheus熱更新機制
        ports:
        - containerPort: 9090
        volumeMounts:
        - name: prometheus-data
          mountPath: /data
        - name: prometheus-config
          mountPath: /etc/config
      volumes:
        - name: prometheus-data
          persistentVolumeClaim:
            claimName: pvc-2
            readOnly: false
        - name: prometheus-config
          configMap:
            name: prometheus-configmap

2.撰寫service資源的yaml檔案
[root@k8s-master ~/k8s_1.19_yaml/configmap]# vim prometheus-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: prometheus-service
  namespace: prometheus
spec:
  selector:
    app: prometheus-pod
  type: NodePort
  ports:
  - port: 9090
    targetPort: 9090

3.創建資源
[root@k8s-master ~/k8s_1.19_yaml/configmap]# kubectl create -f prometheus-statfulset.yaml
statefulset.apps/statefulset-prometheus created

[root@k8s-master ~/k8s_1.19_yaml/configmap]# kubectl create -f prometheus-svc.yaml
service/prometheus-service created

2.4.訪問prometheus

1.查看prometheus創建的資源
[root@k8s-master ~/k8s_1.19_yaml/configmap]# kubectl get all -n prometheus 
NAME                           READY   STATUS    RESTARTS   AGE
pod/statefulset-prometheus-0   1/1     Running   1          133m

NAME                         TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
service/prometheus-service   NodePort   10.99.144.79   <none>        9090:30877/TCP   134m

NAME                                      READY   AGE
statefulset.apps/statefulset-prometheus   1/1     134m


2.訪問node節點ip加30877即可訪問prometheus

2.5.配置configmap熱更新

主要用來當configmap資源檔案更新后，prometheus pod會自動加載配置并進行更新

1.修改yaml檔案增加configmap-reload配置
[root@k8s-master ~/k8s_1.19_yaml/configmap]# vim prometheus-statfulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: statefulset-prometheus
  namespace: prometheus
spec:
  serviceName: "prometheus"
  replicas: 1
  podManagementPolicy: "Parallel"
  updateStrategy:
   type: "RollingUpdate"
  selector:
    matchLabels:
      app: prometheus-pod
  template:
    metadata:
      labels:
        app: prometheus-pod
    spec:
      initContainers:
      - name: "init-chown-data"
        image: "busybox:1.30"
        command: ["chown", "-R", "65534:65534", "/data"]
        volumeMounts:
        - name: prometheus-data 
          mountPath: /data
      containers: 
      - name: prometheus-configmap-reload						#增加一個configmap-reload的容器即可
        image: "jimmidyson/configmap-reload:v0.1"				
        imagePullPolicy: "IfNotPresent"
        args: 												#configmap-reload需要增加的引數
          - --volume-dir=/etc/prometheus
          - --webhook-url=http://localhost:9090/-/reload 
        volumeMounts:									#將configmap掛載到/etc/prometheus
          - name: prometheus-config
            mountPath: /etc/prometheus
            readOnly: true
      - name: prometheus
        image: prom/prometheus:v2.23.0
        args:
          - --config.file=/etc/config/prometheus.yaml
          - --storage.tsdb.path=/data
          - --web.console.libraries=/etc/prometheus/console_libraries
          - --web.console.templates=/etc/prometheus/consoles
          - --web.enable-lifecycle
        ports:
        - containerPort: 9090
        volumeMounts:
        - name: prometheus-data
          mountPath: /data
        - name: prometheus-config
          mountPath: /etc/config
      volumes:
        - name: prometheus-data
          persistentVolumeClaim:
            claimName: pvc-2
            readOnly: false
        - name: prometheus-config
          configMap:
            name: prometheus-configmap
            
2.更新yaml檔案      
[root@k8s-master ~/k8s_1.19_yaml/configmap]# kubectl apply -f prometheus-statfulset.yaml 
statefulset.apps/statefulset-prometheus configured

2.6.更新configmap組態檔觀察prometheus容器是否會自動更新

在node節點安裝node_exporter監控采集器，并修改prometheus configmap的配置，等待一段時間后觀察prometheus是否會自動更新配置

1.所有master、node節點安裝并啟動node_exporter
tar xf node_exporter-1.0.1.linux-amd64.tar.gz 
cp node_exporter-1.0.1.linux-amd64/node_exporter /usr/bin/
nohup /usr/bin/node_exporter &
netstat -lnpt | grep 9100

2.修改configmap中prometheus組態檔的內容增加node節點的配置
#這里只增加了master和node-1節點
[root@k8s-master ~/k8s_1.19_yaml/configmap]# vim prometheus-configmap.yaml 
apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-configmap
  namespace: prometheus
data:
  prometheus.yaml: |
    global:
      scrape_interval:     90s
      evaluation_interval: 90s
    scrape_configs:
    - job_name: prometheus
      static_configs:
      - targets:
        - 127.0.0.1:9090
    - job_name: k8s-node
      static_configs:
      - targets:
        - 192.168.81.210:9100
        - 192.168.81.220:9100

3.更新configmap資源
[root@k8s-master ~/k8s_1.19_yaml/configmap]# kubectl apply -f prometheus-configmap.yaml 
configmap/prometheus-configmap configured

4.訪問prometheus觀察配置是否更新
直接在頁面上查看prometheus的配置是否增加和targets里面是否有主機增加就可以看到效果了