手把手從零開始搭建k8s集群超詳細教程-有解無憂

本教程根據B站課程云原生Java架構師的第一課K8s+Docker+KubeSphere+DevOps同步所做筆記教程

k8s集群搭建超詳細教程

- 1. 基本環境搭建
- - 1. 創建私有網路
  - 2. 創建服務器資源
  - 3. 遠程連接到服務器
  - 4. docker容器化環境安裝
  - 5. kubeadm、kubectl、kubelet安裝
- 2. 使用kubeadm引導集群
- - 1. 下載k8s所需的鏡像
  - 2. 添加k8s中主節點的域名映射
  - 3. 初始化k8s主節點
  - 4. 添加k8s集群中的從節點
- 3. 驗證集群自動恢復功能
- 4. 部署k8s可視化管理界面——dashboard
- - 1. 下載部署dashboard
  - 2. 設定dashboard訪問埠
  - 3. 創建訪問賬號
  - 4. 獲取訪問令牌

搭建集群架構如下圖所示：一共三臺機器，其中一個master節點，兩個worker作業節點，保證每臺機器間能使用內網ip互通

每臺機器首先安裝docker保證容器運行環境，然后安裝核心的三個部件kubelet、kubectl(命令列工具)、kubeadm(初始化集群工具)

1. 基本環境搭建

以下實驗基于第一家混合云上市公司 | 青云QingCloud完成，為什么要選用青云呢？首先是青云自研了KubeSphere，它是基于 Kubernetes 構建的分布式、多租戶、多集群、企業級開源容器平臺，我們稍后會學習該平臺的使用，其次，在使用的程序中，體會到了青云對于各種資源操作的便捷，且附有各種便于理解的可視化界面，整個控制臺界面簡單高效，

1. 創建私有網路

VPC即Virtual Private Cloud，私有網路，可以理解為一個網段，在這個網段內還可以選擇創建子網段，不同的私有網路內實作完全的隔離，保證資源的封閉性，在公有云上構建出一個專屬隔離的網路環境，在 VPC 網路內，您可以自定義 IP 地址范圍，創建子網，并在子網內創建云服務器、資料庫、大資料等各種云資源，

接下來我們新建一個VPC名為k8s-cluster專門用來存放k8s的集群，并在其中創建一個私有網路k8s-cluster-01

創建完成后如圖所示：

2. 創建服務器資源

準備三臺centos服務器，這里以青云QingCloud的云服務器為例，創建三個centos服務器

注意：kubenetes集群安裝要求每臺機器記憶體 >= 2 GB、核心數 >= 2 CPU

選擇按需付費，其中：網路加入到我們自己創建的VPC私有網路k8s-cluster-01中，且每臺服務器新建對應的公網ip，選擇按流量付費

創建完成后可以在VPC私有網路中看到新建的3臺服務器：

注意打開安全組的組內互信，就是保證同一個局域網內的所有機器不受防火墻的限制都可以互相訪問

3. 遠程連接到服務器

利用遠程連接工具連接到3個服務器，其中k8s-01我們作為集群中的主節點

用ip a命令可以查看每個服務器的內網IP，保證3臺服務器間能使用內網ip相互ping通

4. docker容器化環境安裝

首先給每臺服務器安裝docker

# 1.移除以前docker相關包
sudo yum remove docker \
                  docker-client \
                  docker-client-latest \
                  docker-common \
                  docker-latest \
                  docker-latest-logrotate \
                  docker-logrotate \
                  docker-engine

# 2. 配置yum源
sudo yum install -y yum-utils
sudo yum-config-manager \
--add-repo \
http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

# 3. 安裝docker
sudo yum install -y docker-ce docker-ce-cli containerd.io

# 4. 啟動docker
systemctl enable docker --now

# 5. 配置阿里云加速
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://82m9ar63.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

5. kubeadm、kubectl、kubelet安裝

kubernetes集群安裝環境要求：

一臺兼容的 Linux 主機，Kubernetes 專案為基于 Debian 和 Red Hat 的 Linux 發行版以及一些不提供包管理器的發行版提供通用的指令

每臺機器記憶體 >= 2 GB、核心數 >= 2 CPU

設定防火墻放行規則，保證集群中的所有機器的網路彼此均能相互連接（公網和內網都可以）

給每臺機器設定不同hostname，要求節點之間不可以有重復的主機名、MAC 地址或 product_uuid（點擊這里了解更多詳細資訊）

開啟機器上的某些埠（詳細埠資訊點擊這里）

禁用交換磁區，為了保證 kubelet 正常作業，你必須禁用交換磁區

1?? 基本要求完善

在三臺云主機上分別執行以下命令，來保證安裝kubernetes集群的基本要求

# 設定每個機器自己的hostname（這里分別為k8s-master、k8s-node1、k8s-node2）
hostnamectl set-hostname 主機名

# 禁用SELinux安全子系統（將SELinux設定為permissive模式）
sudo setenforce 0 # 臨時
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config # 永久

# 禁用虛擬記憶體（關閉swap）
swapoff -a  
sed -ri 's/.*swap.*/#&/' /etc/fstab

# 允許iptables檢查橋接流量
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

# 使配置生效
sudo sysctl --system

2?? 安裝kubelet、kubeadm、kubectl

在三臺云主機上分別執行以下命令安裝 kubelet、kubeadm、kubectl

# 配置k8s的yum源地址
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
   http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

# 安裝 kubelet，kubeadm，kubectl
sudo yum install -y kubelet-1.20.9 kubeadm-1.20.9 kubectl-1.20.9

# 啟動kubelet
sudo systemctl enable --now kubelet

# 所有機器配置master域名
echo "172.31.0.4  k8s-master" >> /etc/hosts

2. 使用kubeadm引導集群

1. 下載k8s所需的鏡像

在三臺云服務器上執行以下命令，命令中撰寫了一個shell腳本然后執行來幫我們下載安裝k8s集群所需的相關鏡像

# 撰寫shell檔案
sudo tee ./images.sh <<-'EOF'
#!/bin/bash
images=(
kube-apiserver:v1.20.9
kube-proxy:v1.20.9
kube-controller-manager:v1.20.9
kube-scheduler:v1.20.9
coredns:1.7.0
etcd:3.4.13-0
pause:3.2
)
for imageName in ${images[@]} ; do
docker pull registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/$imageName
done
EOF

# 給shell檔案權限并執行
chmod +x ./images.sh && ./images.sh

2. 添加k8s中主節點的域名映射

在三臺機器上執行以下命令來添加master域名映射，其中的ip需要修改自己要配置的主節點私網ip地址

# 所有機器添加master域名映射，以下ip需要修改自己要配置的主節點私網ip地址
echo "172.31.0.2 cluster-endpoint" >> /etc/hosts

配置完成后我們可以在任意機器ping cluster-endpoint進行測驗，ping通則代表配置成功

# 配置完成后直接ping域名測驗
ping cluster-endpoint

3. 初始化k8s主節點

在需要作為直接點的主機中（這里為k8s-01）執行以下命令，使用kubeadm初始化k8s集群中的主節點

注意：修改–apiserver-advertise-address為自己主機的私網ip地址

# 主節點初始化(只對主節點主機執行)
kubeadm init \
--apiserver-advertise-address=172.31.0.2 \
--control-plane-endpoint=cluster-endpoint \
--image-repository registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images \
--kubernetes-version v1.20.9 \
--service-cidr=10.96.0.0/16 \
--pod-network-cidr=192.168.0.0/16

主節點初始化成功如下圖所示：

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:

  kubeadm join cluster-endpoint:6443 --token ut0k7e.j286ljqnnaz8v2dp \
    --discovery-token-ca-cert-hash sha256:71dd29dbcc8438caf523df03c6623bac89df35e958cb0adca0f9d400abe8ca7b \
    --control-plane 

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join cluster-endpoint:6443 --token ut0k7e.j286ljqnnaz8v2dp \
    --discovery-token-ca-cert-hash sha256:71dd29dbcc8438caf523df03c6623bac89df35e958cb0adca0f9d400abe8ca7b

在這里插入圖片描述

其中有進一步的操作提示：

1?? 設定.kube/config

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

2?? 安裝網路組件

calico官網

# 下載calico組態檔
curl https://docs.projectcalico.org/manifests/calico.yaml -O

# 應用calico組件
kubectl apply -f calico.yaml

4. 添加k8s集群中的從節點

在其他兩臺云主機k8s-02、k8s-03上分別執行上述初始化主節點完后的提示命令加入到k8s-master的集群中

kubeadm join cluster-endpoint:6443 --token x5g4uy.wpjjdbgra92s25pp \
	--discovery-token-ca-cert-hash sha256:6255797916eaee52bf9dda9429db616fcd828436708345a308f4b917d3457a22

# 注意：該命令24小時過期，過期后可以通過如下命令生成新的命令
kubeadm token create --print-join-com

然后我們在master上查看部署的所有應用，可以發現兩個節點已經加入

過一段時間待節點初始化完成后即可編程ready狀態

3. 驗證集群自動恢復功能

kubenetes集群有自動恢復功能，如果我們在青云控制臺上將三臺云主機關機重啟，k8s應用仍然會自動恢復，可通過以下命令來驗證

# 查看集群所有節點
kubectl get nodes

# 查看集群部署了哪些應用？類似docker ps(運行中的應用在docker里面叫容器，在k8s里面叫Pod)
kubectl get pods -A

4. 部署k8s可視化管理界面——dashboard

dashboard 是kubernetes官方提供的k8s控制臺可視化界面

1. 下載部署dashboard

k8s中下載創建應用可以采用yaml組態檔的方式，使用以下命令即可創建資源

# 根據組態檔，給集群創建資源
kubectl apply -f xxxx.yaml

接下來我們以組態檔的方式安裝dashboard可視化界面

# 在主節點執行以下命令安裝dashboard
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.3.1/aio/deploy/recommended.yaml

如果下載不下來，則可以創建復制以下組態檔并通過kubectl apply -f 組態檔名命令配置應用

# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v1
kind: Namespace
metadata:
  name: kubernetes-dashboard

---

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 443
      targetPort: 8443
  selector:
    k8s-app: kubernetes-dashboard

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-certs
  namespace: kubernetes-dashboard
type: Opaque

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-csrf
  namespace: kubernetes-dashboard
type: Opaque
data:
  csrf: ""

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-key-holder
  namespace: kubernetes-dashboard
type: Opaque

---

kind: ConfigMap
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-settings
  namespace: kubernetes-dashboard

---

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
rules:
  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
    verbs: ["get", "update", "delete"]
    # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["kubernetes-dashboard-settings"]
    verbs: ["get", "update"]
    # Allow Dashboard to get metrics.
  - apiGroups: [""]
    resources: ["services"]
    resourceNames: ["heapster", "dashboard-metrics-scraper"]
    verbs: ["proxy"]
  - apiGroups: [""]
    resources: ["services/proxy"]
    resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
    verbs: ["get"]

---

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
rules:
  # Allow Metrics Scraper to get metrics from the Metrics server
  - apiGroups: ["metrics.k8s.io"]
    resources: ["pods", "nodes"]
    verbs: ["get", "list", "watch"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
    spec:
      containers:
        - name: kubernetes-dashboard
          image: kubernetesui/dashboard:v2.3.1
          imagePullPolicy: Always
          ports:
            - containerPort: 8443
              protocol: TCP
          args:
            - --auto-generate-certificates
            - --namespace=kubernetes-dashboard
            # Uncomment the following line to manually specify Kubernetes API server Host
            # If not specified, Dashboard will attempt to auto discover the API server and connect
            # to it. Uncomment only if the default does not work.
            # - --apiserver-host=http://my-address:port
          volumeMounts:
            - name: kubernetes-dashboard-certs
              mountPath: /certs
              # Create on-disk volume to store exec logs
            - mountPath: /tmp
              name: tmp-volume
          livenessProbe:
            httpGet:
              scheme: HTTPS
              path: /
              port: 8443
            initialDelaySeconds: 30
            timeoutSeconds: 30
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1001
            runAsGroup: 2001
      volumes:
        - name: kubernetes-dashboard-certs
          secret:
            secretName: kubernetes-dashboard-certs
        - name: tmp-volume
          emptyDir: {}
      serviceAccountName: kubernetes-dashboard
      nodeSelector:
        "kubernetes.io/os": linux
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 8000
      targetPort: 8000
  selector:
    k8s-app: dashboard-metrics-scraper

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: dashboard-metrics-scraper
  template:
    metadata:
      labels:
        k8s-app: dashboard-metrics-scraper
      annotations:
        seccomp.security.alpha.kubernetes.io/pod: 'runtime/default'
    spec:
      containers:
        - name: dashboard-metrics-scraper
          image: kubernetesui/metrics-scraper:v1.0.6
          ports:
            - containerPort: 8000
              protocol: TCP
          livenessProbe:
            httpGet:
              scheme: HTTP
              path: /
              port: 8000
            initialDelaySeconds: 30
            timeoutSeconds: 30
          volumeMounts:
          - mountPath: /tmp
            name: tmp-volume
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1001
            runAsGroup: 2001
      serviceAccountName: kubernetes-dashboard
      nodeSelector:
        "kubernetes.io/os": linux
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
      volumes:
        - name: tmp-volume
          emptyDir: {}

配置成功如下圖所示：

2. 設定dashboard訪問埠

# 1.運行以下命令將dashboard web界面的埠暴露到機器上
# 注意將檔案中的 type: ClusterIP 改為 type: NodePort
kubectl edit svc kubernetes-dashboard -n kubernetes-dashboard

注意將檔案中的 type: ClusterIP 改為 type: NodePort，這里我們只需要知道NodePort表示暴露埠可以使用公網訪問，具體原因后續會介紹

# 2.找到埠，在安全組放行
kubectl get svc -A |grep kubernetes-dashboard

這里為31372埠，然后在青云安全組設定中開放該埠

然后我們使用集群中任意一臺機器的公網IP加上該埠號即可訪問，注意帶上https前綴

注意：如果出現不安全不能繼續前往的情況，直接在頁面輸入thisisunsafe，直接在頁面輸入不需要在地址欄輸入即可自動跳轉

3. 創建訪問賬號

# 1. 創建訪問賬號，準備一個yaml檔案
vim dash-user.yaml

# 檔案內容如下
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard
  
# 2. 然后應用該配置
kubectl apply -f dash-user.yaml

4. 獲取訪問令牌

# 獲取訪問令牌
kubectl -n kubernetes-dashboard get secret $(kubectl -n kubernetes-dashboard get sa/admin-user -o jsonpath="{.secrets[0].name}") -o go-template="{{.data.token | base64decode}}"

eyJhbGciOiJSUzI1NiIsImtpZCI6IjNzY2VQeHZORGhjMENSeGd1dFBTVENQYjZLd0hxY1NwSDJ4cDkxUUFMM00ifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLWR3c3RzIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIwYTVmYjFjNi1iNGE5LTQ1YjQtYWQyNy00YzcwMTMyZGMwY2EiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZXJuZXRlcy1kYXNoYm9hcmQ6YWRtaW4tdXNlciJ9.t2Pp1BIr3kU1h7QBHlFOuQp7VhhZhF64V74DYfbp2tP4HR8nt4ph7PphnxBziNS4PCsCDDqh2l1LJftYBTLqtX52e_PNqO6m_uNUpO5WGm7v9SGDttIGimyDNwAKw-qIXzj3BzjEeORfCebgjP6Z9g9pBuVpyQiGNGQ_IoI4WF7B3LlktcZD9QKbhwiL8qOASU3gUP8PuVqz7GmmskFNCHMXQpDNSKumu_0KcVA6qZjEucFz5emkihtDU7fyj2wLZgPJvjbyrDfodD67EYnelkryw6BUqf0TBYfeti5tNgxqbeKgFdKtKB0HQFUn7jDHcG6rrh3mwgpMV7FkohUp8g

然后復制令牌進行登錄，即可進入到管理界面

轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/342333.html

標籤：其他

上一篇：騰訊云服務器CentOS安裝Jenkins

下一篇：ElasticSearch進階：一文全覽各種ES查詢在Java中的實作