部署 flannel 網路(在master節點上執行）
kubernetes組件kubelet服務依賴docker服務，docker網路需要用flannel來配置docker0網橋的ip地址，所以需要先安裝flannel網路組建
flannel 使用 vxlan 技術為各節點創建一個可以互通的 Pod 網路，使用的埠為 UDP 8472（需要開放該埠，如公有云 AWS 等），
flanneld 第一次啟動時，從 etcd 獲取配置的 Pod 網段資訊，為本節點分配一個未使用的地址段，然后創建 flannedl.1 網路介面（也可能是其它名稱，如 flannel1 等），
flannel 將分配給自己的 Pod 網段資訊寫入 /run/flannel/docker 檔案，docker 后續使用這個檔案中的環境變數設定 docker0 網橋，從而從這個地址段為本節點的所有 Pod 容器分配 IP
-
下載和安裝flanneld 二進制檔案
cd /opt/k8s/work mkdir flannel wget https://github.com/coreos/flannel/releases/download/v0.11.0/flannel-v0.11.0-linux-amd64.tar.gz tar -xzvf flannel-v0.11.0-linux-amd64.tar.gz -C flannel cp flannel/{flanneld,mk-docker-opts.sh} /opt/k8s/bin/ export node_ip=192.168.0.114 scp flannel/{flanneld,mk-docker-opts.sh} root@${192.168.0.114}:/opt/k8s/bin/ -
創建 flanneld 證書和私鑰
flanneld 從 etcd 集群存取網段分配資訊，而 etcd 集群啟用了雙向 x509 證書認證，所以需要為 flanneld 生成證書和私鑰，
-
創建證書簽名請求
cd /opt/k8s/work cat > flanneld-csr.json <<EOF { "CN": "flanneld", "hosts": [], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "NanJing", "L": "NanJing", "O": "k8s", "OU": "system" } ] } EOF -
生成證書和私鑰
cfssl gencert -ca=/opt/k8s/work/ca.pem \ -ca-key=/opt/k8s/work/ca-key.pem \ -config=/opt/k8s/work/ca-config.json \ -profile=kubernetes flanneld-csr.json | cfssljson -bare flanneld ls flanneld*pem -
將生成的證書和私鑰分發到所有節點
cd /opt/k8s/work mkdir -p /etc/flanneld/cert cp flanneld*.pem /etc/flanneld/cert export node_ip=192.168.0.114 ssh root@${node_ip} "mkdir -p /etc/flanneld/cert" scp flanneld*.pem root@${node_ip}:/etc/flanneld/cert
-
-
向 etcd 寫入集群 Pod 網段資訊
cd /opt/k8s/work export FLANNEL_ETCD_PREFIX="/kubernetes/network" export ETCD_ENDPOINTS="https://192.168.0.107:2379" etcdctl \ --endpoints=${ETCD_ENDPOINTS} \ --ca-file=/opt/k8s/work/ca.pem \ --cert-file=/opt/k8s/work/flanneld.pem \ --key-file=/opt/k8s/work/flanneld-key.pem \ mk ${FLANNEL_ETCD_PREFIX}/config '{"Network":"172.30.0.0/16", "SubnetLen": 24, "Backend": {"Type": "vxlan"}}'- 寫入的 Pod 網段 Network 網路段對應的數值（如 /16）必須小于 SubnetLen對應的值（如24）
-
創建 flanneld 服務的啟動檔案
cd /opt/k8s/work export FLANNEL_ETCD_PREFIX="/kubernetes/network" export ETCD_ENDPOINTS="https://192.168.0.107:2379" cat > flanneld.service << EOF [Unit] Description=Flanneld overlay address etcd agent After=network.target After=network-online.target Wants=network-online.target After=etcd.service Before=docker.service [Service] Type=notify ExecStart=/opt/k8s/bin/flanneld \\ -etcd-cafile=/etc/kubernetes/cert/ca.pem \\ -etcd-certfile=/etc/flanneld/cert/flanneld.pem \\ -etcd-keyfile=/etc/flanneld/cert/flanneld-key.pem \\ -etcd-endpoints=${ETCD_ENDPOINTS} \\ -etcd-prefix=${FLANNEL_ETCD_PREFIX} \\ -ip-masq ExecStartPost=/opt/k8s/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker Restart=always RestartSec=5 StartLimitInterval=0 [Install] WantedBy=multi-user.target RequiredBy=docker.service EOF- mk-docker-opts.sh 腳本將分配給 flanneld 的 Pod 子網段資訊，通過-d引數寫入 /run/flannel/docker 檔案，后續 docker 啟動時使用這個檔案中的環境變數配置 docker0 網橋， -k 引數控制生成檔案中變數的名稱，下面docker啟動時會用到這個變數；
- flanneld 使用系統預設路由所在的介面與其它節點通信，對于有多個網路介面（如內網和公網）的節點，可以用 -iface 引數指定通信介面;
- -ip-masq: flanneld 為訪問 Pod 網路外的流量設定 SNAT 規則，同時將傳遞給 Docker 的變數 --ip-masq（/run/flannel/docker 檔案中）設定為 false，這樣 Docker 將不再創建 SNAT 規則； Docker 的 --ip-masq 為 true 時，創建的 SNAT 規則比較“暴力”：將所有本節點 Pod 發起的、訪問非 docker0 介面的請求做 SNAT，這樣訪問其他節點 Pod 的請求來源 IP 會被設定為 flannel.1 介面的 IP，導致目的 Pod 看不到真實的來源 Pod IP， flanneld 創建的 SNAT 規則比較溫和，只對訪問非 Pod 網段的請求做 SNAT
-
分發flanneld服務
cd /opt/k8s/work cp flanneld.service /etc/systemd/system/ export node_ip=192.168.0.114 scp flanneld.service root@${node_ip}:/etc/systemd/system/ -
啟動flanneld服務
systemctl daemon-reload && systemctl enable flanneld && systemctl restart flanneld ssh root@${node_ip) "systemctl daemon-reload && systemctl enable flanneld && systemctl restart flanneld" -
檢查啟動結果
systemctl status flanneld|grep Active export node_ip=192.168.0.114 ssh root@${node_ip} "systemctl status flanneld|grep Active"-
確保狀態為 active (running)，否則查看日志，確認原因
-
如果出現例外，通過如下命令查看
journalctl -u flanneld
-
-
檢查分配給各 flanneld 的 Pod 網段資訊
export FLANNEL_ETCD_PREFIX="/kubernetes/network" export ETCD_ENDPOINTS="https://192.168.0.107:2379" etcdctl \ --endpoints=${ETCD_ENDPOINTS} \ --ca-file=/etc/kubernetes/cert/ca.pem \ --cert-file=/etc/flanneld/cert/flanneld.pem \ --key-file=/etc/flanneld/cert/flanneld-key.pem \ get ${FLANNEL_ETCD_PREFIX}/config輸出結果
{"Network":"172.30.0.0/16", "SubnetLen": 24, "Backend": {"Type": "vxlan"}} -
查看已分配的 Pod 子網段串列
export FLANNEL_ETCD_PREFIX="/kubernetes/network" export ETCD_ENDPOINTS="https://192.168.0.107:2379" etcdctl \ --endpoints=${ETCD_ENDPOINTS} \ --ca-file=/etc/kubernetes/cert/ca.pem \ --cert-file=/etc/flanneld/cert/flanneld.pem \ --key-file=/etc/flanneld/cert/flanneld-key.pem \ ls ${FLANNEL_ETCD_PREFIX}/subnets輸出結果
/kubernetes/network/subnets/172.30.22.0-24 /kubernetes/network/subnets/172.30.78.0-24 -
檢查節點 flannel 網路資訊
root@master:/opt/k8s/work# ip addr show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp2s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000 link/ether 04:92:26:13:92:2b brd ff:ff:ff:ff:ff:ff 3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether d0:c5:d3:57:73:01 brd ff:ff:ff:ff:ff:ff inet 192.168.0.107/24 brd 192.168.0.255 scope global dynamic noprefixroute wlp3s0 valid_lft 6385sec preferred_lft 6385sec inet6 fe80::1fda:e90a:207a:67e4/64 scope link noprefixroute valid_lft forever preferred_lft forever 4: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default link/ether 12:cb:66:43:de:36 brd ff:ff:ff:ff:ff:ff inet 172.30.22.0/32 scope global flannel.1 valid_lft forever preferred_lft forever inet6 fe80::10cb:66ff:fe43:de36/64 scope link valid_lft forever preferred_lft forever root@master:/opt/k8s/work# ip route show |grep flannel.1 172.30.78.0/24 via 172.30.78.0 dev flannel.1 onlink -
驗證各節點能通過 Pod 網段互通
root@master:/opt/k8s/work# ip addr show flannel.1 |grep -w inet inet 172.30.22.0/32 scope global flannel.1 root@master:/opt/k8s/work# ssh 192.168.0.114 "/sbin/ip addr show flannel.1|grep -w inet" inet 172.30.78.0/32 scope global flannel.1 root@master:/opt/k8s/work# ping -c 1 172.30.78.0 PING 172.30.78.0 (172.30.78.0) 56(84) bytes of data. 64 bytes from 172.30.78.0: icmp_seq=1 ttl=64 time=80.7 ms --- 172.30.78.0 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 80.707/80.707/80.707/0.000 ms root@master:/opt/k8s/work# ssh 192.168.0.114 "ping -c 1 172.30.22.0" PING 172.30.22.0 (172.30.22.0) 56(84) bytes of data. 64 bytes from 172.30.22.0: icmp_seq=1 ttl=64 time=4.09 ms --- 172.30.22.0 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 4.094/4.094/4.094/0.000 ms -
生成檔案內容，注意DOCKER_NETWORK_OPTIONS的值
root@master:/opt/k8s/work# cat /run/flannel/subnet.env FLANNEL_NETWORK=172.30.0.0/16 FLANNEL_SUBNET=172.30.22.1/24 FLANNEL_MTU=1450 FLANNEL_IPMASQ=true root@master:/opt/k8s/work# cat /run/flannel/docker DOCKER_OPT_BIP="--bip=172.30.22.1/24" DOCKER_OPT_IPMASQ="--ip-masq=false" DOCKER_OPT_MTU="--mtu=1450" DOCKER_NETWORK_OPTIONS=" --bip=172.30.22.1/24 --ip-masq=false --mtu=1450"
轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/31173.html
標籤：其他