Redis Cluster主從模式
原理
redis cluster 為了保證資料的高可用性,加入了主從模式,一個主節點對應一個或多個從節點,主節點提供資料存取,從節點則是從主節點拉取資料,
備份,當這個主節點掛掉后,就會有這個從節點選取一個來充當主節點,從而保證集群不會掛掉,
說明
假如集群有A、B、C三個主節點, 如果這3個節點都沒有加入從節點,如果B掛掉了,我們就無法訪問整個集群了, A和C的slot也無法訪問,
所以在集群建立的時候,一定要為每個主節點都添加了從節點,,比如,像集群包含主節點A、B、C, 以及從節點A1、B1、C1, 那么即使B掛掉,系統也可以繼續正確作業 ,B1節點替代了B節點,故Redis集群將會選擇B1節點,作為新的主節點,集群將會繼續正確地提供服務 當B重新開啟后,它就會變成B1的從節點,
注意
如果節點B和B1同時掛了,Redis集群就無法繼續正確地提供服務了,
故最小的redis集群需要3個主節點(即Redis Cluster集群至少需要3個master節點,也就是說至少需要6個節點才能構建Redis cluster集群),一臺機器可運行多個redis實體(一般使用兩臺機器,每臺啟動3個redis實體,即三個主節點,三個從節點),
很多案例使用單臺服務器開6個埠,操作差不多,只是配置基本相對簡單點,多臺服務器更接近生產環境 ,
當集群最開始創建好后,要記住各節點的主從關系(或是創建的時候指定主從關系);若是其中一臺機器重啟,重啟后,需重新將其加入到redis cluster集群中;這就需要將這臺機器上的各節點之前的從節點變為主節點(客戶端執行slaveof no one),然后再根據新的主節點,添加這臺機器的各節點到集群中,添加后變為從節點,
搭建環境
環境 : CentOS7
redis01 : 172.16.7.11 埠 : 7000、7001、7002
redis02 : 172.16.7.22 埠 : 7003、7004、7005
redis03 : 172.16.7.33 埠 : 7006、7007、7008
關閉SElinux和防火墻
安裝redis
首先選擇一臺服務器,進行安裝
初始安裝redis01
[root@redis01 ~]# yum install -y gcc g++ make gcc-c++ kernel-devel automake autoconf libtool make wget tcl vim ruby rubygems unzip git
[root@redis01 ~]# wget http://download.redis.io/releases/redis-4.0.1.tar.gz
[root@redis01 ~]# tar xf redis-4.0.1.tar.gz -C /usr/local/src/
[root@redis01 ~]# cd /usr/local/src/redis-4.0.1/
[root@redis01 redis-4.0.1]# make && make test && make PREFIX=/usr/local/redis install
[root@redis01 redis-4.0.1]# mkdir /usr/local/redis/conf
[root@redis01 redis-4.0.1]# cp *.conf /usr/local/redis/conf/
[root@redis01 redis-4.0.1]# cp /usr/local/src/redis-4.0.1/src/redis-trib.rb /usr/local/redis/bin/
[root@redis01 redis-4.0.1]# ln -s /usr/local/redis/bin/* /usr/local/bin/
進行作業系統基礎調優設定
[root@redis01 redis-4.0.1]# echo "* - nofile 10240" >> /etc/security/limits.conf
[root@redis01 redis-4.0.1]# echo "net.core.somaxconn = 10240" >> /etc/sysctl.conf
[root@redis01 redis-4.0.1]# echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf
[root@redis01 redis-4.0.1]# sysctl -p
net.core.somaxconn = 10240
vm.overcommit_memory = 1
[root@redis01 redis-4.0.1]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
[root@redis01 redis-4.0.1]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@redis01 redis-4.0.1]# echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local
[root@redis01 redis-4.0.1]# echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.local
[root@redis01 redis-4.0.1]# su -l
創建節點
創建集群節點目錄
[root@redis01 ~]# mkdir /data/redis/redis-cluster -p
[root@redis01 redis-cluster]# cd /data/redis/redis-cluster/
[root@redis01 redis-cluster]# mkdir 7000 7001 7002
執行下面的腳本,配置各個節點的redis.conf
[root@redis01 redis-cluster]# for i in 0 1 2;
do
echo -e "cluster-enabled yes\n\
bind 0.0.0.0\n\
port 700$i\n\
pidfile /data/redis-cluster/700$i/redis.pid\n\
logfile "/data/redis-cluster/700$i/redis.log"\n\
dir /data/redis-cluster/7000/\n\
tcp-backlog 1024\n\
timeout 0\n\
cluster-node-timeout 10100\n\
tcp-keepalive 0\n\
daemonize yes\n\
loglevel notice\n\
databases 16\n\
stop-writes-on-bgsave-error yes\n\
rdbcompression yes\n\
rdbchecksum yes\n\
dbfilename "dump.rdb"\n\
slave-serve-stale-data yes\n\
slave-read-only yes\n\
repl-diskless-sync no\n\
repl-diskless-sync-delay 5\n\
repl-disable-tcp-nodelay no\n\
slave-priority 100\n\
lazyfree-lazy-eviction no\n\
lazyfree-lazy-expire no\n\
lazyfree-lazy-server-del no\n\
slave-lazy-flush no\n\
appendonly no\n\
appendfilename "appendonly.aof"\n\
appendfsync everysec\n\
no-appendfsync-on-rewrite yes\n\
auto-aof-rewrite-percentage 100\n\
auto-aof-rewrite-min-size 64mb\n\
aof-load-truncated yes\n\
lua-time-limit 5000\n\
slowlog-log-slower-than 10000\n\
slowlog-max-len 128\n\
latency-monitor-threshold 0\n\
notify-keyspace-events ""\n\
hash-max-ziplist-entries 512\n\
hash-max-ziplist-value 64\n\
list-max-ziplist-entries 512\n\
list-max-ziplist-value 64\n\
set-max-intset-entries 512\n\
zset-max-ziplist-entries 128\n\
zset-max-ziplist-value 64\n\
hll-sparse-max-bytes 3000\n\
activerehashing yes\n\
client-output-buffer-limit normal 0 0 0\n\
client-output-buffer-limit slave 256mb 64mb 60\n\
client-output-buffer-limit pubsub 32mb 8mb 60\n\
hz 10\n\
aof-rewrite-incremental-fsync yes" > 700$i/redis.conf;
done
查看各個節點的檔案樹
[root@redis01 redis-cluster]# tree
.
├── 7000
│ └── redis.conf
├── 7001
│ └── redis.conf
└── 7002
└── redis.conf
啟動redis01的集群服務
[root@redis01 redis-cluster]for i in 0 1 2;
do redis-server /data/redis/redis-cluster/700$i/redis.conf
done
68820:C 08 Oct 22:21:14.965 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
68820:C 08 Oct 22:21:14.965 # Redis version=4.0.1, bits=64, commit=00000000, modified=0, pid=68820, just started
68820:C 08 Oct 22:21:14.965 # Configuration loaded
68822:C 08 Oct 22:21:14.972 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
68822:C 08 Oct 22:21:14.972 # Redis version=4.0.1, bits=64, commit=00000000, modified=0, pid=68822, just started
68822:C 08 Oct 22:21:14.972 # Configuration loaded
68824:C 08 Oct 22:21:14.981 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
68824:C 08 Oct 22:21:14.981 # Redis version=4.0.1, bits=64, commit=00000000, modified=0, pid=68824, just started
68824:C 08 Oct 22:21:14.981 # Configuration loaded
redis02 服務器的安裝
redis02 ,redis03不需要完全按上面的步驟,只部分需要復制redis01的安裝解壓目錄
復制redis01的安裝解壓目錄
[root@node02 ~]# scp -r node01:/usr/local/redis /usr/local/

[root@redis02 ~]# scp -r node01:/data/redis/redis-cluster /data/redis/

修改某個服務器的各個節點的redis.conf檔案
[root@redis02 redis-cluster]# mv 7000 7003
[root@redis02 redis-cluster]# mv 7001 7004
[root@redis02 redis-cluster]# mv 7002 7005
[root@redis02 redis-cluster]# ls
7003 7004 7005
[root@redis02 redis-cluster]# sed -i 's#7000#7003#g' 7003/redis.conf
[root@redis02 redis-cluster]# sed -i 's#7001#7004#g' 7005/redis.conf
[root@redis02 redis-cluster]# sed -i 's#7002#7005#g' 7004/redis.conf
做運行腳本加入bin目錄
[root@node02 ~]# ln -s /usr/local/redis/bin/* /usr/local/bin/
進行作業系統基礎調優設定
如同redis01的作業系統基礎調優設定
啟動redis02的集群服務
[root@redis02 redis-cluster]# for i in 3 4 5;
do redis-server /data/redis/redis-cluster/700$i/redis.conf
done
41766:C 08 Oct 22:21:42.369 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
41766:C 08 Oct 22:21:42.369 # Redis version=4.0.1, bits=64, commit=00000000, modified=0, pid=41766, just started
41766:C 08 Oct 22:21:42.369 # Configuration loaded
41768:C 08 Oct 22:21:42.375 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
41768:C 08 Oct 22:21:42.375 # Redis version=4.0.1, bits=64, commit=00000000, modified=0, pid=41768, just started
41768:C 08 Oct 22:21:42.375 # Configuration loaded
41770:C 08 Oct 22:21:42.380 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
41770:C 08 Oct 22:21:42.380 # Redis version=4.0.1, bits=64, commit=00000000, modified=0, pid=41770, just started
41770:C 08 Oct 22:21:42.380 # Configuration loaded
redis03服務器的安裝
類比redis02 服務器的安裝
redis01,redis02,redis03聚合創建集群
查各Redis節點的啟動狀態
#yum -y install net-tools
[root@redis01 redis-cluster]# netstat -antup|grep redis
tcp 0 0 172.16.7.11:17000 0.0.0.0:* LISTEN 68821/redis-server
tcp 0 0 172.16.7.11:17001 0.0.0.0:* LISTEN 68823/redis-server
tcp 0 0 172.16.7.11:17002 0.0.0.0:* LISTEN 68828/redis-server
tcp 0 0 172.16.7.11:7000 0.0.0.0:* LISTEN 68821/redis-server
tcp 0 0 172.16.7.11:7001 0.0.0.0:* LISTEN 68823/redis-server
tcp 0 0 172.16.7.11:7002 0.0.0.0:* LISTEN 68828/redis-server
安裝ruby
[root@redis01 redis-cluster]# yum -y install rh-ruby23
[root@redis01 redis-cluster]# ruby -v
ruby 2.0.0p648 (2015-12-16) [x86_64-linux]
[root@redis01 redis-cluster]# scl enable rh-ruby23 bash
[root@redis01 redis-cluster]# ruby -v
ruby 2.3.6p384 (2017-12-14 revision 61254) [x86_64-linux]

解決方案;
yum install centos-release-scl-rh

再次安裝 rh-ruby23

安裝ruby的redis擴展
[root@redis01 redis-cluster]# gem install redis
Fetching: redis-4.0.2.gem (100%)
Successfully installed redis-4.0.2
Parsing documentation for redis-4.0.2
Installing ri documentation for redis-4.0.2
Done installing documentation for redis after 0 seconds
1 gem installed
使用redis自帶的集群管理工具redis-trib.rb創建集群
在redis01上運行以下腳本
[root@redis01 redis-cluster]# redis-trib.rb create --replicas 1 172.16.7.11:7000 172.16.7.11:7001 172.16.7.11:7002 172.16.7.22:7003 172.16.7.22:7004 172.16.7.22:7005 172.16.7.33:7006 172.16.7.33:7007 172.16.7.33:7008
>>> Creating cluster
>>> Performing hash slots allocation on 9 nodes...
Using 4 masters:
172.16.7.11:7000
172.16.7.22:7003
172.16.7.33:7006
172.16.7.11:7001
Adding replica 172.16.7.22:7004 to 172.16.7.11:7000
Adding replica 172.16.7.33:7007 to 172.16.7.22:7003
Adding replica 172.16.7.11:7002 to 172.16.7.33:7006
Adding replica 172.16.7.22:7005 to 172.16.7.11:7001
Adding replica 172.16.7.33:7008 to 172.16.7.11:7000
M: d8138e15d9cb69962abb7457125a7965b735a658 172.16.7.11:7000
slots:0-4095 (4096 slots) master
M: 72f44da597f73c8d23c6d3fb60fa95489eb0168d 172.16.7.11:7001
slots:12288-16383 (4096 slots) master
S: 711af78396c3ba966f751beef745ae10ff053a3f 172.16.7.11:7002
replicates af2e1c600ed2511155a6cad61cabeb095affff86
M: 691428ae7c1728a76520b8f9d46fa6d83930f191 172.16.7.22:7003
slots:4096-8191 (4096 slots) master
S: e752d865374c8ca4149737d81f19ac95cc9f6a44 172.16.7.22:7004
replicates d8138e15d9cb69962abb7457125a7965b735a658
S: ba4ce74ba7b84bde9ea60cd8bf4c5996f1ca01e5 172.16.7.22:7005
replicates 72f44da597f73c8d23c6d3fb60fa95489eb0168d
M: af2e1c600ed2511155a6cad61cabeb095affff86 172.16.7.33:7006
slots:8192-12287 (4096 slots) master
S: b1e201b2acdc0a0740a764978436fa3946325a11 172.16.7.33:7007
replicates 691428ae7c1728a76520b8f9d46fa6d83930f191
S: 6913f44486938c49eed6efa6f18c782df73e6456 172.16.7.33:7008
replicates d8138e15d9cb69962abb7457125a7965b735a658
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join........
>>> Performing Cluster Check (using node 172.16.7.11:7000)
M: d8138e15d9cb69962abb7457125a7965b735a658 172.16.7.11:7000
slots:0-4095 (4096 slots) master
2 additional replica(s)
S: b1e201b2acdc0a0740a764978436fa3946325a11 172.16.7.33:7007
slots: (0 slots) slave
replicates 691428ae7c1728a76520b8f9d46fa6d83930f191
S: e752d865374c8ca4149737d81f19ac95cc9f6a44 172.16.7.22:7004
slots: (0 slots) slave
replicates d8138e15d9cb69962abb7457125a7965b735a658
M: 691428ae7c1728a76520b8f9d46fa6d83930f191 172.16.7.22:7003
slots:4096-8191 (4096 slots) master
1 additional replica(s)
M: af2e1c600ed2511155a6cad61cabeb095affff86 172.16.7.33:7006
slots:8192-12287 (4096 slots) master
1 additional replica(s)
M: 72f44da597f73c8d23c6d3fb60fa95489eb0168d 172.16.7.11:7001
slots:12288-16383 (4096 slots) master
1 additional replica(s)
S: 711af78396c3ba966f751beef745ae10ff053a3f 172.16.7.11:7002
slots: (0 slots) slave
replicates af2e1c600ed2511155a6cad61cabeb095affff86
S: ba4ce74ba7b84bde9ea60cd8bf4c5996f1ca01e5 172.16.7.22:7005
slots: (0 slots) slave
replicates 72f44da597f73c8d23c6d3fb60fa95489eb0168d
S: 6913f44486938c49eed6efa6f18c782df73e6456 172.16.7.33:7008
slots: (0 slots) slave
replicates d8138e15d9cb69962abb7457125a7965b735a658
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
這樣,三個服務器啟動了9個實體,配置成4主5從,其中一個是一主兩從,其他都是一主一從
集群驗證
選項 -c 可連接集群, 因為組態檔中bind改成了ip,所以必須要用 -h 選項,p選項后接埠號
在redis01上的7000埠的實體上set一個key
[root@redis01 redis-cluster]# redis-cli -h 172.16.7.11 -c -p 7000
172.16.7.11:7000> set name haha
-> Redirected to slot [5798] located at 172.16.7.22:7003
OK
172.16.7.22:7003> get name
"haha"
由上可見 , redis01 set name 后,將資料重定向到了redis02的7003埠的實體上
在redis03的7008實體上get一個key
[root@redis01 redis-cluster]# redis-cli -h 172.16.7.33 -c -p 7008
172.16.7.33:7008> get name
-> Redirected to slot [5798] located at 172.16.7.22:7003
"weihan"
172.16.7.22:7003>
get name 還是會重定向到7003這個實體上 , 這說明redis cluster集群已經是可用的了
redis-trib.rb檢查集群狀態
4個master會分配到4個服務器上, 而且每個服務器上必須有一個master,創建集群的時候,可以指定master,若創建的時候沒指定,是默認分配
[root@redis01 redis-cluster]# redis-trib.rb check 172.16.7.11:7000
>>> Performing Cluster Check (using node 172.16.7.11:7000)
M: d8138e15d9cb69962abb7457125a7965b735a658 172.16.7.11:7000
slots:0-4095 (4096 slots) master
2 additional replica(s)
S: b1e201b2acdc0a0740a764978436fa3946325a11 172.16.7.33:7007
slots: (0 slots) slave
replicates 691428ae7c1728a76520b8f9d46fa6d83930f191
S: e752d865374c8ca4149737d81f19ac95cc9f6a44 172.16.7.22:7004
slots: (0 slots) slave
replicates d8138e15d9cb69962abb7457125a7965b735a658
M: 691428ae7c1728a76520b8f9d46fa6d83930f191 172.16.7.22:7003
slots:4096-8191 (4096 slots) master
1 additional replica(s)
M: af2e1c600ed2511155a6cad61cabeb095affff86 172.16.7.33:7006
slots:8192-12287 (4096 slots) master
1 additional replica(s)
M: 72f44da597f73c8d23c6d3fb60fa95489eb0168d 172.16.7.11:7001
slots:12288-16383 (4096 slots) master
1 additional replica(s)
S: 711af78396c3ba966f751beef745ae10ff053a3f 172.16.7.11:7002
slots: (0 slots) slave
replicates af2e1c600ed2511155a6cad61cabeb095affff86
S: ba4ce74ba7b84bde9ea60cd8bf4c5996f1ca01e5 172.16.7.22:7005
slots: (0 slots) slave
replicates 72f44da597f73c8d23c6d3fb60fa95489eb0168d
S: 6913f44486938c49eed6efa6f18c782df73e6456 172.16.7.33:7008
slots: (0 slots) slave
replicates d8138e15d9cb69962abb7457125a7965b735a658
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
列出集群節點
[root@redis01 redis-cluster]# redis-cli -h 172.16.7.11 -c -p 7000
172.16.7.11:7000> CLUSTER NODES
b1e201b2acdc0a0740a764978436fa3946325a11 172.16.7.33:7007@17007 slave 691428ae7c1728a76520b8f9d46fa6d83930f191 0 1539016061737 8 connected
e752d865374c8ca4149737d81f19ac95cc9f6a44 172.16.7.22:7004@17004 slave d8138e15d9cb69962abb7457125a7965b735a658 0 1539016062000 5 connected
691428ae7c1728a76520b8f9d46fa6d83930f191 172.16.7.22:7003@17003 master - 0 1539016064655 4 connected 4096-8191
af2e1c600ed2511155a6cad61cabeb095affff86 172.16.7.33:7006@17006 master - 0 1539016062000 7 connected 8192-12287
72f44da597f73c8d23c6d3fb60fa95489eb0168d 172.16.7.11:7001@17001 master - 0 1539016063648 2 connected 12288-16383
711af78396c3ba966f751beef745ae10ff053a3f 172.16.7.11:7002@17002 slave af2e1c600ed2511155a6cad61cabeb095affff86 0 1539016063000 7 connected
d8138e15d9cb69962abb7457125a7965b735a658 172.16.7.11:7000@17000 myself,master - 0 1539016063000 1 connected 0-4095
ba4ce74ba7b84bde9ea60cd8bf4c5996f1ca01e5 172.16.7.22:7005@17005 slave 72f44da597f73c8d23c6d3fb60fa95489eb0168d 0 1539016063000 6 connected
6913f44486938c49eed6efa6f18c782df73e6456 172.16.7.33:7008@17008 slave d8138e15d9cb69962abb7457125a7965b735a658 0 1539016063000 9 connected
列印集群資訊
172.16.7.11:7000> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:9
cluster_size:4
cluster_current_epoch:9
cluster_my_epoch:1
cluster_stats_messages_ping_sent:1240
cluster_stats_messages_pong_sent:1251
cluster_stats_messages_sent:2491
cluster_stats_messages_ping_received:1243
cluster_stats_messages_pong_received:1240
cluster_stats_messages_meet_received:8
cluster_stats_messages_received:2491
備注
若指定redis cluster中的master和slave ,可以用下面的方法
# 添加主庫
[root@redis01 redis-cluster]# redis-trib.rb create 172.16.7.11:7000 172.16.7.22:7003 172.16.7.33:7006
# 添加從庫
[root@redis01 redis-cluster]# redis-trib.rb add-node --slave 172.16.7.22:7001 172.16.7.11:7000 # 把172.16.7.11的7001 作為從庫添加到172.16.7.11的7000
轉載請註明出處,本文鏈接:https://www.uj5u.com/qita/147438.html
標籤:AI
下一篇:2017——2019
