56. redis集群搭建及管理¶
1. 经典架构设计图¶
2. 集群说明¶
一个redis集群包含16384个哈希槽（hash slot）
集群使用公式CRC16（key）%16384来计算key属于哪个槽
3. 安装集群插件¶
yum install ruby rubygems -y
[root@152 ~]# yum install ruby rubygems -y
Loaded plugins: fastestmirror
Determining fastest mirrors
* base: mirrors.aliyun.com
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
base | 3.6 kB 00:00:00
epel | 4.7 kB 00:00:00
extras | 2.9 kB 00:00:00
updates | 2.9 kB 00:00:00
(1/4): extras/7/x86_64/primary_db | 236 kB 00:00:00
(2/4): epel/x86_64/updateinfo | 1.0 MB 00:00:00
(3/4): epel/x86_64/primary_db | 6.9 MB 00:00:00
(4/4): updates/7/x86_64/primary_db | 8.0 MB 00:00:03
Resolving Dependencies
--> Running transaction check
---> Package ruby.x86_64 0:2.0.0.648-36.el7 will be installed
--> Processing Dependency: ruby-libs(x86-64) = 2.0.0.648-36.el7 for package: ruby-2.0.0.648-36.el7.x86_64
--> Processing Dependency: rubygem(bigdecimal) >= 1.2.0 for package: ruby-2.0.0.648-36.el7.x86_64
--> Processing Dependency: libruby.so.2.0()(64bit) for package: ruby-2.0.0.648-36.el7.x86_64
---> Package rubygems.noarch 0:2.0.14.1-36.el7 will be installed
--> Processing Dependency: rubygem(rdoc) >= 4.0.0 for package: rubygems-2.0.14.1-36.el7.noarch
--> Processing Dependency: rubygem(psych) >= 2.0.0 for package: rubygems-2.0.14.1-36.el7.noarch
--> Processing Dependency: rubygem(io-console) >= 0.4.2 for package: rubygems-2.0.14.1-36.el7.noarch
--> Running transaction check
---> Package ruby-libs.x86_64 0:2.0.0.648-36.el7 will be installed
---> Package rubygem-bigdecimal.x86_64 0:1.2.0-36.el7 will be installed
---> Package rubygem-io-console.x86_64 0:0.4.2-36.el7 will be installed
---> Package rubygem-psych.x86_64 0:2.0.0-36.el7 will be installed
--> Processing Dependency: libyaml-0.so.2()(64bit) for package: rubygem-psych-2.0.0-36.el7.x86_64
---> Package rubygem-rdoc.noarch 0:4.0.0-36.el7 will be installed
--> Processing Dependency: ruby(irb) = 2.0.0.648 for package: rubygem-rdoc-4.0.0-36.el7.noarch
--> Processing Dependency: rubygem(json) >= 1.7.7 for package: rubygem-rdoc-4.0.0-36.el7.noarch
--> Running transaction check
---> Package libyaml.x86_64 0:0.1.4-11.el7_0 will be installed
---> Package ruby-irb.noarch 0:2.0.0.648-36.el7 will be installed
---> Package rubygem-json.x86_64 0:1.7.7-36.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=========================================================================================================================================================================================
Package Arch Version Repository Size
=========================================================================================================================================================================================
Installing:
ruby x86_64 2.0.0.648-36.el7 base 73 k
rubygems noarch 2.0.14.1-36.el7 base 215 k
Installing for dependencies:
libyaml x86_64 0.1.4-11.el7_0 base 55 k
ruby-irb noarch 2.0.0.648-36.el7 base 94 k
ruby-libs x86_64 2.0.0.648-36.el7 base 2.8 M
rubygem-bigdecimal x86_64 1.2.0-36.el7 base 85 k
rubygem-io-console x86_64 0.4.2-36.el7 base 56 k
rubygem-json x86_64 1.7.7-36.el7 base 81 k
rubygem-psych x86_64 2.0.0-36.el7 base 84 k
rubygem-rdoc noarch 4.0.0-36.el7 base 324 k
Transaction Summary
=========================================================================================================================================================================================
Install 2 Packages (+8 Dependent packages)
Total download size: 3.8 M
Installed size: 13 M
Downloading packages:
(1/10): libyaml-0.1.4-11.el7_0.x86_64.rpm | 55 kB 00:00:00
(2/10): ruby-2.0.0.648-36.el7.x86_64.rpm | 73 kB 00:00:00
(3/10): ruby-irb-2.0.0.648-36.el7.noarch.rpm | 94 kB 00:00:00
(4/10): rubygem-bigdecimal-1.2.0-36.el7.x86_64.rpm | 85 kB 00:00:00
(5/10): rubygem-io-console-0.4.2-36.el7.x86_64.rpm | 56 kB 00:00:00
(6/10): rubygem-json-1.7.7-36.el7.x86_64.rpm | 81 kB 00:00:00
(7/10): rubygem-psych-2.0.0-36.el7.x86_64.rpm | 84 kB 00:00:00
(8/10): rubygem-rdoc-4.0.0-36.el7.noarch.rpm | 324 kB 00:00:00
(9/10): ruby-libs-2.0.0.648-36.el7.x86_64.rpm | 2.8 MB 00:00:00
(10/10): rubygems-2.0.14.1-36.el7.noarch.rpm | 215 kB 00:00:00
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 6.1 MB/s | 3.8 MB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : ruby-libs-2.0.0.648-36.el7.x86_64 1/10
Installing : libyaml-0.1.4-11.el7_0.x86_64 2/10
Installing : rubygem-psych-2.0.0-36.el7.x86_64 3/10
Installing : ruby-irb-2.0.0.648-36.el7.noarch 4/10
Installing : ruby-2.0.0.648-36.el7.x86_64 5/10
Installing : rubygem-bigdecimal-1.2.0-36.el7.x86_64 6/10
Installing : rubygem-json-1.7.7-36.el7.x86_64 7/10
Installing : rubygem-io-console-0.4.2-36.el7.x86_64 8/10
Installing : rubygem-rdoc-4.0.0-36.el7.noarch 9/10
Installing : rubygems-2.0.14.1-36.el7.noarch 10/10
Verifying : rubygems-2.0.14.1-36.el7.noarch 1/10
Verifying : libyaml-0.1.4-11.el7_0.x86_64 2/10
Verifying : ruby-2.0.0.648-36.el7.x86_64 3/10
Verifying : rubygem-psych-2.0.0-36.el7.x86_64 4/10
Verifying : ruby-irb-2.0.0.648-36.el7.noarch 5/10
Verifying : rubygem-bigdecimal-1.2.0-36.el7.x86_64 6/10
Verifying : rubygem-json-1.7.7-36.el7.x86_64 7/10
Verifying : rubygem-io-console-0.4.2-36.el7.x86_64 8/10
Verifying : ruby-libs-2.0.0.648-36.el7.x86_64 9/10
Verifying : rubygem-rdoc-4.0.0-36.el7.noarch 10/10
Installed:
ruby.x86_64 0:2.0.0.648-36.el7 rubygems.noarch 0:2.0.14.1-36.el7
Dependency Installed:
libyaml.x86_64 0:0.1.4-11.el7_0 ruby-irb.noarch 0:2.0.0.648-36.el7 ruby-libs.x86_64 0:2.0.0.648-36.el7 rubygem-bigdecimal.x86_64 0:1.2.0-36.el7
rubygem-io-console.x86_64 0:0.4.2-36.el7 rubygem-json.x86_64 0:1.7.7-36.el7 rubygem-psych.x86_64 0:2.0.0-36.el7 rubygem-rdoc.noarch 0:4.0.0-36.el7
Complete!
4. 使用国内ruby源¶
gem sources -l
gem sources -a http://mirrors.aliyun.com/rubygems/
gem sources --remove https://rubygems.org/
gem sources -l
gem install redis -v 3.3.3
[root@152 ~]# gem sources -l
*** CURRENT SOURCES ***
https://rubygems.org/
[root@152 ~]# gem sources -a http://mirrors.aliyun.com/rubygems/
http://mirrors.aliyun.com/rubygems/ added to sources
[root@152 ~]# gem sources -l
*** CURRENT SOURCES ***
https://rubygems.org/
http://mirrors.aliyun.com/rubygems/
[root@152 ~]# gem sources --remove https://rubygems.org/
https://rubygems.org/ removed from sources
[root@152 ~]# gem sources -l
*** CURRENT SOURCES ***
http://mirrors.aliyun.com/rubygems/
[root@152 ~]# gem install redis -v 3.3.3
Fetching: redis-3.3.3.gem (100%)
Successfully installed redis-3.3.3
Parsing documentation for redis-3.3.3
Installing ri documentation for redis-3.3.3
1 gem installed
5. 准备集群节点¶
5.1 创建节点目录¶
mkdir /data/700{0..7}
5.2 创建配置文件¶
#7000
vim /data/7000/redis.conf
port 7000
daemonize yes
pidfile /data/7000/redis.pid
loglevel notice
logfile "/data/7000/redis.log"
dbfilename dump.rdb
dir /data/7000
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
#7001
vim /data/7001/redis.conf
port 7001
daemonize yes
pidfile /data/7001/redis.pid
loglevel notice
logfile "/data/7001/redis.log"
dbfilename dump.rdb
dir /data/7001
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
#7002
vim /data/7002/redis.conf
port 7002
daemonize yes
pidfile /data/7002/redis.pid
loglevel notice
logfile "/data/7002/redis.log"
dbfilename dump.rdb
dir /data/7002
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
#7003
vim /data/7003/redis.conf
port 7003
daemonize yes
pidfile /data/7003/redis.pid
loglevel notice
logfile "/data/7003/redis.log"
dbfilename dump.rdb
dir /data/7003
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
#7004
vim /data/7004/redis.conf
port 7004
daemonize yes
pidfile /data/7004/redis.pid
loglevel notice
logfile "/data/7004/redis.log"
dbfilename dump.rdb
dir /data/7004
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
#7005
vim /data/7005/redis.conf
port 7005
daemonize yes
pidfile /data/7005/redis.pid
loglevel notice
logfile "/data/7005/redis.log"
dbfilename dump.rdb
dir /data/7005
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
#7006
vim /data/7006/redis.conf
port 7006
daemonize yes
pidfile /data/7006/redis.pid
loglevel notice
logfile "/data/7006/redis.log"
dbfilename dump.rdb
dir /data/7006
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
#7007
vim /data/7007/redis.conf
port 7007
daemonize yes
pidfile /data/7007/redis.pid
loglevel notice
logfile "/data/7007/redis.log"
dbfilename dump.rdb
dir /data/7007
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
5.3 启动节点¶
redis-server /data/7000/redis.conf
redis-server /data/7001/redis.conf
redis-server /data/7002/redis.conf
redis-server /data/7003/redis.conf
redis-server /data/7004/redis.conf
redis-server /data/7005/redis.conf
redis-server /data/7006/redis.conf
redis-server /data/7007/redis.conf
[root@152 ~]# mkdir /data/700{0..7}
[root@152 ~]# vim /data/7000/redis.conf
[root@152 ~]# vim /data/7001/redis.conf
[root@152 ~]# vim /data/7002/redis.conf
[root@152 ~]# vim /data/7003/redis.conf
[root@152 ~]# vim /data/7004/redis.conf
[root@152 ~]# vim /data/7005/redis.conf
[root@152 ~]# vim /data/7006/redis.conf
[root@152 ~]# vim /data/7007/redis.conf
[root@152 ~]#
[root@152 ~]# redis-server /data/7000/redis.conf
[root@152 ~]# redis-server /data/7001/redis.conf
[root@152 ~]# redis-server /data/7002/redis.conf
[root@152 ~]# redis-server /data/7003/redis.conf
[root@152 ~]# redis-server /data/7004/redis.conf
[root@152 ~]# redis-server /data/7005/redis.conf
[root@152 ~]# redis-server /data/7006/redis.conf
[root@152 ~]# redis-server /data/7007/redis.conf
[root@152 ~]#
[root@152 ~]# netstat -tunlp|grep 700
tcp 0 0 0.0.0.0:17000 0.0.0.0:* LISTEN 1602/redis-server *
tcp 0 0 0.0.0.0:17001 0.0.0.0:* LISTEN 1604/redis-server *
tcp 0 0 0.0.0.0:17002 0.0.0.0:* LISTEN 1606/redis-server *
tcp 0 0 0.0.0.0:17003 0.0.0.0:* LISTEN 1612/redis-server *
tcp 0 0 0.0.0.0:17004 0.0.0.0:* LISTEN 1616/redis-server *
tcp 0 0 0.0.0.0:17005 0.0.0.0:* LISTEN 1622/redis-server *
tcp 0 0 0.0.0.0:17006 0.0.0.0:* LISTEN 1624/redis-server *
tcp 0 0 0.0.0.0:17007 0.0.0.0:* LISTEN 1636/redis-server *
tcp 0 0 0.0.0.0:7000 0.0.0.0:* LISTEN 1602/redis-server *
tcp 0 0 0.0.0.0:7001 0.0.0.0:* LISTEN 1604/redis-server *
tcp 0 0 0.0.0.0:7002 0.0.0.0:* LISTEN 1606/redis-server *
tcp 0 0 0.0.0.0:7003 0.0.0.0:* LISTEN 1612/redis-server *
tcp 0 0 0.0.0.0:7004 0.0.0.0:* LISTEN 1616/redis-server *
tcp 0 0 0.0.0.0:7005 0.0.0.0:* LISTEN 1622/redis-server *
tcp 0 0 0.0.0.0:7006 0.0.0.0:* LISTEN 1624/redis-server *
tcp 0 0 0.0.0.0:7007 0.0.0.0:* LISTEN 1636/redis-server *
tcp6 0 0 :::17000 :::* LISTEN 1602/redis-server *
tcp6 0 0 :::17001 :::* LISTEN 1604/redis-server *
tcp6 0 0 :::17002 :::* LISTEN 1606/redis-server *
tcp6 0 0 :::17003 :::* LISTEN 1612/redis-server *
tcp6 0 0 :::17004 :::* LISTEN 1616/redis-server *
tcp6 0 0 :::17005 :::* LISTEN 1622/redis-server *
tcp6 0 0 :::17006 :::* LISTEN 1624/redis-server *
tcp6 0 0 :::17007 :::* LISTEN 1636/redis-server *
tcp6 0 0 :::7000 :::* LISTEN 1602/redis-server *
tcp6 0 0 :::7001 :::* LISTEN 1604/redis-server *
tcp6 0 0 :::7002 :::* LISTEN 1606/redis-server *
tcp6 0 0 :::7003 :::* LISTEN 1612/redis-server *
tcp6 0 0 :::7004 :::* LISTEN 1616/redis-server *
tcp6 0 0 :::7005 :::* LISTEN 1622/redis-server *
tcp6 0 0 :::7006 :::* LISTEN 1624/redis-server *
tcp6 0 0 :::7007 :::* LISTEN 1636/redis-server *
5.4 查看节点¶
ps -ef |grep redis
[root@152 ~]# ps -ef |grep redis
root 1602 1 0 08:31 ? 00:00:00 redis-server *:7000 [cluster]
root 1604 1 0 08:31 ? 00:00:00 redis-server *:7001 [cluster]
root 1606 1 0 08:31 ? 00:00:00 redis-server *:7002 [cluster]
root 1612 1 0 08:31 ? 00:00:00 redis-server *:7003 [cluster]
root 1616 1 0 08:31 ? 00:00:00 redis-server *:7004 [cluster]
root 1622 1 0 08:31 ? 00:00:00 redis-server *:7005 [cluster]
root 1624 1 0 08:31 ? 00:00:00 redis-server *:7006 [cluster]
root 1636 1 0 08:31 ? 00:00:00 redis-server *:7007 [cluster]
root 1890 125223 0 08:31 pts/0 00:00:00 grep --color=auto redis
root 73723 1 0 05:25 ? 00:00:32 redis-server 192.168.178.152:6379
root 104140 1 0 07:10 ? 00:00:12 redis-server *:6382
root 120884 1 0 07:56 ? 00:00:05 redis-server *:6381
root 122065 1 0 08:00 ? 00:00:13 redis-sentinel *:26380 [sentinel]
root 126505 1 0 08:14 ? 00:00:02 redis-server *:6380
root 128277 125223 0 08:19 pts/0 00:00:05 redis-sentinel *:26381 [sentinel]
6. 未创建集群时测试¶
#1.连接节点测试
[root@152 ~]# redis-cli -p 7000
127.0.0.1:7000> set a 1
(error) CLUSTERDOWN Hash slot not served
127.0.0.1:7000> get a
(error) CLUSTERDOWN Hash slot not served
127.0.0.1:7000> quit
[root@152 ~]# redis-cli -p 7001
127.0.0.1:7001> get a
(error) CLUSTERDOWN Hash slot not served
127.0.0.1:7001> set a 2
(error) CLUSTERDOWN Hash slot not served
127.0.0.1:7001> quit
[root@152 ~]# redis-cli -p 7007
127.0.0.1:7007> get a
(error) CLUSTERDOWN Hash slot not served
127.0.0.1:7007> set a 2
(error) CLUSTERDOWN Hash slot not served
127.0.0.1:7007> quit
#2.查看集群状态
[root@152 ~]# redis-cli -p 7000 cluster nodes
58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 :7000 myself,master - 0 0 0 connected
#3.未安装ruby源时，查看命令位置
[root@152 ~]# which redis-trib.rb
/data/redis/src/redis-trib.rb
#4.未安装ruby源时测试
[root@152 ~]# redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 \
> 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
/usr/bin/env: ruby: No such file or directory
#5.再次使用如下命令测试：redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
[root@152 ~]# redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
/usr/bin/env: ruby: No such file or directory
[root@152 ~]# whereis redis-trib.rb
redis-trib: /data/redis/src/redis-trib.rb
#6.看样子必须得有依赖环境才行（等网络恢复后测试）
7. 创建集群¶
redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 \
127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
#前三个是主节点，后三个是从节点。主从节点最好是分布到不同的服务器上
[root@152 ~]# redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 \
> 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
127.0.0.1:7000
127.0.0.1:7001
127.0.0.1:7002
Adding replica 127.0.0.1:7003 to 127.0.0.1:7000
Adding replica 127.0.0.1:7004 to 127.0.0.1:7001
Adding replica 127.0.0.1:7005 to 127.0.0.1:7002
M: 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000
slots:0-5460 (5461 slots) master
M: c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001
slots:5461-10922 (5462 slots) master
M: 9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002
slots:10923-16383 (5461 slots) master
S: 98121a6c59dbaab55895fc93e8ab117d91ad4780 127.0.0.1:7003
replicates 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592
S: 02fbee35159f58c854e5931e582d825850033bb4 127.0.0.1:7004
replicates c3b18b156bcbefa24087c667be475755fe2b1488
S: 7c58f48395db82555b218af6382822b65797a2dc 127.0.0.1:7005
replicates 9d9d6e0a351f17d679523b55f2a4d6874957240c
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 02fbee35159f58c854e5931e582d825850033bb4 127.0.0.1:7004
slots: (0 slots) slave
replicates c3b18b156bcbefa24087c667be475755fe2b1488
S: 98121a6c59dbaab55895fc93e8ab117d91ad4780 127.0.0.1:7003
slots: (0 slots) slave
replicates 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592
M: c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 7c58f48395db82555b218af6382822b65797a2dc 127.0.0.1:7005
slots: (0 slots) slave
replicates 9d9d6e0a351f17d679523b55f2a4d6874957240c
M: 9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002
slots:10923-16383 (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
8. 查看集群状态¶
#集群主节点状态
redis-cli -p 7000 cluster nodes |grep master
#集群从节点状态
redis-cli -p 7000 cluster nodes |grep slave
[root@152 ~]# redis-cli -p 7000 cluster nodes
02fbee35159f58c854e5931e582d825850033bb4 127.0.0.1:7004 slave c3b18b156bcbefa24087c667be475755fe2b1488 0 1620652136720 5 connected
98121a6c59dbaab55895fc93e8ab117d91ad4780 127.0.0.1:7003 slave 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 0 1620652135194 4 connected
c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001 master - 0 1620652136009 2 connected 5461-10922
58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460
7c58f48395db82555b218af6382822b65797a2dc 127.0.0.1:7005 slave 9d9d6e0a351f17d679523b55f2a4d6874957240c 0 1620652136213 6 connected
9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002 master - 0 1620652135703 3 connected 10923-16383
#集群主节点状态
[root@152 ~]# redis-cli -p 7000 cluster nodes |grep master
c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001 master - 0 1620652125750 2 connected 5461-10922
58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460
9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002 master - 0 1620652125218 3 connected 10923-16383
#集群从节点状态
[root@152 ~]# redis-cli -p 7000 cluster nodes |grep slave
02fbee35159f58c854e5931e582d825850033bb4 127.0.0.1:7004 slave c3b18b156bcbefa24087c667be475755fe2b1488 0 1620652129302 5 connected
98121a6c59dbaab55895fc93e8ab117d91ad4780 127.0.0.1:7003 slave 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 0 1620652126827 4 connected
7c58f48395db82555b218af6382822b65797a2dc 127.0.0.1:7005 slave 9d9d6e0a351f17d679523b55f2a4d6874957240c 0 1620652127900 6 connected
redis集群添加节点¶
1. 添加主节点¶
redis-trib.rb add-node 127.0.0.1:7006 127.0.0.1:7000 #127.0.0.1:7000 是集群名
[root@152 ~]# redis-trib.rb add-node 127.0.0.1:7006 127.0.0.1:7000
>>> Adding node 127.0.0.1:7006 to cluster 127.0.0.1:7000
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 02fbee35159f58c854e5931e582d825850033bb4 127.0.0.1:7004
slots: (0 slots) slave
replicates c3b18b156bcbefa24087c667be475755fe2b1488
S: 98121a6c59dbaab55895fc93e8ab117d91ad4780 127.0.0.1:7003
slots: (0 slots) slave
replicates 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592
M: c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 7c58f48395db82555b218af6382822b65797a2dc 127.0.0.1:7005
slots: (0 slots) slave
replicates 9d9d6e0a351f17d679523b55f2a4d6874957240c
M: 9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002
slots:10923-16383 (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:7006 to make it join the cluster.
[OK] New node added correctly.
2. 查看主节点状态¶
redis-cli -p 7000 cluster nodes |grep master
#新加的节点没有哈希槽
[root@152 ~]# redis-cli -p 7000 cluster nodes |grep master
c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001 master - 0 1620652262285 2 connected 5461-10922
2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006 master - 0 1620652262792 0 connected
58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460
9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002 master - 0 1620652263803 3 connected 10923-16383
3. 添加从节点¶
#1.找到master7006的id
redis-cli -p 7000 cluster nodes |grep master |grep 7006
#2.添加从节点
redis-trib.rb add-node --slave --master-id <7006-master-id> 127.0.0.1:7007 127.0.0.1:7000
#1.找到master7006的id : 2713612b892f94fa866672394235ae8ab79f360e
[root@152 ~]# redis-cli -p 7000 cluster nodes |grep master |grep 7006
2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006 master - 0 1620652289940 0 connected
#2.根据上一步得到的id，添加从节点7007
#redis-trib.rb add-node --slave --master-id 2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7007 127.0.0.1:7000
[root@152 ~]# redis-trib.rb add-node --slave --master-id 2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7007 127.0.0.1:7000
>>> Adding node 127.0.0.1:7007 to cluster 127.0.0.1:7000
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 02fbee35159f58c854e5931e582d825850033bb4 127.0.0.1:7004
slots: (0 slots) slave
replicates c3b18b156bcbefa24087c667be475755fe2b1488
S: 98121a6c59dbaab55895fc93e8ab117d91ad4780 127.0.0.1:7003
slots: (0 slots) slave
replicates 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592
M: c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: 2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006
slots: (0 slots) master
0 additional replica(s)
S: 7c58f48395db82555b218af6382822b65797a2dc 127.0.0.1:7005
slots: (0 slots) slave
replicates 9d9d6e0a351f17d679523b55f2a4d6874957240c
M: 9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002
slots:10923-16383 (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:7007 to make it join the cluster.
Waiting for the cluster to join.
>>> Configure node as replica of 127.0.0.1:7006.
[OK] New node added correctly.
4. 转移slot（重新分片）¶
redis-trib.rb reshard 127.0.0.1:7000
#分别输入4096、7006masterid、all、yes
#注意：生产环境要在业务不繁忙的时候做
[root@152 ~]# echo $((16384/4))
4096
[root@152 ~]# redis-trib.rb reshard 127.0.0.1:7000
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: e9a6d166740ff2047deed0bbc12cc5b1bbac91a1 127.0.0.1:7007
slots: (0 slots) slave
replicates 2713612b892f94fa866672394235ae8ab79f360e
S: 02fbee35159f58c854e5931e582d825850033bb4 127.0.0.1:7004
slots: (0 slots) slave
replicates c3b18b156bcbefa24087c667be475755fe2b1488
S: 98121a6c59dbaab55895fc93e8ab117d91ad4780 127.0.0.1:7003
slots: (0 slots) slave
replicates 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592
M: c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: 2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006
slots: (0 slots) master
1 additional replica(s)
S: 7c58f48395db82555b218af6382822b65797a2dc 127.0.0.1:7005
slots: (0 slots) slave
replicates 9d9d6e0a351f17d679523b55f2a4d6874957240c
M: 9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002
slots:10923-16383 (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 4096
What is the receiving node ID? 2713612b892f94fa866672394235ae8ab79f360e
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:all
Ready to move 4096 slots.
Source nodes:
M: 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: 9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002
slots:10923-16383 (5461 slots) master
1 additional replica(s)
Destination node:
M: 2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006
slots: (0 slots) master
1 additional replica(s)
Resharding plan:
Moving slot 5461 from c3b18b156bcbefa24087c667be475755fe2b1488
Moving slot 5462 from c3b18b156bcbefa24087c667be475755fe2b1488
...
...
...
Do you want to proceed with the proposed reshard plan (yes/no)? yes
Moving slot 5461 from 127.0.0.1:7001 to 127.0.0.1:7006:
Moving slot 5462 from 127.0.0.1:7001 to 127.0.0.1:7006:
Moving slot 5463 from 127.0.0.1:7001 to 127.0.0.1:7006:
Moving slot 5464 from 127.0.0.1:7001 to 127.0.0.1:7006:
Moving slot 5465 from 127.0.0.1:7001 to 127.0.0.1:7006:
...
...
...
5. 再次查看主节点状态¶
redis-cli -p 7000 cluster nodes|grep master
[root@152 ~]# redis-cli -p 7000 cluster nodes|grep master
c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001 master - 0 1620652644118 2 connected 6827-10922
2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006 master - 0 1620652645631 7 connected 0-1364 5461-6826 10923-12287
58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000 myself,master - 0 0 1 connected 1365-5460
9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002 master - 0 1620652645128 3 connected 12288-16383
#可以看出，新加的7006节点的slot多了3段，加起来是4096个
[root@152 ~]# redis-cli -p 7000 cluster nodes|grep master|grep 7006
2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006 master - 0 1620652680969 7 connected 0-1364 5461-6826 10923-12287
#根据上面得到的信息，可以看出如下3段的slot个数分别如下
0-1364=1365
5461-6826=1366
10923-12287=1365
redis集群删除节点¶
1. 将需要删除节点的slot移走¶
#由于有3段slot，所以要移3次
redis-trib.rb reshard 127.0.0.1:7000
#1.1365
#2.7000masterid
#3.7006masterid
#4.done
redis-trib.rb reshard 127.0.0.1:7000
#1.1366
#2.7001masterid
#3.7006masterid
#4.done
redis-trib.rb reshard 127.0.0.1:7000
#1.1365
#2.7002masterid
#3.7006masterid
#4.done
#第一段slot
[root@152 ~]# redis-trib.rb reshard 127.0.0.1:7000
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000
slots:1365-5460 (4096 slots) master
1 additional replica(s)
S: e9a6d166740ff2047deed0bbc12cc5b1bbac91a1 127.0.0.1:7007
slots: (0 slots) slave
replicates 2713612b892f94fa866672394235ae8ab79f360e
S: 02fbee35159f58c854e5931e582d825850033bb4 127.0.0.1:7004
slots: (0 slots) slave
replicates c3b18b156bcbefa24087c667be475755fe2b1488
S: 98121a6c59dbaab55895fc93e8ab117d91ad4780 127.0.0.1:7003
slots: (0 slots) slave
replicates 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592
M: c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001
slots:6827-10922 (4096 slots) master
1 additional replica(s)
M: 2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006
slots:0-1364,5461-6826,10923-12287 (4096 slots) master
1 additional replica(s)
S: 7c58f48395db82555b218af6382822b65797a2dc 127.0.0.1:7005
slots: (0 slots) slave
replicates 9d9d6e0a351f17d679523b55f2a4d6874957240c
M: 9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002
slots:12288-16383 (4096 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 1365
What is the receiving node ID? 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:2713612b892f94fa866672394235ae8ab79f360e
Source node #2:
*** The specified node is not known or is not a master, please retry.
Source node #2:done
Ready to move 1365 slots.
Source nodes:
M: 2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006
slots:0-1364,5461-6826,10923-12287 (4096 slots) master
1 additional replica(s)
Destination node:
M: 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000
slots:1365-5460 (4096 slots) master
1 additional replica(s)
Resharding plan:
Moving slot 0 from 2713612b892f94fa866672394235ae8ab79f360e
Moving slot 1 from 2713612b892f94fa866672394235ae8ab79f360e
。。。
Do you want to proceed with the proposed reshard plan (yes/no)? yes
。。。
[root@152 ~]# redis-cli -p 7000 cluster nodes|grep master|grep 7006
2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006 master - 0 1620653421836 7 connected 5461-6826 10923-12287
[root@152 ~]# redis-cli -p 7000 cluster nodes|grep master
c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001 master - 0 1620653428908 2 connected 6827-10922
2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006 master - 0 1620653426883 7 connected 5461-6826 10923-12287
58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000 myself,master - 0 0 8 connected 0-5460
9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002 master - 0 1620653427893 3 connected 12288-16383
#第二段slot
[root@152 ~]# redis-trib.rb reshard 127.0.0.1:7000
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: e9a6d166740ff2047deed0bbc12cc5b1bbac91a1 127.0.0.1:7007
slots: (0 slots) slave
replicates 2713612b892f94fa866672394235ae8ab79f360e
S: 02fbee35159f58c854e5931e582d825850033bb4 127.0.0.1:7004
slots: (0 slots) slave
replicates c3b18b156bcbefa24087c667be475755fe2b1488
S: 98121a6c59dbaab55895fc93e8ab117d91ad4780 127.0.0.1:7003
slots: (0 slots) slave
replicates 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592
M: c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001
slots:6827-10922 (4096 slots) master
1 additional replica(s)
M: 2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006
slots:5461-6826,10923-12287 (2731 slots) master
1 additional replica(s)
S: 7c58f48395db82555b218af6382822b65797a2dc 127.0.0.1:7005
slots: (0 slots) slave
replicates 9d9d6e0a351f17d679523b55f2a4d6874957240c
M: 9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002
slots:12288-16383 (4096 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 1366
What is the receiving node ID? c3b18b156bcbefa24087c667be475755fe2b1488
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:2713612b892f94fa866672394235ae8ab79f360e
Source node #2:done
。。。
[root@152 ~]# redis-cli -p 7000 cluster nodes|grep master
c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001 master - 0 1620653507508 9 connected 5461-10922
2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006 master - 0 1620653507003 7 connected 10923-12287
58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000 myself,master - 0 0 8 connected 0-5460
9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002 master - 0 1620653505992 3 connected 12288-16383
#第三段slot
[root@152 ~]# redis-trib.rb reshard 127.0.0.1:7000
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: e9a6d166740ff2047deed0bbc12cc5b1bbac91a1 127.0.0.1:7007
slots: (0 slots) slave
replicates 2713612b892f94fa866672394235ae8ab79f360e
S: 02fbee35159f58c854e5931e582d825850033bb4 127.0.0.1:7004
slots: (0 slots) slave
replicates c3b18b156bcbefa24087c667be475755fe2b1488
S: 98121a6c59dbaab55895fc93e8ab117d91ad4780 127.0.0.1:7003
slots: (0 slots) slave
replicates 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592
M: c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: 2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006
slots:10923-12287 (1365 slots) master
1 additional replica(s)
S: 7c58f48395db82555b218af6382822b65797a2dc 127.0.0.1:7005
slots: (0 slots) slave
replicates 9d9d6e0a351f17d679523b55f2a4d6874957240c
M: 9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002
slots:12288-16383 (4096 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 1365
What is the receiving node ID? 9d9d6e0a351f17d679523b55f2a4d6874957240c
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:2713612b892f94fa866672394235ae8ab79f360e
Source node #2:done
。。。
[root@152 ~]# redis-cli -p 7000 cluster nodes|grep master
c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001 master - 0 1620653581864 9 connected 5461-10922
2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006 master - 0 1620653580853 7 connected
58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000 myself,master - 0 0 8 connected 0-5460
9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002 master - 0 1620653581864 10 connected 10923-16383
#以上可以看出，7006上已无slot
2. 删除主从节点¶
#主节点
redis-trib.rb del-node 127.0.0.1:7006 7006id
#从节点
redis-trib.rb del-node 127.0.0.1:7007 7007id
#1.得到7006的id
[root@152 ~]# redis-cli -p 7000 cluster nodes|grep master|grep 7006
2713612b892f94fa866672394235ae8ab79f360e 127.0.0.1:7006 master - 0 1620653754740 7 connected
#2.删除7006
[root@152 ~]# redis-trib.rb del-node 127.0.0.1:7006 2713612b892f94fa866672394235ae8ab79f360e
>>> Removing node 2713612b892f94fa866672394235ae8ab79f360e from cluster 127.0.0.1:7006
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
#3.查看是否删除成功
[root@152 ~]# redis-cli -p 7000 cluster nodes|grep master
c3b18b156bcbefa24087c667be475755fe2b1488 127.0.0.1:7001 master - 0 1620653805973 9 connected 5461-10922
58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 127.0.0.1:7000 myself,master - 0 0 8 connected 0-5460
9d9d6e0a351f17d679523b55f2a4d6874957240c 127.0.0.1:7002 master - 0 1620653804457 10 connected 10923-16383
#4.查看7007的id
[root@152 ~]# redis-cli -p 7000 cluster nodes|grep 7007
e9a6d166740ff2047deed0bbc12cc5b1bbac91a1 127.0.0.1:7007 slave 9d9d6e0a351f17d679523b55f2a4d6874957240c 0 1620653852833 10 connected
#5.删除7007
[root@152 ~]# redis-trib.rb del-node 127.0.0.1:7007 e9a6d166740ff2047deed0bbc12cc5b1bbac91a1
>>> Removing node e9a6d166740ff2047deed0bbc12cc5b1bbac91a1 from cluster 127.0.0.1:7007
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
#4.查看是否删除成功
[root@152 ~]# redis-cli -p 7000 cluster nodes|grep 7007
[root@152 ~]# redis-cli -p 7000 cluster nodes|grep slave
02fbee35159f58c854e5931e582d825850033bb4 127.0.0.1:7004 slave c3b18b156bcbefa24087c667be475755fe2b1488 0 1620654546234 9 connected
98121a6c59dbaab55895fc93e8ab117d91ad4780 127.0.0.1:7003 slave 58ff76bd83c9abb5f848b8ee22c312f9e9fc1592 0 1620654545216 8 connected
7c58f48395db82555b218af6382822b65797a2dc 127.0.0.1:7005 slave 9d9d6e0a351f17d679523b55f2a4d6874957240c 0 1620654544708 10 connected
最后更新:
2022-02-20 08:44:07