基于Docker redis3.2.4 集群 Cluster

xiaoxiao2021-03-25 67

环境搭建步骤

一　准备

Docker环境（centos7 + docker1.12.1）

Redis 3.2.4 wget http://download.redis.io/releases/redis-3.2.4.tar.gz

因为cluster模式最少三个主数据库才能正常运行，所以先做三主三从

172.17.0.3:7000 172.17.0.4:7001 172.17.0.5:7002 172.17.0.6:7003 172.17.0.7:7004 172.17.0.7:7005

另外：搭建完成后，为了验证增加节点的操作，增加如下节点 172.17.0.8:7006 172.17.0.9:7007

遗留问题：

1　cluster setslot 分配slot后，通过cluster slots命令在分配的这台机器看可以看到分配正确，但是在其它节点查看时还是旧的分配情况

2 客户端代码如何连接集群

3 节点宕机后测试(已测试，对应的从节点会接管主节点，主节点故障修复后启动变成这个新的主节点的从数据库)

二　基础镜像制作

Dockerfile如下：

FROM centos:6.7 MAINTAINER loomz loomz@163.com ENV REFRESHED_AT 2017-03-08 ENV REDIS_HOME /opt/redis/redis_default ADD redis-3.2.4.tar.gz /opt/redis/ RUN ln -s /opt/redis/redis-3.2.4 $REDIS_HOME RUN yum -y install gcc WORKDIR $REDIS_HOME RUN make && make install EXPOSE 6379 ENTRYPOINT [ "/usr/local/bin/redis-server", "/etc/redis/redis.conf" ]

相关命令: docker命令总结

三　容器启动

配置文件内容：redis.conf

port 6379 bind 172.17.0.5 daemonize no pidfile "/var/run/redis.pid" cluster-enabled yes cluster-config-file nodes.conf cluster-node-timeout 15000 appendonly yes

容器启动脚本start-7000.sh：

#!/bin/bash docker run -d -p 7000:6379 --restart always -h redis-7000 --name redis-7000 -v /home/loomz/dockerfiles/redis_3.2_cluster/redis_cluster/7000:/etc/redis/:rw redis:v3.2.4 start-7001.sh start-7002.sh start-7003.sh start-7004.sh start-7005.sh　依此类推

启动所有容器后，节点并未形成集群，docker logs redis-7000，内容显示：

No cluster configuration found, I'm 0435ad36e3c8a12b4850e0d472cc07d5b3854b11

四　创建集群

1 创建集群需要用的ruby脚本，先安装ruby 和　rubygems，进入其中一台容器创建

docker exec -it redis-7000 /bin/bash yum install -y ruby rubygems ruby -v

2 创建集群节点

在redis编译源码包的src目录，将脚本拷贝到/usr/local/bin目录

cp redis-trib.rb /usr/local/bin

创建节点命令：

redis-trib.rb create --replicas 1 172.17.0.5 6379 172.17.0.6 6379 172.17.0.7 6379 172.17.0.8 6379 172.17.0.9 6379 172.17.0.10 6379

输入命令后会展示集群分配情况，输入yes确认，最终节点分配以及slot分配情况如下：

S: 4c98b473b20ca9487b6cd042015269aaedb550dc 172.17.0.5:6379 slots: (0 slots) slave replicates c469f66da77cd61fe8aef25e553e5a388e5b32bb S: cd289a8c8d618fd8f8918bbae59264e3a5a27403 172.17.0.6:6379 slots: (0 slots) slave replicates fa0a9e0fbc24ddb871c09ca7534bc2f170c77fc8 M: c469f66da77cd61fe8aef25e553e5a388e5b32bb 172.17.0.8:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s) M: fa0a9e0fbc24ddb871c09ca7534bc2f170c77fc8 172.17.0.9:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s) M: c0ab7e8c1934b51dc03767c8866bc4dcc9ace93d 172.17.0.10:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) S: 0435ad36e3c8a12b4850e0d472cc07d5b3854b11 172.17.0.7:6379 slots: (0 slots) slave replicates c0ab7e8c1934b51dc03767c8866bc4dcc9ace93d

3 也可以登录其中一台客户端验证

redis-cli > cluster nodes cd289a8c8d618fd8f8918bbae59264e3a5a27403 172.17.0.6:6379 slave fa0a9e0fbc24ddb871c09ca7534bc2f170c77fc8 0 1489315757449 5 connected c469f66da77cd61fe8aef25e553e5a388e5b32bb 172.17.0.8:6379 master - 0 1489315758452 4 connected 10923-16383 fa0a9e0fbc24ddb871c09ca7534bc2f170c77fc8 172.17.0.9:6379 master - 0 1489315760456 5 connected 5461-10922 c0ab7e8c1934b51dc03767c8866bc4dcc9ace93d 172.17.0.10:6379 master - 0 1489315761456 6 connected 0-5460 4c98b473b20ca9487b6cd042015269aaedb550dc 172.17.0.5:6379 myself,slave c469f66da77cd61fe8aef25e553e5a388e5b32bb 0 0 1 connected 0435ad36e3c8a12b4850e0d472cc07d5b3854b11 172.17.0.7:6379 slave c0ab7e8c1934b51dc03767c8866bc4dcc9ace93d 0 1489315759454 6 connected

五　增加节点

命令：cluster meet　集群中任意节点和端口

redis-cli > cluster meet 172.17.0.10 6379 redis-cli > cluster nodes cd289a8c8d618fd8f8918bbae59264e3a5a27403 172.17.0.6:6379 slave fa0a9e0fbc24ddb871c09ca7534bc2f170c77fc8 0 1489319127890 5 connected 7ff58bf488f4d6a1062c106f018ae41e28164b28 172.17.0.11:6379 myself,master - 0 0 0 connected c0ab7e8c1934b51dc03767c8866bc4dcc9ace93d 172.17.0.10:6379 master - 0 1489319128893 6 connected 0-5460 0435ad36e3c8a12b4850e0d472cc07d5b3854b11 172.17.0.7:6379 slave c0ab7e8c1934b51dc03767c8866bc4dcc9ace93d 0 1489319126888 6 connected c469f66da77cd61fe8aef25e553e5a388e5b32bb 172.17.0.8:6379 master - 0 1489319124884 4 connected 10923-16383 4c98b473b20ca9487b6cd042015269aaedb550dc 172.17.0.5:6379 slave c469f66da77cd61fe8aef25e553e5a388e5b32bb 0 1489319125084 4 connected fa0a9e0fbc24ddb871c09ca7534bc2f170c77fc8 172.17.0.9:6379 master - 0 1489319125886 5 connected 5461-10922

增加的节点172.17.0.11 默认变成master但是没有分配到slot

redis-cli > cluster slots 1) 1) (integer) 0 2) (integer) 5460 3) 1) "172.17.0.10" 2) (integer) 6379 3) "c0ab7e8c1934b51dc03767c8866bc4dcc9ace93d" 4) 1) "172.17.0.7" 2) (integer) 6379 3) "0435ad36e3c8a12b4850e0d472cc07d5b3854b11" 2) 1) (integer) 10923 2) (integer) 16383 3) 1) "172.17.0.8" 2) (integer) 6379 3) "c469f66da77cd61fe8aef25e553e5a388e5b32bb" 4) 1) "172.17.0.5" 2) (integer) 6379 3) "4c98b473b20ca9487b6cd042015269aaedb550dc" 3) 1) (integer) 5461 2) (integer) 10922 3) 1) "172.17.0.9" 2) (integer) 6379 3) "fa0a9e0fbc24ddb871c09ca7534bc2f170c77fc8" 4) 1) "172.17.0.6" 2) (integer) 6379 3) "cd289a8c8d618fd8f8918bbae59264e3a5a27403" 可以使用以下两种方式，这里不再细说：

/usr/local/bin/redis-trib.rb reshard　命令重新分配slot，支持数据迁移

cluster setslot 5460 node 7ff58bf488f4d6a1062c106f018ae41e28164b28　　　数据迁移需要手动

增加节点设置为某个主节点的从数据库

cluster nodes

CLUSTER REPLICATE <node_id> 将当前节点设置为 node_id 指定的节点的从节点。

客户端连接代码

public static void main(String[] args) { JedisPoolConfig poolconfig = new JedisPoolConfig(); poolconfig.setMaxIdle(30); poolconfig.setMaxTotal(1000); poolconfig.setMaxWaitMillis(3000); Set<HostAndPort> nodes = new HashSet<HostAndPort>(); HostAndPort hostAndPort = new HostAndPort("172.17.0.1", 7000); HostAndPort hostAndPort1 = new HostAndPort("172.17.0.1", 7001); HostAndPort hostAndPort2 = new HostAndPort("172.17.0.1", 7002); HostAndPort hostAndPort3 = new HostAndPort("172.17.0.1", 7003); HostAndPort hostAndPort4 = new HostAndPort("172.17.0.1", 7004); HostAndPort hostAndPort5 = new HostAndPort("172.17.0.1", 7005); HostAndPort hostAndPort6 = new HostAndPort("172.17.0.1", 7006); HostAndPort hostAndPort7 = new HostAndPort("172.17.0.1", 7007); nodes.add(hostAndPort); nodes.add(hostAndPort1); nodes.add(hostAndPort2); nodes.add(hostAndPort3); nodes.add(hostAndPort4); nodes.add(hostAndPort5); nodes.add(hostAndPort6); nodes.add(hostAndPort7); JedisCluster jedisCluster = new JedisCluster(nodes, poolconfig); String string = jedisCluster.get("aa"); System.out.println(string); }

六　集群相关命令

cluster集群相关命令,更多redis相关命令见文档:http://redis.readthedocs.org/en/latest 集群 CLUSTER INFO 打印集群的信息 CLUSTER NODES 列出集群当前已知的所有节点（node），以及这些节点的相关信息。节点 CLUSTER MEET <ip> <port> 将 ip 和 port 所指定的节点添加到集群当中，让它成为集群的一份子。 CLUSTER FORGET <node_id> 从集群中移除 node_id 指定的节点。 CLUSTER REPLICATE <node_id> 将当前节点设置为 node_id 指定的节点的从节点。 CLUSTER SAVECONFIG 将节点的配置文件保存到硬盘里面。槽(slot) CLUSTER ADDSLOTS <slot> [slot ...] 将一个或多个槽（slot）指派（assign）给当前节点。 CLUSTER DELSLOTS <slot> [slot ...] 移除一个或多个槽对当前节点的指派。 CLUSTER FLUSHSLOTS 移除指派给当前节点的所有槽，让当前节点变成一个没有指派任何槽的节点。 CLUSTER SETSLOT <slot> NODE <node_id> 将槽 slot 指派给 node_id 指定的节点，如果槽已经指派给另一个节点，那么先让另一个节点删除该槽>，然后再进行指派。 CLUSTER SETSLOT <slot> MIGRATING <node_id> 将本节点的槽 slot 迁移到 node_id 指定的节点中。 CLUSTER SETSLOT <slot> IMPORTING <node_id> 从 node_id 指定的节点中导入槽 slot 到本节点。 CLUSTER SETSLOT <slot> STABLE 取消对槽 slot 的导入（import）或者迁移（migrate）。键 CLUSTER KEYSLOT <key> 计算键 key 应该被放置在哪个槽上。 CLUSTER COUNTKEYSINSLOT <slot> 返回槽 slot 目前包含的键值对数量。 CLUSTER GETKEYSINSLOT <slot> <count> 返回 count 个 slot 槽中的键。

其它：

疑问１信息

172.17.0.8:6379> cluster slots 1) 1) (integer) 0 2) (integer) 5459 3) 1) "172.17.0.10" 2) (integer) 6379 3) "c0ab7e8c1934b51dc03767c8866bc4dcc9ace93d" 4) 1) "172.17.0.7" 2) (integer) 6379 3) "0435ad36e3c8a12b4850e0d472cc07d5b3854b11" 2) 1) (integer) 10923 2) (integer) 16383 3) 1) "172.17.0.5" 2) (integer) 6379 3) "4c98b473b20ca9487b6cd042015269aaedb550dc" 4) 1) "172.17.0.8" 2) (integer) 6379 3) "c469f66da77cd61fe8aef25e553e5a388e5b32bb" 3) 1) (integer) 5460 2) (integer) 5461 3) 1) "172.17.0.11" 2) (integer) 6379 3) "7ff58bf488f4d6a1062c106f018ae41e28164b28" 4) 1) "172.17.0.12" 2) (integer) 6379 3) "a36dbb0895a24d82f8cc6bb2c7af1adb6432a7da" 4) 1) (integer) 5462 2) (integer) 10922 3) 1) "172.17.0.9" 2) (integer) 6379 3) "fa0a9e0fbc24ddb871c09ca7534bc2f170c77fc8" 4) 1) "172.17.0.6" 2) (integer) 6379 3) "cd289a8c8d618fd8f8918bbae59264e3a5a27403" 172.17.0.8:6379> exit [root@redis-7003 redis-3.2.4]# redis-cli -h 172.17.0.9 172.17.0.9:6379> 172.17.0.9:6379> cluster slots 1) 1) (integer) 5461 2) (integer) 5461 3) 1) "172.17.0.11" 2) (integer) 6379 3) "7ff58bf488f4d6a1062c106f018ae41e28164b28" 4) 1) "172.17.0.12" 2) (integer) 6379 3) "a36dbb0895a24d82f8cc6bb2c7af1adb6432a7da" 2) 1) (integer) 5462 2) (integer) 10922 3) 1) "172.17.0.9" 2) (integer) 6379 3) "fa0a9e0fbc24ddb871c09ca7534bc2f170c77fc8" 4) 1) "172.17.0.6" 2) (integer) 6379 3) "cd289a8c8d618fd8f8918bbae59264e3a5a27403" 3) 1) (integer) 10923 2) (integer) 16383 3) 1) "172.17.0.5" 2) (integer) 6379 3) "4c98b473b20ca9487b6cd042015269aaedb550dc" 4) 1) "172.17.0.8" 2) (integer) 6379 3) "c469f66da77cd61fe8aef25e553e5a388e5b32bb" 4) 1) (integer) 0 2) (integer) 5460 3) 1) "172.17.0.10" 2) (integer) 6379 3) "c0ab7e8c1934b51dc03767c8866bc4dcc9ace93d" 4) 1) "172.17.0.7" 2) (integer) 6379 3) "0435ad36e3c8a12b4850e0d472cc07d5b3854b11"

转载请注明原文地址: https://ju.6miu.com/read-36632.html

技术

最新回复(0)