在宿主机的springboot项目访问docker中的redis集群出错
描述:在docker中部署了3个节点的cluster,使用java客户端连接,测试java客户端连接超时。@Beanpublic JedisCluster JedisClusterFactory() {Set<HostAndPort> jedisClusterNodes = new HashSet<HostAndPort>();...
描述:在docker中部署了3个节点的cluster,使用java客户端连接,测试java客户端连接超时。
@Bean
public JedisCluster JedisClusterFactory() {
Set<HostAndPort> jedisClusterNodes = new HashSet<HostAndPort>();
jedisClusterNodes.add(new HostAndPort("localhost", 6380));
jedisClusterNodes.add(new HostAndPort("localhost", 6381));
jedisClusterNodes.add(new HostAndPort("localhost", 6382));
JedisCluster jedisCluster = new JedisCluster(jedisClusterNodes);
return jedisCluster;
}
docker环境如下:
KumasMBA:~ kumas$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f3a8ac6612a6 redis:3.2-alpine "docker-entrypoint..." 2 days ago Up 3 hours 0.0.0.0:6382->6382/tcp redis03
f43c6d1ae35e redis:3.2-alpine "docker-entrypoint..." 2 days ago Up 3 hours 0.0.0.0:6381->6381/tcp redis02
836c324c3dc1 redis:3.2-alpine "docker-entrypoint..." 2 days ago Up 3 hours 0.0.0.0:6380->6380/tcp redis01
分析:debug后发现,客户端配置的localhost信息,JedisCluster客户端通过cluster nodes获取集群节点信息
127.0.0.1:6380> cluster nodes
2ed694f1727c992e82e83d81ab9c20771d5a4bd0 192.168.0.11:6381 master - 0 1534163329853 2 connected 6001-12000
45ee887bee3e9ef256a05cbc8e4c69ca84d5837a 192.168.0.15:6385 slave,fail 4eff07974aae139af1432d33a23c25b598334cbe 1534162704450 1534162704450 5 connected
3e8c33f7e3c46bc56cdd8342589fa1e55668be14 192.168.0.14:6384 slave,fail 2ed694f1727c992e82e83d81ab9c20771d5a4bd0 1534162704449 1534162704449 4 connected
66c1333eaec58a43e17702970906c3c2d94ca91a 192.168.0.13:6383 slave,fail 2527f88935b78ae2d26a7187bc35473db876a241 1534162704449 1534162704449 1 connected
2527f88935b78ae2d26a7187bc35473db876a241 192.168.0.10:6380 myself,master - 0 0 1 connected 0-6000
4eff07974aae139af1432d33a23c25b598334cbe 192.168.0.12:6382 master - 0 1534163329338 3 connected 12001-16383
解析节点信息后将节点存入map,也就是最初配置的set被替换了(具体代码看源码):
public static final int SLOT_INFORMATIONS_START_INDEX = 8;
public static final int HOST_AND_PORT_INDEX = 1;
public ClusterNodeInformation parse(String nodeInfo, HostAndPort current) {
String[] nodeInfoPartArray = nodeInfo.split(" ");
HostAndPort node = getHostAndPortFromNodeLine(nodeInfoPartArray, current);
ClusterNodeInformation info = new ClusterNodeInformation(node);
if (nodeInfoPartArray.length >= SLOT_INFORMATIONS_START_INDEX) {
String[] slotInfoPartArray = extractSlotParts(nodeInfoPartArray);
fillSlotInformation(slotInfoPartArray, info);
}
return info;
}
关键信息:String[] slotInfoPartArray = extractSlotParts(nodeInfoPartArray);
获取slot槽分配的区间值,比如0-5460。
parse函数中另一个关键信息:fillSlotInformation(slotInfoPartArray, info);
fillSlotInformation函数调用了fillSlotInformationFromSlotRange函数,fillSlotInformationFromSlotRange函数作用是把所有的slot槽的index值存放到addAvailableSlot集合中。
初始化核心工作:把当前节点存放入nodes集合中,key为节点host:port,value为JedisPool实例;把slot槽index索引值与当前节点的JedisPool进行映射,存入Map
关键信息:slots.put(slot, targetPool);实现slot槽index索引值与当前节点的JedisPool进行映射
解决思路:网上有pipework的解决方案是最符合预期的,由于脚本针对mac系统需要做较大的调整,等有精力再研究了。先留作备忘,待解决。
如果只是测试,有一个思路(尚未验证,理论上应该可行)主从节点都在一个容器内配置,这样cluster nodes返回的ip都是127.0.0.1。这样即使返回客户端的ip为127.0.0.1再次连接时也不会连不上了。
2ed694f1727c992e82e83d81ab9c20771d5a4bd0 127.0.0.1:6381 master - 0 1530144760135 2 connected 6001-12000
2527f88935b78ae2d26a7187bc35473db876a241 127.0.0.1:6380 master - 0 1530144760014 1 connected 0-6000
4eff07974aae139af1432d33a23c25b598334cbe 127.0.0.1:6382 master - 0 1530144760135 3 connected 12001-16383
45ee887bee3e9ef256a05cbc8e4c69ca84d5837a 127.0.0.1:6385 myself,slave 4eff07974aae139af1432d33a23c25b598334cbe 0 0 5 connected
3e8c33f7e3c46bc56cdd8342589fa1e55668be14 127.0.0.1:6384 slave 2ed694f1727c992e82e83d81ab9c20771d5a4bd0 0 1530144760135 4 connected
66c1333eaec58a43e17702970906c3c2d94ca91a 127.0.0.1:6383 slave 2527f88935b78ae2d26a7187bc35473db876a241 0 1530144760136 1 connected
PS:上述测试思路已经验证可行。
先自定义entrypoint脚本,如下:保存为cluster_in_one.sh。
#!/bin/sh
#注意配置文件daemonized要设为true。
redis-server /data/conf/redis6380.conf && redis-server /data/conf/redis6381.conf && redis-server /data/conf/redis6382.conf && redis-server /data/conf/redis6383.conf && redis-server /data/conf/redis6384.conf && redis-server /data/conf/redis6385.conf
将上述脚本映射为docker-entrypoint.sh
docker run -d --name rediscls --hostname rediscls --expose 6380-6385 -p 6380-6385:6380-6385 -v /Users/kumas/Applications/Docker/etc/localtime:/etc/localtime:ro -v /Users/kumas/Applications/Docker/redis/cluster_in_one:/data -v /Users/kumas/Applications/Docker/redis/cluster_in_one/cluster_in_one.sh:/usr/local/bin/docker-entrypoint.sh redis:3.2-alpine
望遇到此问题的同志们能给出更好的解决方案。
------------PS:20180819------------------
借鉴kafkaclient访问docker中kafka集群的经验,redis的内网网使用hostname方式分别配置不同的ip而且不要使用回路ip(即127.0.0.1)比如:
内网:
192.168.0.10 redis01
192.168.0.11 redis02
192.168.0.12 redis03
外网宿主机:
10.10.10.10 redis01
10.10.10.10 redis02
10.10.10.10 redis03
注:10.10.10.10为宿主机真实网卡配的ip。
只是一个思路,未经验证。有时间再去验证。
更多推荐
所有评论(0)