前一篇提到了容器ip变化后集群自发现的问题，现在接着讲这个问题引申出的另两个问题：

1.虽然集群ip问题解决了，但java client还是连接报错，似乎连的是老的ip地址。

2.当key所在的master挂了，slave切换到master后，java客户端却一直尝试重连，直到超时，并没有随着master的切换去连主拿key。以下代码和例子为环境模拟。

集群信息：

192.168.27.128:7000> cluster nodes
ffd1619ad6b8f8db6d5764ab6739f4b03661cbe3 192.168.27.128:7001@17001 master - 0 1568640842000 70 connected 11404 11589-16383
8ca3773af76228ac6a864ef660a7e701d207aa3c 192.168.27.129:7002@17002 master - 0 1568640844271 35 connected 666-5460
a92e5ab3de251dd49a25c313380fb2ce8a81513c 192.168.27.128:7003@17003 slave a2d327c3be89c685524fe12e26b8e7227b500e20 0 1568640842000 37 connected
8c81e214847a2e0ed609a432f15f8ef048cff047 192.168.27.129:7001@17001 slave ffd1619ad6b8f8db6d5764ab6739f4b03661cbe3 0 1568640842355 70 connected
7e0686f3bec7c103a794aa6402c7b9ad0f6c632e 192.168.27.128:7000@17000 myself,slave 8ca3773af76228ac6a864ef660a7e701d207aa3c 0 1568640842000 1 connected
88ecdc0b150b3cbfdc4d791204bfd0d58cabd2ff 192.168.27.129:7003@17003 master - 0 1568640842264 8 connected 0-665 5461-6127 10923-11403 11405-11588
a2d327c3be89c685524fe12e26b8e7227b500e20 192.168.27.129:7000@17000 master - 0 1568640843260 37 connected 6128-10922

模拟客户端程序，循环插入key和获取key的值：

@Test
public void redisClusterTest() throws InterruptedException {
        ValueOperations<String,String> operations=redisTemplate.opsForValue();
        int i =0;
        while (true){         
                operations.set("mchf","mchf");        
                System.out.println(operations.get("mchf"));
                i++;
                System.out.println(i);
            }catch (Exception e){
                e.printStackTrace();
            }

            Thread.sleep(2000);
        }

获取key所在的slot和节点：

192.168.27.128:7000> get mchf
(error) MOVED 11404 192.168.27.128:7001

我们kill掉192.168.27.128:7001这个redis实例，看看客户端会发生什么：

3
mchf
4
2019-09-16 21:37:42.393  INFO 3612 --- [xecutorLoop-1-3] i.l.core.protocol.ConnectionWatchdog     : Reconnecting, last destination was /192.168.27.128:7001
2019-09-16 21:37:44.400  WARN 3612 --- [ioEventLoop-6-4] i.l.core.protocol.ConnectionWatchdog     : Cannot reconnect: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: /192.168.27.128:7001
2019-09-16 21:37:48.693  INFO 3612 --- [xecutorLoop-1-5] i.l.core.protocol.ConnectionWatchdog     : Reconnecting, last destination was 192.168.27.128:7001
2019-09-16 21:37:50.698  WARN 3612 --- [ioEventLoop-6-1] i.l.core.protocol.ConnectionWatchdog     : Cannot reconnect: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: /192.168.27.128:7001
2019-09-16 21:37:54.992  INFO 3612 --- [xecutorLoop-1-2] i.l.core.protocol.ConnectionWatchdog     : Reconnecting, last destination was 192.168.27.128:7001
2019-09-16 21:37:57.000  WARN 3612 --- [ioEventLoop-6-4] i.l.core.protocol.ConnectionWatchdog     : Cannot reconnect: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: /192.168.27.128:7001
2019-09-16 21:38:02.091  INFO 3612 --- [xecutorLoop-1-5] i.l.core.protocol.ConnectionWatchdog     : Reconnecting, last destination was 192.168.27.128:7001
2019-09-16 21:38:04.096  WARN 3612 --- [ioEventLoop-6-1] i.l.core.protocol.ConnectionWatchdog     : Cannot reconnect: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: /192.168.27.128:7001
2019-09-16 21:38:09.292  INFO 3612 --- [xecutorLoop-1-3] i.l.core.protocol.ConnectionWatchdog     : Reconnecting, last destination was 192.168.27.128:7001
2019-09-16 21:38:11.298  WARN 3612 --- [ioEventLoop-6-3] i.l.core.protocol.ConnectionWatchdog     : Cannot reconnect: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: /192.168.27.128:7001
org.springframework.dao.QueryTimeoutException: Redis command timed out; nested exception is io.lettuce.core.RedisCommandTimeoutException: Command timed out after 30 second(s)

这边一直尝试重连，超时抛出异常。这样当slave提升到master后，客户端还在尝试连接原来的master，而原来的master已经被kill掉了，客户端并没有去连接新的master。

我们是基于springboot2.0做的开发，而springboot2.0中redis的连接池用的是lettuce，查文档后发现，需要开启自适应或定期刷新ClusterTopology，以下为修改后的代码：

  @Autowired
    private RedisProperties redisProperties;


    @Bean
    public LettuceConnectionFactory redisConnectionFactory() {

        RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration(redisProperties.getCluster().getNodes());


        GenericObjectPoolConfig genericObjectPoolConfig = new GenericObjectPoolConfig();
        //支持自适应集群拓扑刷新和静态刷新源
        ClusterTopologyRefreshOptions clusterTopologyRefreshOptions =  ClusterTopologyRefreshOptions.builder()
               //.enablePeriodicRefresh(Duration.ofSeconds(5))
                .enableAllAdaptiveRefreshTriggers()
                .adaptiveRefreshTriggersTimeout(Duration.ofSeconds(10))
                //.enablePeriodicRefresh(Duration.ofSeconds(10))
                .build();

        ClusterClientOptions clusterClientOptions = ClusterClientOptions.builder().timeoutOptions(TimeoutOptions.enabled(Duration.ofSeconds(30)))  超时修改为30秒
                //.autoReconnect(false)  是否自动重连
                //.pingBeforeActivateConnection(Boolean.TRUE) 
                //.cancelCommandsOnReconnectFailure(Boolean.TRUE)
                //.disconnectedBehavior(ClientOptions.DisconnectedBehavior.REJECT_COMMANDS)
                .topologyRefreshOptions(clusterTopologyRefreshOptions).build();

        LettuceClientConfiguration lettuceClientConfiguration = LettucePoolingClientConfiguration.builder()
                .poolConfig(genericObjectPoolConfig)
                //.readFrom(ReadFrom.NEAREST)
                .clientOptions(clusterClientOptions).build();

        return new LettuceConnectionFactory(redisClusterConfiguration, lettuceClientConfiguration);
    }


    @Bean
    public RedisTemplate<String,String> redisTemplate(RedisConnectionFactory redisConnectionfactory){
        RedisTemplate<String,String> redisTemplate=new RedisTemplate<>();
        redisTemplate.setConnectionFactory(redisConnectionfactory);
        redisTemplate.setKeySerializer(new StringRedisSerializer());
        redisTemplate.setValueSerializer(new StringRedisSerializer());
        redisTemplate.afterPropertiesSet();
        return redisTemplate;
    }

这边需要注意两点：

1.默认客户端重连超时时间为60秒，这边修改到了30秒，redis集群认为master宕机slave切换到master的默认时间为15秒，可以在redis.conf中修改

2.启用adaptive cluster topology view当某个节点无法连接上之后，会生效，并且会和集群中所有的节点每隔一段时间尝试建立ESTABLISH连接，时间间隔取决于代码中的刷新频率，而启用periodic cluster topology不管集群有没有发生改变，都会每隔一段时间和集群中所有节点尝试建立ESTABLISH连接。集群使用客户端比较多的时候需要注意这点。

代码修改后可以从变化后的master拿到key：

前面发生了主从切换所以现在192.168.27.129:7001是主，我们kill掉，查看客户端变化

192.168.27.128:7000> get mchf
(error) MOVED 11404 192.168.27.129:7001

Caused by: io.lettuce.core.RedisCommandTimeoutException: Command timed out after 30 second(s)
	at io.lettuce.core.ExceptionFactory.createTimeoutException(ExceptionFactory.java:51)
	at io.lettuce.core.LettuceFutures.awaitOrCancel(LettuceFutures.java:114)
	at io.lettuce.core.cluster.ClusterFutureSyncInvocationHandler.handleInvocation(ClusterFutureSyncInvocationHandler.java:123)
	at io.lettuce.core.internal.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:80)
	at com.sun.proxy.$Proxy85.set(Unknown Source)
	at org.springframework.data.redis.connection.lettuce.LettuceStringCommands.set(LettuceStringCommands.java:146)
	... 39 more
mchf
5
2019-09-16 22:07:05.372  INFO 13520 --- [xecutorLoop-1-1] i.l.core.protocol.ConnectionWatchdog     : Reconnecting, last destination was 192.168.27.129:7001
mchf
6

当配置自适应或定期刷新cluster topology后，上面两个问题得以解决。

参考：https://github.com/lettuce-io/lettuce-core/wiki/Redis-Cluster