springboot Too many open files错误 打开文件数一直增加,已解决
问题描述:前端框架VUE,请求插件axios,后端框架springboot。操作:在页面中请求springboot的controller接口,在centos7中使用lsof -p 1605 | wc -l命令查询打开文件数每请求一次,查询打开文件数的结果就会多出48个文件,一直到4000多的时候,就会报tomcat Too many open files异常,而且打开文件数只会上升不会下降,本人觉
问题描述:
前端框架VUE,请求插件axios,后端框架springboot。
操作:
在页面中请求springboot的controller接口,在centos7中使用lsof -p 1605 | wc -l命令查询打开文件数
每请求一次,查询打开文件数的结果就会多出48个文件,一直到4000多的时候,就会报tomcat Too many open files异常,而且打开文件数只会上升不会下降,本人觉得原因应该是socket没关闭吧,但是代码看着没有任何问题,不知道有没有人遇到过这类的问题
解决方法:
**
https://blog.csdn.net/yangleiGJ/article/details/103121278
**
场景:
本地从kafka获取数据,模拟设备,通过808协议给第三方平台发送数据,一个设备一个连接。
问题:
使用netty做性能测试时,并发过大造成Too Many open files问题
每个设备对象,内部就一个netty对象请求,结果1000多个设备,使用服务器句柄达到了17w+,
解决如下: 最后通过修改bootstrap.group(new NioEventLoopGroup(1));然后句柄数据正常了。之前使用默认的new NioEventLoopGroup()
查看进程对应的句柄数
[root@xxxxx]# lsof -n|awk ‘{print $2}’|sort|uniq -c|sort -nr|more|grep ‘36576’
6619 36576
查看对应的连接
[root@xxxxx]# netstat -lnaop|grep 7008|wc -l
17
查看进程对应线程数
[root@xxxxx]# cat /proc/36576/status|grep ‘Threads’
Threads: 4323
这还不是最终的解决方法,解决方法是用连接池代替netty的connect放回的future.
PooledConnection
package com.siblings.iredis.client.netty;
import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import lombok.Data;
@Data
public class PooledConnection {
private ChannelFuture future;
private boolean isBusy;
public PooledConnection() {
}
public PooledConnection(ChannelFuture future) {
this.future = future;
this.isBusy = true;
}
public Channel getChannel(){
return future.channel();
}
}
Connection
package com.siblings.iredis.client.netty;
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.LineBasedFrameDecoder;
import io.netty.handler.codec.string.StringDecoder;
import io.netty.handler.codec.string.StringEncoder;
import java.util.ArrayList;
import java.util.List;
public class Connection {
private static List<PooledConnection> list = new ArrayList<>();
public static PooledConnection connect(String ip, int port) {
PooledConnection free = match();
if (free == null) {
ChannelFuture future = getCon(ip, port);
if (future != null) {
free = new PooledConnection(future);
list.add(free);
}
}
return free;
}
public static PooledConnection match() {
for (PooledConnection pool : list) {
if (!pool.isBusy()) {
pool.setBusy(true);
return pool;
}
}
return null;
}
public static ChannelFuture getCon(String ip, int port) {
EventLoopGroup group = new NioEventLoopGroup(1);
final Bootstrap bootstrap = new Bootstrap()
.channel(NioSocketChannel.class)
.handler(new ChannelInitializer<Channel>() {
@Override
protected void initChannel(Channel channel) throws Exception {
channel.pipeline()
.addLast(new LineBasedFrameDecoder(10240))
.addLast(new StringDecoder())
.addLast(new StringEncoder());
}
})
.group(group);
try {
ChannelFuture f = bootstrap.connect(ip, port).sync();
return f;
} catch (InterruptedException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
}
netty中如果无限制的创建客户端,netty客户端的channel调用close方法后nio线程的打开文件不会释放,原因还在查找中。
原因找到了,跟工作线程有关系,由于每次创建netty客户端,我都在代码中创建新的EventLoopGroup,导致线程的打开文件数不断上升。
错误代码:
public static ChannelFuture getCon(String ip, int port) {
EventLoopGroup group = new NioEventLoopGroup(1);
final Bootstrap bootstrap = new Bootstrap()
.channel(NioSocketChannel.class)
.handler(new ChannelInitializer<Channel>() {
@Override
protected void initChannel(Channel channel) throws Exception {
channel.pipeline()
.addLast(new LineBasedFrameDecoder(10240))
.addLast(new StringDecoder())
.addLast(new StringEncoder());
}
})
.group(group);
try {
ChannelFuture f = bootstrap.connect(ip, port).sync();
return f;
} catch (InterruptedException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
正确代码:
private static EventLoopGroup group = new NioEventLoopGroup(1);
public static ChannelFuture getCon(String ip, int port) {
final Bootstrap bootstrap = new Bootstrap()
.channel(NioSocketChannel.class)
.handler(new ChannelInitializer<Channel>() {
@Override
protected void initChannel(Channel channel) throws Exception {
channel.pipeline()
.addLast(new LineBasedFrameDecoder(10240))
.addLast(new StringDecoder())
.addLast(new StringEncoder());
}
})
.group(group);
try {
ChannelFuture f = bootstrap.connect(ip, port).sync();
return f;
} catch (InterruptedException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
更多推荐
所有评论(0)