最近在用spark消费kafka数据时,在集群上跑着跑着程序就挂掉了。报的错误是

Could not initialize class org.xerial.snappy.Snappy

在网上查了下,报错的原因是snappy将.so文件解压到了/tmp目录下,如果/tmp目录因为种种原因撑爆了,那就gg了。

This is generally caused by by snappy library extracting to /tmp (generally defined in java.io.tmpdir)

解决方法链接

Create a tmp directory where nifi user has access. For example, create tmp directory /home/tmp
Then go into ambari and find parameter
Template for bootstrap.conf
Scroll past all the java.arg.* arguments
Add the following
java.arg.snappy=-Dorg.xerial.snappy.tempdir=/usr/hdf/current/nifi/tmp
Logo

为开发者提供学习成长、分享交流、生态实践、资源工具等服务,帮助开发者快速成长。

更多推荐