一.查看系统日志

目前服务器上的Tomcat服务经常出现宕机的情况,想通过查看系统日志/var/log/messages来确定问题,但是系统日志没有开启记录,故执行以下语句重启日志服务

sudo /etc/init.d/rsyslog restart

等待下一次的问题出现后,查看日志文件内容

vim /var/log/messages

摘取主要内容如下:

Apr 15 23:09:41 water abrtd: Executable '/usr/java/jdk1.8.0_131/bin/java' doesn't belong to any package and ProcessUnpackaged is set to 'no'
Apr 15 23:09:41 water abrtd: 'post-create' on '/var/spool/abrt/ccpp-2020-04-15-23:09:38-26521' exited with 1
Apr 15 23:09:41 water abrtd: Deleting problem directory '/var/spool/abrt/ccpp-2020-04-15-23:09:38-26521'
Apr 16 09:17:59 water abrtd: Got signal 15, exiting
Apr 16 09:18:00 water abrtd: Init complete, entering main loop
Apr 16 09:24:19 water abrt[30511]: Saved core dump of pid 30214 (/usr/java/jdk1.8.0_131/bin/java) to /var/spool/abrt/ccpp-2020-04-16-09:24:16-30214 (1048576000 bytes)
Apr 16 09:24:19 water abrtd: Directory 'ccpp-2020-04-16-09:24:16-30214' creation detected
Apr 16 09:24:19 water abrtd: sosreport run failed with exit code 1, log follows:
Apr 16 09:24:19 water abrtd: Traceback (most recent call last):
Apr 16 09:24:19 water abrtd:   File "/usr/sbin/sosreport", line 20, in <module>
Apr 16 09:24:19 water abrtd:     from sos.sosreport import main
Apr 16 09:24:19 water abrtd: ImportError: No module named sos.sosreport

二.确定问题

主要定位系统日志的第一行

Apr 15 23:09:41 water abrtd: Executable '/usr/java/jdk1.8.0_131/bin/java' doesn't belong to any package and ProcessUnpackaged is set to 'no'

字面意思为没有属于任何一个包,并且ProcessUnpackaged被设置成了no.
通过查阅资料得知深层次原因为无法创建ccpp文件导致的,可以修改/etc/abrt/abrt-action-save-package-data.conf文件中的ProcessUnpackaged参数为yes,

vim /etc/abrt/abrt-action-save-package-data.conf

文件内容如下:

# With this option set to "yes",
# only crashes in signed packages will be analyzed.
# the list of public keys used to check the signature is
# in the file gpg_keys
#
# How can I check the GPG key used to sign an installed pacakge on
# Red hat Enterprise Linux:
# https://access.redhat.com/solutions/1120013
#
# Product Signing (GPG) Keys:
# https://access.redhat.com/security/team/key
#
OpenGPGCheck = yes

# Blacklisted packages
#
BlackList = nspluginwrapper, valgrind, strace, mono-core

# Process crashes in executables which do not belong to any package?
#
ProcessUnpackaged = no

# Blacklisted executable paths (shell patterns)
#
BlackListedPaths = /usr/share/doc/*, */example*, /usr/bin/nspluginviewer, /usr/lib/xulrunner-*/plugin-container

主要修改ProcessUnpackaged的值从no改为yes,修改完之后刷新一下配置使得生效,以上步骤可以用一句命令搞定:

sed -i 's/ProcessUnpackaged = no/ProcessUnpackaged = yes/g' /etc/abrt/abrt-action-save-package-data.conf&& service abrtd restart

或者你也可以分成两步,手动修改,即将/etc/abrt/abrt-action-save-package-data.conf文件中的ProcessUnpackaged参数修改为ProcessUnpackaged=yes,并执行重启命令刷新配置使得生效
sudo service abrtd restart

三.附带情况

以上就是针对系统日志报错进行的修复过程,但是细心的读者其实通过上文中的系统日志发现除了ProcessUnpackaged的问题以外,还有sosreport run failed的问题,博主通过重装sosreport服务

yum install sosreport

并执行,sosreport,依旧会返回同样的错误

Traceback (most recent call last):
  File "/usr/sbin/sosreport", line 20, in <module>
    from sos.sosreport import main
ImportError: No module named sos.sosreport

查看File "/usr/sbin/sosreport"文件,发现这是一个Python文件,内容如下:

#!/usr/bin/python
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.

# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.

# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

""" sos entry point. """

import sys
try:
    from sos.sosreport import main
except KeyboardInterrupt:
    raise SystemExit()

if __name__ == '__main__':
    main(sys.argv[1:])

# vim:ts=4 et sw=4

博主服务器的Python版本为2.7.4,考虑是否由于Python版本问题导致无法正确找到包文件No module named sos.sosreport.目前暂不确定该问题是否也会影响Tomcat网页服务,故暂不考虑深究.

四.总结

关键需要在系统日志上查找重点内容,并对关键内容进行查错处理.

参考资料:

• How to solve abrtd doesn’t belong to any package and ProcessUnpackaged is set to ‘no’
• abrt服务导致进程挂掉