Fluentd+ElasticSearch+Kibana 搭建日誌類文件分析器 安裝篇
一 查看當前Linux版本[coolbeans@coolbeans-fluentd ~]$ cd /proc[coolbeans@coolbeans-fluentd proc]$ lsb_release -aLSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-
一 查看當前機器信息
1. 操作系統
[coolbeans@coolbeans-fluentd proc]$ lsb_release -a
LSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID: CentOS
Description: CentOS release 6.4 (Final)
Release: 6.4
Codename: Final
2. hostname
[coolbeans@coolbeans-fluentd ~]$ cat /proc/sys/kernel/hostname
coolbeans-fluentd
[coolbeans@coolbeans-fluentd ~]$ hostname
coolbeans-fluentd
3. 内存
[coolbeans@coolbeans-fluentd ~]$ free
total used free shared buffers cached
Mem: 1922464 1067420 855044 0 256628 230204
-/+ buffers/cache: 580588 1341876
Swap: 4128760 0 4128760
[coolbeans@coolbeans-fluentd ~]$ cat /proc/meminfo
MemTotal: 1922464 kB
MemFree: 854912 kB
Buffers: 256628 kB
Cached: 230204 kB
SwapCached: 0 kB
Active: 690624 kB
Inactive: 193840 kB
Active(anon): 397784 kB
Inactive(anon): 4 kB
Active(file): 292840 kB
Inactive(file): 193836 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 4128760 kB
SwapFree: 4128760 kB
Dirty: 16 kB
Writeback: 0 kB
AnonPages: 397540 kB
Mapped: 26532 kB
Shmem: 164 kB
Slab: 150432 kB
SReclaimable: 91164 kB
SUnreclaim: 59268 kB
KernelStack: 1528 kB
PageTables: 4836 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 5089992 kB
Committed_AS: 645828 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 274520 kB
VmallocChunk: 34359447820 kB
HardwareCorrupted: 0 kB
AnonHugePages: 321536 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 8192 kB
DirectMap2M: 2088960 kB
4. CPU
[coolbeans@coolbeans-fluentd ~]$ cat /proc/cpuinfo | grep 'model name' |uniq
model name : Intel(R) Xeon(R) CPU X5660 @ 2.80GHz
5. Disk
[coolbeans@coolbeans-fluentd /]$ df -h
文件系统 容量 已用 可用 已用%% 挂载点
/dev/mapper/vg_coolbeansfluentd-lv_root
50G 3.1G 44G 7% /
tmpfs 939M 0 939M 0% /dev/shm
/dev/sda1 485M 64M 397M 14% /boot
/dev/mapper/vg_coolbeansfluentd-lv_home
439G 1.6G 415G 1% /home
ulimit -n
三 安裝git
安裝其所依賴之包：
yum install zlib-devel
yum install openssl-devel
yum install perl
yum install cpio
yum install expat-devel
yum install gettext-devel
yum install autoconf
//安裝GIT
wget http://www.codemonkey.org.uk/projects/git-snapshots/git/git-latest.tar.gz
tar xzvf git-latest.tar.gz
cd git-{date}
autoconf
./configure --with-curl=/usr/local
make
四 安裝RUBY
下載ruby-2.0.0-p247.tar.gz
tar -zxvf ruby-2.0.0-p247.tar.gz
cd ruby-2.0.0-p247
./configure -prefix=/opt/modules/ruby
make
make install
添加環境變量：export PATH=/opt/modules/ruby/bin:$PATH
測試RUBY GEM:
[coolbeans@coolbeans-fluentd ~]$ gem -v
2.0.3
備註：
在安裝RUBY前請確保OPEN-SSL已安裝
五 安裝elasticsearch
下載：elasticsearch-0.90.0.RC2.tar.gz
tar -zxvf elasticsearch-0.90.0.RC2.tar.gz
cd elasticsearch-0.90.0.RC2
啟動： ./bin/elasticsearch -f
六 安裝bundle
[coolbeans@coolbeans-fluentd bin]$ sudo ./gem install bundle
[sudo] password for coolbeans:
Fetching: bundler-1.3.5.gem (100%)
Successfully installed bundler-1.3.5
Fetching: bundle-0.0.1.gem (100%)
Successfully installed bundle-0.0.1
Parsing documentation for bundler-1.3.5
Installing ri documentation for bundler-1.3.5
Parsing documentation for bundle-0.0.1
Installing ri documentation for bundle-0.0.1
2 gems installed
七 安裝Kibana
[coolbeans@coolbeans-fluentd ~]$ cd Kibana-0.2.0
[coolbeans@coolbeans-fluentd Kibana-0.2.0]$ bundle install --path vendor/bundle
Fetching gem metadata from http://rubygems.org/.........
Fetching gem metadata from http://rubygems.org/..
Resolving dependencies...
Installing rake (10.1.0)
Installing daemons (1.1.9)
Installing diff-lcs (1.1.3)
Installing eventmachine (1.0.0)
Installing fastercsv (1.5.5)
Installing json (1.7.5)
Installing rack (1.4.1)
Installing rack-protection (1.2.0)
Installing tilt (1.3.3)
Installing sinatra (1.3.3)
Installing thin (1.5.0)
Installing tzinfo (0.3.35)
Using kibana (0.0.1) from source at .
Installing rspec-core (2.11.1)
Installing rspec-expectations (2.11.3)
Installing rspec-mocks (2.11.3)
Installing rspec (2.11.0)
Using bundler (1.3.5)
Your bundle is complete!
It was installed into ./vendor/bundle
啟動Kibana:
bundle exec ruby kibana.rb
默認新動態下，只可在本地訪問：
八 安裝Fluented RPM:
執行該腳本即可
http://toolbelt.treasure-data.com/sh/install-redhat.sh
九 安裝ElasticSearch plugin for Fluentd: fluent-plugin-elasticsearch
/usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-elasticsearch
[coolbeans@coolbeans-fluentd bin]$ sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-elasticsearch
[sudo] password for coolbeans:
Fetching: fluent-plugin-elasticsearch-0.1.2.gem (100%)
Successfully installed fluent-plugin-elasticsearch-0.1.2
1 gem installed
Installing ri documentation for fluent-plugin-elasticsearch-0.1.2...
Installing RDoc documentation for fluent-plugin-elasticsearch-0.1.2...
修改
/etc/td-agent/td-agent.conf為：
<source> type syslog port 42185 tag syslog </source> <source> type forward </source> <match syslog.**> type elasticsearch logstash_format true flush_interval 10s # for testing </match>
啟動td-agent
sudo /etc/init.d/td-agent start
十 fluent-plugin-notifier的安裝
學前最新版本為：0.2.1
sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-notifier -v 0.2.1
十一 fluent-plugin-mail的安裝
sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-mail
十二 fluent-plugin-tail-ex安裝
在in_tail plugin 的基礎上擴展，其主要新增功能為支持目錄文件收集
安裝命令：sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-tail-ex
十二 rsyslogd
系統已自帶
更多推荐
0
0- 0
所有评论(0)
已为社区贡献1条内容
修改 KibanaConfig.rb 16行. 將 127.0.0.1 修改為 0.0.0.0 以綁定所有IP
# The adress ip Kibana should listen on
KibanaHost = '127.0.0.1'