Docker 搭建 ElasticSearch 集群
Ubuntu DockerElasticSearch Version: 5.6.2 Download1 Master + 2 NodeStep 1: 创建 Ubuntu 本地文件首先在 Ubuntu 本地创建以下目录结构:ES /├── config│├── es0.yml│├── es1.yml│└── es2.yml├── data0├── data1└── data2# mkdir ES#
- Ubuntu Docker
- ElasticSearch Version: 5.6.2 Download
- 1 Master + 2 Node
Step 1: 创建 Ubuntu 本地文件
首先在 Ubuntu 本地创建以下目录结构:
ES /
├── config
│ ├── es0.yml
│ ├── es1.yml
│ └── es2.yml
├── data0
├── data1
└── data2
# mkdir ES
# cd ES
# mkdir config
# mkdir data0
# mkdir data1
# mkdir data2
创建完文件夹后,在 config 路径下创建 3 个 .yml 文件。并分别在 3 个 .yml 文件中填入如下内容:
注意: 3 个 .yml 文件中的以下内容,需要在创建好 3 个 docker 容器之后修改为读者自己环境中的参数。
- network.publish_host: 172.17.0.2
- http.port: 9200
- transport.tcp.port: 9300
- discovery.zen.ping.unicast.hosts: [“172.17.0.2:9300”,“172.17.0.3:9301”,“172.17.0.4:9302”]
节点0
# es0.yml
cluster.name: elasticsearch
node.name: es-node0
network.bind_host: 0.0.0.0
network.publish_host: 172.17.0.2
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true
node.data: true
discovery.zen.ping.unicast.hosts: ["172.17.0.2:9300","172.17.0.3:9301","172.17.0.4:9302"]
discovery.zen.minimum_master_nodes: 2
节点1
# es1.yml
cluster.name: elasticsearch
node.name: es-node1
network.bind_host: 0.0.0.0
network.publish_host: 172.17.0.3
http.port: 9201
transport.tcp.port: 9301
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: false
node.data: true
discovery.zen.ping.unicast.hosts: ["172.17.0.2:9300","172.17.0.3:9301","172.17.0.4:9302"]
discovery.zen.minimum_master_nodes: 2
节点2
# es2.yml
cluster.name: elasticsearch
node.name: es-node2
network.bind_host: 0.0.0.0
network.publish_host: 172.17.0.4
http.port: 9202
transport.tcp.port: 9302
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: false
node.data: true
discovery.zen.ping.unicast.hosts: ["172.17.0.2:9300","172.17.0.3:9301","172.17.0.4:9302"]
discovery.zen.minimum_master_nodes: 2
Step 2: 创建 3 个节点的容器
2.1 拉取 ElasticSearch Docker 镜像
docker pull elasticsearch:5.6.2
docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.elastic.co/elasticsearch/elasticsearch 5.6.2 59b11c02b218 4 years ago 657MB
2.2 创建 Master 容器
docker run -d --name=esmaster -p 9200:9200 -p 9300:9300 -v /home/harrytsz/Workspace/harrytsz-escluster/ES/config/es0.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /home/harrytsz/Workspace/harrytsz-escluster/ES/data0:/usr/share/elasticsearch/data 59b11c02b218
安装节点的过程中可能遇到以下问题:
1、端口号冲突
docker: Error response from daemon: driver failed programming external connectivity on endpoint esmaster (f219351232a9ec9ed24463238f668d7445bd170afd223c900fdd65c38bd16df4): Error starting userland proxy: listen tcp4 0.0.0.0:9300: bind: address already in use.
查找正在占用 9300 端口的进程,并将其 kill 掉,重新启动 docker 容器
sudo netstat -tanlp
激活Internet连接 (服务器和已建立连接的)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 138 192.168.31.27:45248 203.119.175.170:443 LAST_ACK -
tcp6 0 0 127.0.0.1:3200 :::* LISTEN 5190/java
tcp6 0 0 ::1:6379 :::* LISTEN 895/redis-server 12
tcp6 0 0 127.0.0.1:40622 :::* LISTEN 4650/java
tcp6 0 0 :::9200 :::* LISTEN 11564/docker-proxy
tcp6 0 0 :::9300 :::* LISTEN 11542/docker-proxy
tcp6 0 0 127.0.0.1:46005 :::* LISTEN 4650/java
tcp6 0 0 ::1:631 :::* LISTEN 753/cupsd
tcp6 0 0 127.0.0.1:48260 127.0.0.1:35831 TIME_WAIT -
tcp6 0 0 127.0.0.1:48334 127.0.0.1:35831 ESTABLISHED 4650/java
(base) harrytsz@potter:~$ sudo kill 11556
重新启动 docker 容器,没有报错即创建成功。节点1和节点2的创建流程和上面一样。
2.3 创建 Node1 容器
docker run -d --name=esnode1 -p 9201:9201 -p 9301:9301 -v /home/harrytsz/Workspace/harrytsz-escluster/ES/config/es1.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /home/harrytsz/Workspace/harrytsz-escluster/ES/data1:/usr/share/elasticsearch/data 59b11c02b218
2.4 创建 Node2 容器
docker run -d --name=esnode2 -p 9202:9202 -p 9302:9302 -v /home/harrytsz/Workspace/harrytsz-escluster/ES/config/es2.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /home/harrytsz/Workspace/harrytsz-escluster/ES/data2:/usr/share/elasticsearch/data 59b11c02b218
至此,已经创建了 3 个节点 docker 容器,接下来分别启动这 3 个节点的 docker 容器。
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d16259915320 59b11c02b218 "/bin/bash bin/es-do…" 59 minutes ago Up 59 minutes 9200/tcp, 0.0.0.0:9202->9202/tcp, :::9202->9202/tcp, 9300/tcp, 0.0.0.0:9302->9302/tcp, :::9302->9302/tcp esnode2
b35c4b95c054 59b11c02b218 "/bin/bash bin/es-do…" 59 minutes ago Up 59 minutes 9200/tcp, 0.0.0.0:9201->9201/tcp, :::9201->9201/tcp, 9300/tcp, 0.0.0.0:9301->9301/tcp, :::9301->9301/tcp esnode1
afd20c20b836 59b11c02b218 "/bin/bash bin/es-do…" About an hour ago Up About an hour 0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 0.0.0.0:9300->9300/tcp, :::9300->9300/tcp esmaster
Step3 : 修改 3 个容器中的 .yml
3.1 Master 节点
此处用 root 权限启动,这样可以解决 elasticsearch docker 内切换为 su 权限时,需要密码的问题。
docker exec -it --user root afd20c20b836
以下操作均在 elasticsearch docker 容器内部执行:
1、安装 net-tools 工具
yum install net-tools
安装 net-tools 成功之后,执行:
[root@afd20c20b836 config]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.17.0.2 netmask 255.255.0.0 broadcast 175.47.255.255
ether 02:42:dc:41:00:02 txqueuelen 0 (Ethernet)
RX packets 25749 bytes 50314289 (47.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 25191 bytes 3086559 (2.9 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1000 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
记录下 ip 172.17.0.2,接下来去 config/es0.yml 中修改其中的 ip 值。
2、安装 vim 工具
[root@afd20c20b836 elasticsearch]# yum install vim
Loaded plugins: fastestmirror, ovl
base | 3.6 kB 00:00:00 extras | 2.9 kB 00:00:00
updates | 2.9 kB 00:00:00 (1/4): base/7/x86_64/group_gz | 153 kB 00:00:00
(2/4): extras/7/x86_64/primary_db
[root@afd20c20b836 elasticsearch]# cd config/
[root@afd20c20b836 config]# ls
elasticsearch.yml ingest-geoip jvm.options log4j2.properties scripts x-pack
[root@afd20c20b836 config]# vim elasticsearch.yml
# elasticsearch.yml
cluster.name: elasticsearch
node.name: es-node0
network.bind_host: 0.0.0.0
network.publish_host: 172.17.0.2
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true
node.data: true
discovery.zen.ping.unicast.hosts: ["172.17.0.2:9300","172.17.0.3:9301","172.17.0.4:9302"]
discovery.zen.minimum_master_nodes: 2
将 es0.yml 中的 ip 地址修改为自己环境中的 ip 即可。
至此 Master 节点已经创建完毕,其他两个节点按照同样的步骤,修改各自 elasticsearch.yml 中的 ip 和 hosts 字段。此处,不再重复,读者自行完成。
Step4 : 测试 elasticsearch docker 集群
在 Ubuntu 主机中执行以下命令:
curl -u elastic 172.17.0.2:9200
Enter host password for user 'elastic':
{
"name" : "es-node0",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "_na_",
"version" : {
"number" : "5.6.2",
"build_hash" : "57e20f3",
"build_date" : "2017-09-23T13:16:45.703Z",
"build_snapshot" : false,
"lucene_version" : "6.6.1"
},
"tagline" : "You Know, for Search"
}
curl -u elastic 172.17.0.3:9201
Enter host password for user 'elastic':
{
"name" : "es-node1",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "_na_",
"version" : {
"number" : "5.6.2",
"build_hash" : "57e20f3",
"build_date" : "2017-09-23T13:16:45.703Z",
"build_snapshot" : false,
"lucene_version" : "6.6.1"
},
"tagline" : "You Know, for Search"
}
curl -u elastic 172.17.0.4:9202
Enter host password for user 'elastic':
{
"name" : "es-node2",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "_na_",
"version" : {
"number" : "5.6.2",
"build_hash" : "57e20f3",
"build_date" : "2017-09-23T13:16:45.703Z",
"build_snapshot" : false,
"lucene_version" : "6.6.1"
},
"tagline" : "You Know, for Search"
}
注意:
错误如下:
{“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“missing authentication token for REST request [/idx]”,“header”:{“WWW-Authenticate”:"Basic realm=“security” charset=“UTF-8"”}}],“type”:“security_exception”,“reason”:“missing authentication token for REST request [/idx]”,“header”:{“WWW-Authenticate”:"Basic realm=“security” charset=“UTF-8"”}},“status”:401}
错入原因:
是 Elasticsearch 和 kibana 在加入插件 x-pack 之后的安全问题,在加入了 x-pack 需要输入用户名和密码才能进行对 ELK 的操作。
错误解决:
加入用户名和密码属性,其中 elastic是用户名(默认),changme是密码(默认)
curl -u elastic 172.17.0.2:9200
更多推荐
所有评论(0)