k8s安装elasticsearch+filebeat+kibana
搬自知乎 https://zhuanlan.zhihu.com/p/113629660es集群比较占用机器性能,我使用了4台2核8G的服务器搭建,空跑cluster内存占用了20%1,安装helm repohelm add https://helm.elastic.co2, 创建数据盘创建存储类,storageClass.name=efk-nfs-client,es节点通过name自动绑定helm
1,安装helm repo
helm add https://helm.elastic.co
2, 创建数据盘
创建存储类,storageClass.name=efk-nfs-client,es节点通过name自动绑定
helm install efk-nfs-storage -n nfs
–set nfs.server=172.21.2.159,nfs.path=/data/NFS/EFK
–set storageClass.name=efk-nfs-client,storageClass.reclaimPolicy=Retain
nfs-client-provisioner
3,角色分配
es-master 搭建一个 elasticsearch 至少需要 3 个 Pod 以防止集群脑裂。
es-data 数据节点至少需要 2 个 Pod 。数据节点将保留数据、接收查询和索引请求。
es-client 做为协调 elasticsearch 集群。至少需要 2 个。用于集群连接,并充当 HTTP 代理。如果不使用 es-clinet 那么 es-data 充当协调,尽量避免在较大的集群上这样做。
4,生成es证书,选择版本为7.6.1
#使用es容器生成证书
docker run --name elastic-charts-certs -i -w /app
elasticsearch:7.6.1
/bin/sh -c "
elasticsearch-certutil ca --out /app/elastic-stack-ca.p12 --pass ‘’ &&
elasticsearch-certutil cert --name security-master --dns security-master --ca /app/elastic-stack-ca.p12 --pass ‘’ --ca-pass ‘’ --out /app/elastic-certificates.p12"
从容器中将生成的证书拷贝出来
docker cp elastic-charts-certs:/app/elastic-certificates.p12 ./
证书生成成功该容器删除
docker rm -f elastic-charts-certs
#证书转换
openssl pkcs12 -nodes -passin pass:’’ -in elastic-certificates.p12 -out elastic-certificate.pem
运行完成会获得 elastic-certificate.pem 与 elastic-certificates.p12
5,将证书,es集群密码导入k8s
添加证书
kubectl create ns efk
kubectl create secret -n efk generic elastic-certificates --from-file=elastic-certificates.p12
kubectl create secret -n efk generic elastic-certificate-pem --from-file=elastic-certificate.pem
设置集群用户名密码,用户名不建议修改
kubectl create secret -n efk generic elastic-credentials --from-literal=password=admin --from-literal=username=elastic
6,部署 es-master 节点
cat > es-master.yaml << EOF
使用镜像
image: “elasticsearch”
es 集群名称
clusterName: “es-aka”
es 节点名称
nodeGroup: “master”
es 节点角色
roles:
master: “true”
ingest: “false”
data: “false”
副本数量
replicas: 3
资源限制
resources:
requests:
cpu: “300m”
memory: “1Gi”
limits:
cpu: “1000m”
memory: “2Gi”
volumeClaimTemplate:
该volume只能被单个节点以读写的方式映射
accessModes: [ “ReadWriteOnce” ]
自动绑定动态 pv
storageClassName: “efk-nfs-client”
resources:
requests:
storage: 4Gi
是否 SSH 开启改为 https
protocol: http
添加配置
esConfig:
elasticsearch.yml: |
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
# 是否启用 htpps 启用 head 无法连接,开启还需要将 protocol 修改为 https
# xpack.security.http.ssl.enabled: true
# xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
# xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
环境变量
extraEnvs:
- name: ELASTIC_PASSWORD
valueFrom:
secretKeyRef:
name: elastic-credentials
key: password - name: ELASTIC_USERNAME
valueFrom:
secretKeyRef:
name: elastic-credentials
key: username
证书
secretMounts:
- name: elastic-certificates
secretName: elastic-certificates
path: /usr/share/elasticsearch/config/certs
EOF
helm 部署 es-master 节点并安装指定版本 elasticsearch 7.6.1
helm install es-master -n efk --values es-master.yaml elastic/elasticsearch --version 7.6.1
7,部署 es-data 节点
cat > es-data.yaml <<EOF
使用镜像
image: “elasticsearch”
es 集群名称
clusterName: “es-aka”
es 节点名称
nodeGroup: “data”
es 节点角色
roles:
master: “false”
ingest: “true”
data: “true”
副本数量
replicas: 3
资源限制
resources:
requests:
cpu: “300m”
memory: “1Gi”
limits:
cpu: “1000m”
memory: “2Gi”
PVC
volumeClaimTemplate:
该volume只能被单个节点以读写的方式映射
accessModes: [ “ReadWriteOnce” ]
自动绑定动态 pv
storageClassName: “efk-nfs-client”
resources:
requests:
storage: 60Gi
是否 SSH 开启改为 https
protocol: http
添加配置
esConfig:
elasticsearch.yml: |
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
# 是否启用 htpps 启用 head 无法连接,开启还需要将 protocol 修改为 https
# xpack.security.http.ssl.enabled: true
# xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
# xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
环境变量
extraEnvs:
- name: ELASTIC_PASSWORD
valueFrom:
secretKeyRef:
name: elastic-credentials
key: password - name: ELASTIC_USERNAME
valueFrom:
secretKeyRef:
name: elastic-credentials
key: username
证书
secretMounts:
- name: elastic-certificates
secretName: elastic-certificates
path: /usr/share/elasticsearch/config/certs
EOF
helm 部署 es-data 节点并安装指定版本 elasticsearch 7.6.1
helm install es-data -n efk --values es-data.yaml elastic/elasticsearch --version 7.6.1
8,部署 es-client 节点
下面代码直接复制黏贴即可
cat > es-client.yaml <<EOF
使用镜像
image: “elasticsearch”
es 集群名称
clusterName: “es-aka”
es 节点名称
nodeGroup: “client”
es 节点角色
roles:
master: “false”
ingest: “false”
data: “false”
副本数量
replicas: 2
资源限制
resources:
requests:
cpu: “300m”
memory: “1Gi”
limits:
cpu: “1000m”
memory: “2Gi”
是否启用 PVC
persistence:
enabled: false
设置 es-clinet 默认为 NodePort
service:
type: NodePort
设置 NodePort 默认端口
nodePort: 30920
是否 SSH 开启改为 https
protocol: http
添加配置
esConfig:
elasticsearch.yml: |
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
# 是否启用 htpps 启用 head 无法连接,开启还需要将 protocol 修改为 https
# xpack.security.http.ssl.enabled: true
# xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
# xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
环境变量
extraEnvs:
- name: ELASTIC_PASSWORD
valueFrom:
secretKeyRef:
name: elastic-credentials
key: password - name: ELASTIC_USERNAME
valueFrom:
secretKeyRef:
name: elastic-credentials
key: username
证书
secretMounts:
- name: elastic-certificates
secretName: elastic-certificates
path: /usr/share/elasticsearch/config/certs
EOF
helm 部署 es-client 节点并安装指定版本 elasticsearch 7.6.1
helm install es-client -n efk --values es-client.yaml elastic/elasticsearch --version 7.6.1
查看 Elasticsearch状态
kubectl get pv
kubectl get pods --namespace=efk -w
kubectl get svc -n efk
9,部署filebeat
安装 Filebeat 7.6.1 版本
需要填写集群账号与密码
cat > es-filebeat.yaml <<EOF
使用镜像
image: “elastic/filebeat”
添加配置
filebeatConfig:
filebeat.yml: |
filebeat.inputs:
- type: docker
containers.ids:
- ‘*’
processors:
- add_kubernetes_metadata:
in_cluster: true
output.elasticsearch:
# elasticsearch 用户
username: ‘elastic’
# elasticsearch 密码
password: ‘akiraka’
# elasticsearch 主机
hosts: [“es-aka-client:9200”]
环境变量
extraEnvs:
- name: ‘ELASTICSEARCH_USERNAME’
valueFrom:
secretKeyRef:
name: elastic-credentials
key: username - name: ‘ELASTICSEARCH_PASSWORD’
valueFrom:
secretKeyRef:
name: elastic-credentials
key: password
EOF
helm 安装指定版本 filebeat 7.6.1
helm install filebeat -n efk --values es-filebeat.yaml elastic/filebeat --version 7.6.1
10,部署 Kibana
helm repo add elastic https://helm.elastic.co
安装 Kibana 7.6.1 版本
设置 kibana 默认简体中文
Kibana 无需填写集群账号与密码
service.type 设置为: NodePort
service.nodePort 固定端口: 32323
elasticsearchHosts 填写集群地址,格式为: http://es-aka-client:9200
cat > es-kibana.yaml << EOF
使用镜像
image: “kibana”
集群地址
elasticsearchHosts: “http://es-aka-client:9200”
添加配置
kibanaConfig:
kibana.yml: |
# 设置 kibana 简体中文
i18n.locale: “zh-CN”
否 SSH 开启改为 https 确保集群也是 https
protocol: http
服务设置
service:
type: NodePort
nodePort: 32323
环境变量
extraEnvs:
- name: ‘ELASTICSEARCH_USERNAME’
valueFrom:
secretKeyRef:
name: elastic-credentials
key: username - name: ‘ELASTICSEARCH_PASSWORD’
valueFrom:
secretKeyRef:
name: elastic-credentials
key: password
EOF
helm 安装指定版本 kibana 7.6.1
helm install kibana -n efk --values es-kibana.yaml elastic/kibana --version 7.6.1
11,访问
通过 Elasticsearch Head 访问es
其他浏览器我不清楚,Chrome 浏览器扩展商店搜索 ElasticSearch Head 然后安装该扩展
条件已知 elasticsearch-client 使用了 NodePort 端口为: 30920
使用方式: 集群随便一台机器 IP 地址,格式: http://节点IP:30920
访问 Kibana 仪表盘
Kibana 默认端口为:32323
访问方式: http://集群ip:32323
默认设置中文界面
默认用户与密码为自己设置,我设置
本集群默认用户为: elastic
本集群默认用户为: admin
更多推荐
所有评论(0)