k8s 部署syslog-kafka-es
三、开启消费线程,监听kafka的demo topic。消费处理,解析,格式转换,存储到elasticsearch。一、syslog-kafka-es程序启动,启动514端口监听,接收syslog日志,netty实现数据接收。二、日志处理、解析,avro格式转换等,存储kafka的demo topic上。
·
1、部署三节点的Kafka、zookeeper,elasticsearch
2、部署kafka-ui
3、k8s部署springboot程序syslog-kafka-es
kafka数据
流程:
一、syslog-kafka-es程序启动,启动514端口监听,接收syslog日志,netty实现数据接收。
二、日志处理、解析,avro格式转换等,存储kafka的demo topic上。
三、开启消费线程,监听kafka的demo topic。消费处理,解析,格式转换,存储到elasticsearch。
apiVersion: v1
kind: Service
metadata:
name: syslog-kafka-es
labels:
service: syslog-kafka-es
spec:
ports:
- name: http
port: 8080
targetPort: 8080
protocol: TCP
nodePort: 30180
- name: debug
port: 12306
targetPort: 12306
protocol: TCP
nodePort: 30181
- name: log
port: 514
targetPort: 514
protocol: UDP
nodePort: 30182
type: NodePort
selector:
app: syslog-kafka-es
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: syslog-kafka-es
labels:
app: syslog-kafka-es
spec:
replicas: 1
selector:
matchLabels:
app: syslog-kafka-es
template:
metadata:
labels:
app: syslog-kafka-es
spec:
containers:
- name: syslog-kafka-es
imagePullPolicy: IfNotPresent
image: zhxl1989/syslog-kafka-es:latest
ports:
- containerPort: 8080
name: http
protocol: TCP
- containerPort: 12306
name: debug
protocol: TCP
- containerPort: 514
name: log
protocol: UDP
resources:
requests:
memory: 1Gi
limits:
memory: 1Gi
volumeMounts:
- mountPath: /etc/localtime
name: volume-localtime
volumes:
- hostPath:
path: /etc/localtime
type: ''
name: volume-localtime
java.net.UnknownHostException: es-0.es-headless.default.svc.cluster.local,es-1.es-headless.default.svc.cluster.local,es-2.es-headless.default.svc.cluster.local: Name or service not known, executionId: 2
更多推荐
已为社区贡献8条内容
所有评论(0)