k8s笔记23--使用kubernetes-event-exporter采集集群events日志

1 介绍

kubernetes-event-exporter 是一个用于采集k8s事件的工具，它允许我们将经常遗漏的 Kubernetes 事件导出到第三方平台或者数据库，以便用于可观察性或警报目的。
event-exporter 可以将k8s事件存储到 Opsgenie、Webhooks、kafka、es等十几种平台|数据库，本文基于该工具介绍两种最常见的采集方式，方法一将事件日志采集到 kafka，方法二将事件日志直接采集到 es集群。

2 部署测试

2.1 写入kafka

部署脚本：

apiVersion: v1
kind: Namespace
metadata:
  name: lens-metrics
---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: lens-metrics
  name: event-exporter
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: event-exporter
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
subjects:
  - kind: ServiceAccount
    namespace: lens-metrics
    name: event-exporter
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: event-exporter-cfg
  namespace: lens-metrics
data:
  config.yaml: |该工具允许将经常遗漏的Kubernetes事件导出到各种输出，以便用于可观察性或警报目的。你不会相信你错过了什么。
    logLevel: error
    logFormat: json
    route:
      routes:
      - match:
        - receiver: "kafka"
    receivers:
      - name: "kafka"
        kafka:
          clientId: "kubernetes"
          topic: "k8s-event-log"
          brokers:
          - "192.168.2.11:9092"
          compressionCodec: "gzip"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: event-exporter
  namespace: lens-metrics
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: event-exporter
        version: v1
    spec:
      serviceAccountName: event-exporter
      containers:
      - name: event-exporter
        image: opsgenie/kubernetes-event-exporter:0.9
        imagePullPolicy: IfNotPresent
        args:
        - -conf=/data/config.yaml
        volumeMounts:
        - mountPath: /data
          name: cfg
      volumes:
        - name: cfg
          configMap:
            name: event-exporter-cfg
  selector:
    matchLabels:
      app: event-exporter
      version: v1

采集数据：
服务部署后，很快就采集到kafka，如下所示：
在这里插入图片描述
offset 4077的具体数据如下所示：

{
	"metadata": {
		"name": "calico-node-4p4jb.171cf6be0f3d946d",
		"namespace": "kube-system",
		"uid": "53d89898-44f3-471b-a3fc-9362567ac846",
		"resourceVersion": "4225520",
		"creationTimestamp": "2022-10-11T08:34:18Z",
		"managedFields": [
			{
				"manager": "kubelet",
				"operation": "Update",
				"apiVersion": "v1",
				"time": "2022-10-11T08:34:18Z"
			}
		]
	},
	"reason": "Unhealthy",
	"message": "Readiness probe failed: 2022-10-11 08:34:18.377 [INFO][503] confd/health.go 180: Number of node(s) with BGP peering established = 0\ncalico/node is not ready: BIRD is not ready: BGP not established with 192.168.2.12\n",
	"source": {
		"component": "kubelet",
		"host": "kmaster"
	},
	"firstTimestamp": "2022-10-11T08:34:18Z",
	"lastTimestamp": "2022-10-11T08:34:18Z",
	"count": 1,
	"type": "Warning",
	"eventTime": null,
	"reportingComponent": "",
	"reportingInstance": "",
	"involvedObject": {
		"kind": "Pod",
		"namespace": "kube-system",
		"name": "calico-node-4p4jb",
		"uid": "069010fe-ca85-4ede-a4c5-033f02975433",
		"apiVersion": "v1",
		"resourceVersion": "4223433",
		"fieldPath": "spec.containers{calico-node}",
		"labels": {
			"controller-revision-hash": "84f97c77db",
			"k8s-app": "calico-node",
			"pod-template-generation": "2"
		}
	}
}

2.2 写入es

部署脚本：

apiVersion: v1
kind: Namespace
metadata:
  name: lens-metrics
---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: lens-metrics
  name: event-exporter
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: event-exporter
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
subjects:
  - kind: ServiceAccount
    namespace: lens-metrics
    name: event-exporter
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: event-exporter-cfg
  namespace: lens-metrics
data:
  config.yaml: |
    logLevel: error
    logFormat: json
    route:
      routes:
      - match:
        - receiver: "es"
    receivers:
      - name: "es"
        elasticsearch:
          hosts:
            - http://your-es-ip:9200
          index: kube-events
          indexFormat: "kube-events-{2006-01-02}"
          username: your-user
          password: your-pwd
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: event-exporter
  namespace: lens-metrics
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: event-exporter
        version: v1
    spec:
      serviceAccountName: event-exporter
      containers:
      - name: event-exporter
        image: opsgenie/kubernetes-event-exporter:0.9
        imagePullPolicy: IfNotPresent
        args:
        - -conf=/data/config.yaml
        volumeMounts:
        - mountPath: /data
          name: cfg
      volumes:
        - name: cfg
          configMap:
            name: event-exporter-cfg
  selector:
    matchLabels:
      app: event-exporter
      version: v1

采集数据：
服务部署后，很快就采集到es，如下所示：
在这里插入图片描述
其中一条数据如下：

{
    "metadata":{
        "name":"event-exporter-f5b88ddb-vqrwc.171f1c2cb500b852",
        "namespace":"lens-metrics",
        "uid":"374afa8d-70dd-43f5-bd04-757ad3bdf5ec",
        "resourceVersion":"4501276",
        "creationTimestamp":"2022-10-18T08:22:45Z",
        "managedFields":[
            {
                "manager":"kubelet",
                "operation":"Update",
                "apiVersion":"v1",
                "time":"2022-10-18T08:22:45Z"
            }
        ]
    },
    "reason":"Created",
    "message":"Created container event-exporter",
    "source":{
        "component":"kubelet",
        "host":"kmaster"
    },
    "firstTimestamp":"2022-10-18T08:22:45Z",
    "lastTimestamp":"2022-10-18T08:22:45Z",
    "count":1,
    "type":"Normal",
    "eventTime":null,
    "reportingComponent":"",
    "reportingInstance":"",
    "involvedObject":{
        "kind":"Pod",
        "namespace":"lens-metrics",
        "name":"event-exporter-f5b88ddb-vqrwc",
        "uid":"41fb89bf-8f54-4810-af81-68893f6ed02a",
        "apiVersion":"v1",
        "resourceVersion":"4501249",
        "fieldPath":"spec.containers{event-exporter}",
        "labels":{
            "app":"event-exporter",
            "pod-template-hash":"f5b88ddb",
            "version":"v1"
        },
        "annotations":{
            "cni.projectcalico.org/containerID":"2199b2c0056cfb84979a0d94429b0c209877c5593b65fb0e2b890419d228e7c7",
            "cni.projectcalico.org/podIP":"10.224.189.42/32",
            "cni.projectcalico.org/podIPs":"10.224.189.42/32"
        }
    }
}

3 注意事项

  1. 本文第一种方法将事件日志写入到 kafka， 然后再根据需要消费kafka数据，并存放到用户自己的系统中。

4 说明

resmoio/kubernetes-event-exporter
kubernetes-event-exporter

# kubernetes # docker # kafka
Logo
Cloudpods

开源、云原生的融合云平台

更多推荐

面向未来的 IT 基础设施管理架构——融合云（Unified IaaS）

随着数字化时代的到来，IT系统已成为人类社会正常运转不可或缺的组成部分。不远的未来，智能制造，5G和人工智能等技术将成为推动生产力发展的重要引擎，人类社会将面临前所未有的全面彻底的数字化浪潮。IT基础设施作为IT系统运行的平台和载体，是实现数字化的基石。在这场数字化浪潮中，企业必须积极拥抱云计算技术，采用符合技术发展趋势、面向未来的IT基础构架，才能在未来的竞争中赢得先机。 一、云计算历经十余年

avatar Cloudpods

Cloudpods负载均衡的功能介绍

作者:周有松 今天的内容会从以下几个方面展开： 负载均衡产品简介。主要介绍负载均衡作为一个云上产品，它的功能模型是怎样的，日常使用中会遇到的业务词汇负载均衡的功能与典型应用场景。这部分主要结合业务词汇，对负载均衡服务中常见的一些功能选项进行介绍，并举例介绍一些典型的应用场景最后，我们做一下总结，讨论一下负载均衡产品相比传统方式的优点 一、产品简介​ 1. 以NGINX为例​ 提到负载均衡，我们以

avatar Cloudpods

使用Linux vfio将Nvidia GPU透传给QEMU虚拟机

Linux 上虚拟机 GPU 透传需要使用 vfio 的方式。主要是因为在 vfio 方式下对虚拟设备的权限和 DMA 隔离上做的更好。但是这么做也有个缺点，这个物理设备在主机和其他虚拟机都不能使用了。 qemu 直接使用物理设备本身命令行是很简单的，关键在于事先在主机上对系统、内核和物理设备的一些配置。 单纯从 qemu 的命令行来看，其实和普通虚拟机启动就差了最后那个-device的选项。这

avatar Cloudpods