持久化配置

# k8s-master-1 Master节点配置持久化存储,如果有其他方式提供存储在Grafana-Deployment中配置即可，或者使用Statefulset配置Storageclass来实现持久化存储
	mkdir -p /grafana
	chmod -R 755 /grafana

Grafana-Deployment

官网配置参考链接：Grafana官网部署参考配置：https://grafana.com/docs/grafana/latest/installation/kubernetes/

[root@k8s-master-1 grafana]# cat grafana-deployment.yaml 
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: grafana
  name: grafana
  namespace: kube-prometheus
spec:
  selector:
    matchLabels:
      app: grafana
  template:
    metadata:
      labels:
        app: grafana
    spec:
      nodeName: k8s-master-1      # 设置在k8s-master-1上运行
      tolerations:                # 设置能容忍在master节点运行
      - key: "node-role.kubernetes.io/master"
        operator: "Exists"
        effect: "NoSchedule"
      securityContext:
        fsGroup: 472
        supplementalGroups:
        - 0
      containers:
      - name: grafana
        image: grafana/grafana:8.5.1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 3000
          name: http-grafana
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /robots.txt
            port: 3000
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 30
          successThreshold: 1
          timeoutSeconds: 2
        livenessProbe:
          failureThreshold: 3
          initialDelaySeconds: 30
          periodSeconds: 10
          successThreshold: 1
          tcpSocket:
            port: 3000
          timeoutSeconds: 1
        resources:
          requests:
            cpu: 250m
            memory: 750Mi
        volumeMounts:
        - name: grafana-data
          mountPath: /var/lib/grafana
      volumes:
      - name: grafana-data
        hostPath:
          path: /grafana
          type: Directory
---
apiVersion: v1
kind: Service
metadata:
  name: grafana
  namespace: kube-prometheus
spec:
  selector:
    app: grafana
  sessionAffinity: None
  type: NodePort                   # 将其暴露出来，便于外部访问
  ports:
    - port: 3000
      protocol: TCP
      targetPort: http-grafana
      nodePort: 45000                # 固定端口，便于访问

# 查看部署后的状态
[root@k8s-master-1 grafana]# kubectl get pods -n kube-prometheus
NAME                                 READY   STATUS    RESTARTS   AGE
grafana-5c987bb879-wq7ng             1/1     Running   0          2m48s
node-exporter-b7h6c                  1/1     Running   2          20h
node-exporter-r9stp                  1/1     Running   2          20h
prometheus-server-5ffd56fdf4-pkcjd   1/1     Running   0          35m

接入Prometheus

默认账号：admin/admin

Dashboard部署

Grafana Plugin 链接： https://grafana.com/grafana/dashboards/

Node Exporter

链接：https://grafana.com/grafana/dashboards/1860

Nearly all default values exported by Prometheus node exporter graphed.

Only requires the default job_name: node, add as many targets as you need in ‘/etc/prometheus/prometheus.yml’.

  - job_name: node
    static_configs:
      - targets: ['localhost:9100']

Recommended for prometheus-node-exporter the arguments ‘–collector.systemd --collector.processes’ because the graph uses some of their metrics.

根据需求修改后，实际效果如下(效果挺好的)：

Kubernetes Cluster Monitor

链接：https://grafana.com/grafana/dashboards/315

根据要求修改prometheus-server配置文件后，效果如下：

Kubernetes Pod Monitor

链接：https://grafana.com/grafana/dashboards/6417 有BUG

链接：https://grafana.com/grafana/dashboards/13332

Docker Monitor

链接：https://grafana.com/grafana/dashboards/11600