环境	centos7
k8s部署方式	kubeadm
k8s版本号	v1.17.0

前提条件：

1. 部署nfs服务
2. 目录权限必须是777

[root@k8s-node1 data]# cat /etc/exports
/nfs 192.168.1.0/24(rw)

目录结构：（最终）

[root@k8s-node1 nfs]# tree
.
├── grafana
│   └── data
│      
└── prometheus
    └── data
      

[root@k8s-node1 data]# ll /nfs/
总用量 0
drwxrwxrwx. 3 root root 18 8月   7 12:48 grafana
drwxrwxrwx. 3 root root 18 7月  27 10:50 prometheus

1.在kubernetest集群中创建namespace

apiVersion: v1
kind: Namespace
metadata: 
  name: ns-monitor
  labels:
    name: ns-monitor
    

[root@k8s-master Prometheus+Grafana╝р┐╪Kubernetes]# kubectl apply -f namespace.yaml 
namespace/ns-monitor created
[root@k8s-master Prometheus+Grafana╝р┐╪Kubernetes]# kubectl get ns
NAME                   STATUS   AGE
ns-monitor             Active   6s

2.安装node-exporter
在kubernetest集群中部署node-exporter.
Node-exporter用于采集kubernetes集群中各个节点的物理指标，比如：Memory、CPU等。
可以直接在每个物理节点是直接安装，这里我们使用DaemonSet部署到每个节点上.
使用 hostNetwork: true 和 hostPID: true 使其获得Node的物理指标信息，配置tolerations使其在master节点也启动一个pod。

[root@k8s-master Prometheus_Grafana]# vim node-exporter.yaml 

kind: DaemonSet
apiVersion: apps/v1
metadata:
  labels:
    app: node-exporter
  name: node-exporter
  namespace: ns-monitor
spec:
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: node-exporter
  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      containers:
        - name: node-exporter
          image: prom/node-exporter:v0.16.0
          ports:
            - containerPort: 9100
              protocol: TCP
              name:     http
      hostNetwork: true
      hostPID: true
      tolerations:
        - effect: NoSchedule
          operator: Exists

---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: node-exporter
  name: node-exporter-service
  namespace: ns-monitor
spec:
  ports:
    - name:     http
      port: 9100
      nodePort: 31672
      protocol: TCP
  type: NodePort
  selector:
    app: node-exporter
                      

检验node-exporter是否成功运行

[root@k8s-master Prometheus_Grafana]# kubectl get pod -n ns-monitor
NAME                  READY   STATUS    RESTARTS   AGE
node-exporter-66gfv   1/1     Running   0          73s
node-exporter-9bmhb   1/1     Running   0          73s
node-exporter-hkjwm   1/1     Running   0          73s
[root@k8s-master Prometheus_Grafana]# kubectl get svc -n ns-monitor 
NAME                    TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)          AGE
node-exporter-service   NodePort   10.1.25.245   <none>        9100:31672/TCP   2m21s

3.部署Prometheus pod
prometheus.yaml 中包含rbac认证、ConfigMap等。

kubectl apply -f prometheus.yaml 

[root@k8s-master Prometheus_Grafana]# kubectl get pod -n ns-monitor
NAME                          READY   STATUS    RESTARTS   AGE
node-exporter-66gfv           1/1     Running   0          10d
node-exporter-9bmhb           1/1     Running   0          10d
node-exporter-hkjwm           1/1     Running   0          10d
prometheus-6b4f48745c-4gzck   1/1     Running   0          8s

[root@k8s-master Prometheus_Grafana]# kubectl get svc -n ns-monitor
NAME                    TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
node-exporter-service   NodePort   10.1.25.245    <none>        9100:31672/TCP   10d
prometheus-service      NodePort   10.1.244.194   <none>        9090:30790/TCP   7m32s

浏览器访问： http://主机ip:30790/graph
查看集群中所有pod在1min捏的cpu使用情况：

rate函数获取样本变化率

4.在kubernetest中部署grafana

kubectl apply -f grafana.yaml

[root@k8s-master Prometheus_Grafana]# kubectl get svc -n ns-monitor
NAME                    TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
grafana-service         NodePort   10.1.76.150    <none>        3000:30105/TCP   9m16s

浏览器访问： http://主机ip:30105/graph/login 默认用户名和密码：admin/admin
5.配置grafana数据源

6.导入Prometheus Dashboard

6.导入kubernetes Dashboard
把 kubernetes的Dashboard的模板导入进来，直接把JSON格式内容复制进来。
效果图：
7：官网dashboard下载地址：
https://grafana.com/grafana/dashboards