kubernetes集群实战——资源监控(Metrics-Server、Dashboard部署)
1.什么是Metrics-Server?Metrics-Server是集群核心监控数据的聚合器,用来替换之前的heapster。容器相关的 Metrics 主要来自于 kubelet 内置的 cAdvisor 服务,有了MetricsServer之后,用户就可以通过标准的 Kubernetes API 来访问到这些监控数据。Metrics API 只可以查询当前的度量数据,并不保存历史数据。Met
1.什么是Metrics-Server?
Metrics-Server是集群核心监控数据的聚合器,用来替换之前的heapster。
容器相关的 Metrics 主要来自于 kubelet 内置的 cAdvisor 服务,有了MetricsServer之后,用户就可以通过标准的 Kubernetes API 来访问到这些监控数据。
Metrics API 只可以查询当前的度量数据,并不保存历史数据。
Metrics API URI 为 /apis/metrics.k8s.io/,在 k8s.io/metrics 维护。
必须部署 metrics-server 才能使用该 API,metrics-server 通过调用 Kubelet Summary API 获取数据。
Metrics Server 并不是 kube-apiserver 的一部分,而是通过 Aggregator 这种插件机制,在独立部署的情况下同 kube-apiserver 一起统一对外服务的。
kube-aggregator 其实就是一个根据 URL 选择具体的 API 后端的代理服务器。
Metrics-server属于Core metrics(核心指标),提供API metrics.k8s.io,仅提供Node和Pod的 CPU和内存使用情况。而其他Custom Metrics(自定义指标)由Prometheus等组件来完成。
2.Metrics-Server安装部署
(1)Metrics-Server安装
[root@server2 ~]# mkdir ms
[root@server2 ~]# cd ms
[root@server2 ms]# wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.6/components.yaml
--2020-07-10 08:45:51-- https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.6/components.yaml
Resolving github.com (github.com)... 13.229.188.59
Connecting to github.com (github.com)|13.229.188.59|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/92132038/cf781b00-7752-11ea-9faf-0b397e7b8445?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200710%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200710T004553Z&X-Amz-Expires=300&X-Amz-Signature=789a7d64440556513b5a85483ba5d3aac7d562fafd68c8d4e5f4dd2c7cbf81a9&X-Amz-SignedHeaders=host&actor_id=0&repo_id=92132038&response-content-disposition=attachment%3B%20filename%3Dcomponents.yaml&response-content-type=application%2Foctet-stream [following]
--2020-07-10 08:45:53-- https://github-production-release-asset-2e65be.s3.amazonaws.com/92132038/cf781b00-7752-11ea-9faf-0b397e7b8445?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200710%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200710T004553Z&X-Amz-Expires=300&X-Amz-Signature=789a7d64440556513b5a85483ba5d3aac7d562fafd68c8d4e5f4dd2c7cbf81a9&X-Amz-SignedHeaders=host&actor_id=0&repo_id=92132038&response-content-disposition=attachment%3B%20filename%3Dcomponents.yaml&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.230.155
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.230.155|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3335 (3.3K) [application/octet-stream]
Saving to: ‘components.yaml’
100%[=============================>] 3,335 --.-K/s in 0s
2020-07-10 08:45:55 (7.17 MB/s) - ‘components.yaml’ saved [3335/3335]
[root@server2 ms]# ls
components.yaml
[root@server2 ms]# vim components.yaml
[root@server2 ms]# kubectl apply -f components.yaml
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
serviceaccount/metrics-server created
deployment.apps/metrics-server created
service/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
[root@server2 ms]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-bd97f9cd9-8lqqc 1/1 Running 1 47h
coredns-bd97f9cd9-cdk94 1/1 Running 1 46h
etcd-server2 1/1 Running 5 7d17h
kube-apiserver-server2 1/1 Running 5 7d17h
kube-controller-manager-server2 1/1 Running 5 7d17h
kube-flannel-ds-amd64-g8hg7 1/1 Running 5 7d16h
kube-flannel-ds-amd64-tlmh2 1/1 Running 1 46h
kube-proxy-dkr49 1/1 Running 4 7d16h
kube-proxy-dw7lv 1/1 Running 5 7d17h
kube-proxy-z7r6x 1/1 Running 1 46h
kube-scheduler-server2 1/1 Running 5 7d17h
metrics-server-7cdfcc6666-ptv48 1/1 Running 0 13s
[root@server2 ms]# kubectl describe pod -n kube-system metrics-server-7cdfcc6666-ptv48
Name: metrics-server-7cdfcc6666-ptv48
Namespace: kube-system
Priority: 0
Node: server4/172.25.12.4
Start Time: Fri, 10 Jul 2020 08:56:15 +0800
Labels: k8s-app=metrics-server
pod-template-hash=7cdfcc6666
Annotations: <none>
Status: Running
IP: 10.244.2.44
IPs:
IP: 10.244.2.44
Controlled By: ReplicaSet/metrics-server-7cdfcc6666
Containers:
metrics-server:
Container ID: docker://1f24dc0545b0150912ecb6766c4ac08f2a9a0d338642ccd28dfffa0928b5f871
Image: metrics-server-amd64:v0.3.6
Image ID: docker-pullable://metrics-server-amd64@sha256:c9c4e95068b51d6b33a9dccc61875df07dc650abbf4ac1a19d58b4628f89288b
Port: 4443/TCP
Host Port: 0/TCP
Args:
--cert-dir=/tmp
--secure-port=4443
State: Running
Started: Fri, 10 Jul 2020 08:56:18 +0800
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/tmp from tmp-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from metrics-server-token-8t6zs (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
tmp-dir:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
metrics-server-token-8t6zs:
Type: Secret (a volume populated by a Secret)
SecretName: metrics-server-token-8t6zs
Optional: false
QoS Class: BestEffort
Node-Selectors: kubernetes.io/arch=amd64
kubernetes.io/os=linux
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 42s default-scheduler Successfully assigned kube-system/metrics-server-7cdfcc6666-ptv48 to server4
Normal Pulling 41s kubelet, server4 Pulling image "metrics-server-amd64:v0.3.6"
Normal Pulled 39s kubelet, server4 Successfully pulled image "metrics-server-amd64:v0.3.6"
Normal Created 39s kubelet, server4 Created container metrics-server
Normal Started 39s kubelet, server4 Started container metrics-server
[root@server2 ms]#
下载yaml文件
下载并上传实验所需镜像metrics-server到私有仓库
应用yaml文件,metrics-server成功运行
(2)解决Metrics-server的Pod日志报错
部署后查看Metrics-server的Pod日志:
kubectl logs -n kube-system metrics-server-7cdfcc6666-ptv48
报错1:dial tcp: lookup server2 on 10.96.0.10:53: no such host
这是因为没有内网的DNS服务器,所以metrics-server无法解析节点名字。可以直接修改 coredns的configmap,将各个节点的主机名加入到hosts中,这样所有Pod都可以从 CoreDNS中解析各个节点的名字。
解决:
kubectl edit configmap coredns -n kube-system
作内部地址解析
apiVersion: v1
data:
Corefile: |
.:53 {
errors
health {
lameduck 5s
}
ready
hosts {
172.25.12.2 server2
172.25.12.3 server3
172.25.12.4 server4
fallthrough
}
报错2:x509: certificate signed by unknown authority
Metric Server 支持一个参数 --kubelet-insecure-tls,可以跳过这一检查,然而官 方也明确说了,这种方式不推荐生产使用
解决:
启用TLS Bootstrap 证书签发(在master和各个节点中)
vim /var/lib/kubelet/config.yaml
[root@server2 ms]# vim /var/lib/kubelet/config.yaml ##添加参数
serverTLSBootstrap: true
[root@server2 ms]# systemctl restart kubelet
[root@server2 ms]# kubectl get csr
NAME AGE SIGNERNAME REQUESTOR CONDITION
csr-57blk 13s kubernetes.io/kubelet-serving system:node:server2 Pending
[root@server2 ms]# kubectl certificate approve csr-57blk
certificatesigningrequest.certificates.k8s.io/csr-57blk approved
[root@server2 ms]# ubectl get csr
-bash: ubectl: command not found
[root@server2 ms]# kubectl get csr
NAME AGE SIGNERNAME REQUESTOR CONDITION
csr-57blk 76s kubernetes.io/kubelet-serving system:node:server2 Approved,Issued
报错3: Error from server (ServiceUnavailable): the server is currently unable to handle the request (get nodes.metrics.k8s.io)
如果metrics-server正常启动,没有错误,应该就是网络问题。修改metricsserver的Pod 网络模式:hostNetwork: true
3. Dashboard部署
Dashboard可以给用户提供一个可视化的 Web 界面来查看当前集群的各种信息。用户可以 用 Kubernetes Dashboard 部署容器化的应用、监控应用的状态、执行故障排查任务以及管 理 Kubernetes 各种资源。
(1)下载yaml文件:
拉取并推送实验所需镜像到私有仓库
(2)应用yaml文件:
(3) 修改为NodePort类型,供外部访问
kubectl edit svc -n kubernetes-dashboard
(4)获取dashboard pod的token
访问测试https://172.25.12.2:31416
复制token值,成功登陆
更多推荐
所有评论(0)