k8s配置中心

在生产环境中经常会遇到需要修改配置文件的情况，传统的修改方式不仅会影响到服务的正常运行，而且操作步骤也很繁琐。为了解决这个问题，kubernetes项目从1.2版本引入了ConfigMap功能，ConfigMap && Secret 是K8S中的针对应用的配置中心，它有效的解决了应用挂载的问题，并且支持加密以及热更新等功能，可以说是一个k8s提供的一件非常好用的功能。

configmap

configmap用于将应用的配置信息与程序的分离。这种方式不仅可以实现应用程序被的复用，而且还可以通过不同的配置实现更灵活的功能。在创建容器时，用户可以将应用程序打包为容器镜像后，通过环境变量或者外接挂载文件的方式进行配置注入。
注意：configmap会先覆盖挂载的文件夹，所以挂载需谨慎，挂载文件时可以使用相对路径

创建configmap

示例1：指定配置文件

#编辑配置文件
[root@k8s-master-01 ~]# vim test.yaml
kind: Pod
apiVersion: v1
metadata:
  name: tcp-pod
spec:
  containers:
    - name: nginx
      imagePullPolicy: IfNotPresent
      image: nginx
      readinessProbe:
        tcpSocket:
          port: 80

# 指定文件创建configmap
[root@k8s-master-01 ~]# kubectl create configmap file --from-file=test.yaml
configmap/file created

#获取configmap
[root@k8s-master-01 ~]# kubectl get configmaps 
NAME               DATA   AGE
file               1      39s
kube-root-ca.crt   1      21h

#查看详细描述信息
[root@k8s-master-01 ~]# kubectl describe configmaps file 
Name:         file
Namespace:    default
Labels:       <none>
Annotations:  <none>

Data
====
test.yaml:
----
kind: Pod
apiVersion: v1
metadata:
  name: tcp-pod
spec:
  containers:
    - name: nginx
      imagePullPolicy: IfNotPresent
      image: nginx
      readinessProbe:
        tcpSocket:
          port: 80

Events:  <none>

示例2：指定配置目录（configmap里的内容是目录中所有文件的内容）

[root@k8s-master-01 ~]# mkdir configmap
[root@k8s-master-01 ~]# touch configmap/index.html
[root@k8s-master-01 ~]# echo "index" > configmap/index.html

# 创建configmap
[root@k8s-master-01 ~]# kubectl create configmap catalog --from-file=configmap/
configmap/catalog created

# 查看configmap详细描述
[root@k8s-master-01 ~]# kubectl describe configmaps catalog
Name:         catalog
Namespace:    default
Labels:       <none>
Annotations:  <none>
Data
====
index.html:
----
index
Events:  <none>

示例3：指定配置项（类似于指定变量）

[root@k8s-master-01 ~]# kubectl create configmap calue --from-literal=key=value --from-literal=key1=value1
configmap/calue created
[root@k8s-master-01 ~]# kubectl describe configmaps calue
Name:         calue
Namespace:    default
Labels:       <none>
Annotations:  <none>

Data
====
key:
----
value
key1:
----
value1
Events:  <none>

示例4：通过配置清单的方式创建configmap

[root@k8s-master-01 ~]# cat test.yaml 
kind: ConfigMap
apiVersion: v1
metadata:
  name: configmap-yaml
  labels:
    app: configmap
data:
  key: value
  nginx_config: |-
    upstream tomcatserver1 {
      server 192.168.13.13:8081;
    }
    upstream tomcatserver2 {
      server 192.168.13.13:8082;
    }
    server {
      listen 80;
      server_name 8081.max.com;
      location / {
        proxy_pass http://tomcatserver1;
        index index.html index.htm;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
      }
    }

[root@k8s-master-01 ~]# kubectl apply -f test.yaml
configmap/configmap-yaml created

[root@k8s-master-01 ~]# kubectl describe configmaps configmap-yaml
Name:         configmap-yaml
Namespace:    default
Labels:       app=configmap
Annotations:  <none>

Data
====
key:
----
value
nginx_config:
----
upstream tomcatserver1 {
  server 192.168.13.13:8081;
}
upstream tomcatserver2 {
  server 192.168.13.13:8082;
}
server {
  listen 80;
  server_name 8081.max.com;
  location / {
    proxy_pass http://tomcatserver1;
    index index.html index.htm;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
  }
}
Events:  <none>
使用configmap

一般情况下，我们是通过挂载的方式使用configmap。

[root@k8s-master-01 ~]# cat test.yaml 
kind: ConfigMap
apiVersion: v1
metadata:
  name: configmap-yaml
  labels:
    app: configmap
data:
  key: value
  nginx_config: |-
    upstream tomcatserver1 {
      server 192.168.13.13:8081;
    }
    upstream tomcatserver2 {
      server 192.168.13.13:8082;
    }
    server {
      listen 80;
      server_name 8081.max.com;
      location / {
        proxy_pass http://tomcatserver1;
        index index.html index.htm;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
      }
   
---
kind: Pod
apiVersion: v1
metadata:
  name: configmap-pod
  labels:
    app: configmap-pod
spec:
  containers:
    - name: nginx
      image: nginx
      imagePullPolicy: IfNotPresent
      ports:
        - containerPort: 80
      volumeMounts:
        - mountPath: /usr/share/nginx/demo
          name: conf
  volumes:
    - name: conf
      configMap:
        name: configmap-yaml
        items:
          - key: nginx_config
            path: nginx_config
          - key: key
            path: key 
[root@k8s-master-01 ~]# kubectl apply -f test.yaml
configmap/configmap-yaml created
pod/configmap-pod created

[root@k8s-master-01 ~]# kubectl get pod
NAME                                          READY   STATUS    RESTARTS   AGE
configmap-pod                                 1/1     Running   0          3m6s


[root@k8s-master-01 ~]# kubectl exec -it configmap-pod -- bash
root@configmap-pod:/# cd /usr/share/nginx/
root@configmap-pod:/usr/share/nginx# ls
demo  html
root@configmap-pod:/usr/share/nginx# cd demo/
root@configmap-pod:/usr/share/nginx/demo# ls
key  nginx_config

root@configmap-pod:/usr/share/nginx/demo# cat nginx_config 
upstream tomcatserver1 {
  server 192.168.13.13:8081;
}
upstream tomcatserver2 {
  server 192.168.13.13:8082;
}
server {
  listen 80;
  server_name 8081.max.com;
  location / {
    proxy_pass http://tomcatserver1;
    index index.html index.htm;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
  }
  
root@configmap-pod:/usr/share/nginx/demo# cat key 
value

configmap的热更新

注意：热更新需要一段时间，因为需要更改etcd中的数据
示例：

[root@k8s-master-01 ~]# cat > test.yaml <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      run: my-nginx
  template:
    metadata:
      labels:
        run: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx
        ports:
        - containerPort: 80
        envFrom:
        - configMapRef:
            name: env-config
        volumeMounts:
          - mountPath: /usr/share/nginx/demo/
            name: config
      volumes:
        - name: config
          configMap:
            name: env-config
            items:
              - key: log_level
                path: log_level
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: env-config
  namespace: default
data:
  log_level: INFO
EOF
[root@k8s-master-01 ~]# kubectl apply -f test.yaml 
deployment.apps/my-nginx created
configmap/env-config created
[root@k8s-master-01 ~]#  kubectl exec my-nginx-6947589dc-8wwrd  -- cat /usr/share/nginx/demo/log_level
INFO

#编辑配置文件
[root@k8s-master-01 kubectl edit configmaps env-configevel
更改如下图：将INFO改为DEBU
configmap/env-config edited

#过一会儿查看（这里不要立即查看，立即查看可能会导致新数据还没来得及生成）
[root@k8s-master-01 ~]#  kubectl exec my-nginx-6947589dc-8wwrd  -- cat /usr/share/nginx/demo/log_level
DEBU

在这里插入图片描述

subPath

mountPath结合subPath(也可解决多个configmap挂载同一目录，导致覆盖)作用。注意：使用subPath就失去热更新了，挂载文件时不可以使用相对路径，必须指定文件名

示例：

[root@k8s-master-01 ~]# vim test.yaml 
    upstream tomcatserver2 {
      server 192.168.13.13:8082;
    }
    server {
      listen 80;
      server_name 8081.max.com;
      location / {
        proxy_pass http://tomcatserver1;
        index index.html index.htm;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
      }

---
kind: Pod
apiVersion: v1
metadata:
  name: configmap-pod
  labels:
    app: configmap-pod
spec:
  containers:
    - name: nginx
      image: nginx
      imagePullPolicy: IfNotPresent
      ports:
        - containerPort: 80
      volumeMounts:
        - mountPath: /usr/share/nginx/html/nginx.conf
          name: conf
          subPath: nginx_config
  volumes:
    - name: conf
      configMap:
        name: configmap-yaml
        items:
          - key: nginx_config
            path: nginx_config

[root@k8s-master-01 ~]#  kubectl apply -f test.yaml
configmap/configmap-yaml created
pod/configmap-pod created

[root@k8s-master-01 ~]# kubectl get pod
NAME                                          READY   STATUS    RESTARTS   AGE
configmap-pod                                 1/1     Running   0          31s

[root@k8s-master-01 ~]# kubectl exec -it configmap-pod -- bash
root@configmap-pod:/# cd /usr/share/nginx/html/
root@configmap-pod:/usr/share/nginx/html# ls
50x.html  index.html  nginx.conf
root@configmap-pod:/usr/share/nginx/html# cat nginx.conf 
upstream tomcatserver1 {
  server 192.168.13.13:8081;
}
upstream tomcatserver2 {
  server 192.168.13.13:8082;
}
server {
  listen 80;
  server_name 8081.max.com;
  location / {
    proxy_pass http://tomcatserver1;
    index index.html index.htm;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;

secret

Secret解决了密码、token、密钥等敏感数据的配置问题，Secret 可以作为数据卷被挂载，或作为环境变量 暴露出来以供 Pod 中的容器使用而不需要把这些敏感数据暴露到镜像或者Pod Spec中。Secret可以以Volume或者环境变量的方式使用,但这只是一种相对的安全，可以通过编码格式将密码反解出来。
反解示例：

[root@k8s-master-01 ~]# echo "123456" | base64
MTIzNDU2Cg==
[root@k8s-master-01 ~]# echo "MTIzNDU2Cg==" | base64 -d
123456

Secret有三种类型：
Service Account ：用来访问Kubernetes API，由Kubernetes自动创建，并且会自动挂载到Pod的/run/secrets/kubernetes.io/serviceaccount目录中；
Opaque ：base64编码格式的Secret，用来存储密码、密钥等；
kubernetes.io/dockerconfigjson ：用来存储私有docker registry的认证信息。

Opaque Secret

Opaque类型的数据是一个map类型，要求value是base64编码格式，不是密文会报错
示例：

[root@k8s-master-01 ~]#  echo -n "oldboy" | base64
b2xkYm95
[root@k8s-master-01 ~]#  echo -n "oldboy123" | base64
b2xkYm95MTIz

编写secret配置清单
[root@k8s-master-01 ~]# vim secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
type: Opaque
data:
  password: b2xkYm95MTIz
  username: b2xkYm95

创建secret资源

[root@k8s-master-01 ~]#  kubectl apply -f secret.yaml
secret/mysecret created
[root@k8s-master-01 ~]# kubectl describe secrets mysecret
Name:         mysecret
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
password:  10 bytes
username:  6 bytes

编写测试应用配置清单

kind: Deployment
apiVersion: apps/v1
metadata:
  name: mysecret
spec:
  selector:
    matchLabels:
      app: mysecret
  template:
    metadata:
      labels:
        app: mysecret
    spec:
      containers:
        - name: nginx
          imagePullPolicy: IfNotPresent
          image: nginx
          volumeMounts:
            - name: mysecret
              mountPath: "/opt/secrets"
              readOnly: true
      volumes:
        - name: mysecret
          secret:
            secretName: mysecret

创建测试资源

[root@k8s-master-01 ~]#  kubectl apply -f secret.yaml
deployment.apps/mysecret created
secret/mysecret unchanged
[root@k8s-master-01 ~]# kubectl exec -it mysecret-5bcb897fff-77bn5  -- bash
root@mysecret-5bcb897fff-77bn5:/# cd /opt/secrets/
root@mysecret-5bcb897fff-77bn5:/opt/secrets# ls
password  username
root@mysecret-5bcb897fff-77bn5:/opt/secrets# cat username
oldboyroot@mysecret-5bcb897fff-77bn5:/opt/secrets# cat password
oldboy123
kubernetes.io/dockerconfigjson

用来存储私有docker registry的认证信息。

创建secret

export DOCKER_REGISTRY_SERVER=10.0.0.100
export DOCKER_USER=root
export DOCKER_PASSWORD=root@123
export DOCKER_EMAIL=root@123.com

kubectl create secret docker-registry myregistrykey --docker-server=DOCKER_REGISTRY_SERVER --docker-username=DOCKER_USER --docker-password=DOCKER_PASSWORD --docker-email=DOCKER_EMAIL

[root@k8s-master-01 ~]#  export DOCKER_REGISTRY_SERVER=10.0.0.100
[root@k8s-master-01 ~]#  export DOCKER_USER=root
[root@k8s-master-01 ~]# export DOCKER_PASSWORD=root@123
[root@k8s-master-01 ~]#  export DOCKER_EMAIL=root@123.com

[root@k8s-master-01 ~]#  kubectl create secret docker-registry myregistrykey --docker-server=DOCKER_REGISTRY_SERVER --docker-username=DOCKER_USER --docker-password=DOCKER_PASSWORD --docker-email=DOCKER_EMAIL
secret/myregistrykey created
[root@k8s-master-01 ~]#  kubectl describe secret myregistrykey
Name:         myregistrykey
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  kubernetes.io/dockerconfigjson

Data
====
.dockerconfigjson:  161 bytes

创建测试secret

kind: Deployment
apiVersion: apps/v1
metadata:
  name: mysecret
spec:
  selector:
    matchLabels:
      app: mysecret
  template:
    metadata:
      labels:
        app: mysecret
    spec:
      containers:
        - name: nginx
      imagePullSecrets:
        - name: myregistrykey
Service Account

Service Account用来访问Kubernetes API，由Kubernetes自动创建，并且会自动挂载到Pod的/run/secrets/kubernetes.io/serviceaccount目录中。

[root@k8s-master-01 ~]#  kubectl exec nginx-6cf7488b57-bd887 -- ls /run/secrets/kubernetes.io/serviceaccount
ca.crt
namespace
Token

部署中遇到的问题

问题：进入容器失败
[root@k8s-master-01 ~]# kubectl exec -it configmap-pod – bash
Error from server: error dialing backend: dial tcp 192.168.13.13:10250: connect: no route to host

问题分析：告诉你没有主机或路由，可能是机器没起来或者防火墙没关

问题解决：
检查Pod所在节点的防火墙是否开启

#查看Pod所在的Node
kubectl get pods -o wide
#在Node节点查看防火墙状态
systemctl status firewalld
#如果出现绿色running状态，可考虑关闭防火墙，或通过iptables放行入方向10250端口，这里仅关闭防火墙
systemctl disable --now firewalld

问题：Error getting node" err="node “k8s-master-01” not found"
问题分析：可能是kubelet.conf中的ip不对

问题解决：查看IP是否是当前master节点在配置文件中是否正确，正确的话，查看master节点与工作节点上的kubelet服务是否启动（kubernetes启动要有两个主节点）
其他节点：
[root@k8s-master-02 ~]# systemctl enable --now kubelet
[root@k8s-node-02 ~]# systemctl enable --now kubelet.service
[root@k8s-node-01 ~]# systemctl enable --now kubelet.service

Logo
K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 |  韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号​前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。​长夜有穷，真水无香。领略过了 K8s“身在江

avatar K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub

avatar K8S/Kubernetes

k8s 火了！

2020，上云之年，产品云端化成为一种趋势。在一线城市，很多公司都已经构建了自己的私有云环境，比如阿里云、网易云、华为云等。而Kubernetes 作为基于容器编排领域的王者，具备扩展...

avatar K8S/Kubernetes