原生Kubernetes容器云平台应用部署与运维

IP主机名节点192.168.117.14mastermaster节点192.168.117.15nodenode节点192.168.117.16node2新增节点环境准备（所有节点）配置主机映射[root@master ~]# vim /etc/hosts192.168.117.14master192.168.117.15node192.168.117.16node2关闭Swap[root@ma

cruize3

2578人浏览 · 2020-06-16 15:54:07

cruize3 · 2020-06-16 15:54:07 发布

IP	主机名	节点
192.168.117.14	master	master节点
192.168.117.15	node	node节点
192.168.117.16	node2	新增节点

环境准备（所有节点）

配置主机映射

[root@master ~]# vim /etc/hosts
192.168.117.14  master
192.168.117.15  node
192.168.117.16  node2

关闭Swap

[root@master ~]# swapoff -a
[root@master ~]# sed -i 's@\(.*a2f86.*\)@\#\1@g' /etc/fstab

配置时间同步

master节点：
[root@master ~]# yum install -y chrony
[root@master ~]# sed -i 's/^server/#&/' /etc/chrony.conf
[root@master ~]# vim /etc/chrony.conf     //添加以下配置
local stratum 10
server master iburst
allow all
[root@master ~]# systemctl enable chronyd
[root@master ~]# systemctl restart chronyd
[root@master ~]# timedatectl set-ntp true

node节点：
[root@node ~]# yum install -y chrony
[root@node ~]# sed -i 's/^server/#&/' /etc/chrony.conf
[root@node ~]# vim /etc/chrony.conf    //添加如下配置
server 192.168.117.14 iburst
[root@node ~]# systemctl enable chronyd
[root@node ~]# systemctl restart chronyd

配置路由转发

[root@master ~]# echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.d/k8s.conf
[root@master ~]# echo 'net.bridge.bridge-nf-call-ip6tables = 1' >> /etc/sysctl.d/k8s.conf
[root@master ~]# echo 'net.bridge.bridge-nf-call-iptables = 1' >> /etc/sysctl.d/k8s.conf
[root@master ~]# modprobe br_netfilter
[root@master ~]# sysctl -p /etc/sysctl.d/k8s.conf 
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1

配置IPVS

[root@master ~]# vim /etc/sysconfig/modules/ipvs.modules
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
[root@master ~]# chmod 755 /etc/sysconfig/modules/ipvs.modules 
[root@master ~]# sh /etc/sysconfig/modules/ipvs.modules

安装ipset软件包

[root@master ~]# yum install -y ipset ipvsadm

添加阿里云yum源并安装Docker，执行提供的kubernetes_base.sh脚本获取镜像

[root@master ~]# wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
[root@master ~]# wget -O /etc/yum.repos.d/docker-ce.repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
[root@master ~]# yum clean all
[root@master ~]# yum makecache
[root@master ~]# yum install -y yum-utils device-mapper-persistent-data
[root@master ~]# yum install -y docker-ce docker-ce-cli containerd.io
[root@master ~]# mkdir /etc/docker
[root@master ~]# vim /etc/docker/daemon.json
{
    "exec-opts":["native.cgroupdriver=systemd"]
}
[root@master ~]# systemctl start docker
[root@master ~]# ./kubernetes_base.sh 
[root@master ~]# systemctl daemon-reload
[root@master ~]# systemctl restart docker
[root@master ~]# systemctl enable docker

配置Kubernetes yum源

[root@master ~]# vim /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

安装Kubernetes工具并启动Kubelet

[root@master ~]# yum install -y kubelet-1.14.1 kubeadm-1.14.1 kubectl-1.14.1
[root@master ~]# systemctl enable --now kubelet

案例实施

部署wordpress应用

进入master节点，初始化Kubernetes集群

[root@master ~]# kubeadm init --apiserver-advertise-address 192.168.117.14 --kubernetes-version="v1.14.1" --pod-network-cidr=192.188.0.0/16 --image-repository=registry.aliyuncs.com/google_containers
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.117.14:6443 --token w8ybfg.qhsi4tvrrhk2h2xz \
    --discovery-token-ca-cert-hash sha256:575ddcd924f82ace0768212c175a593a198cfde5e23f972d06cec875eded0d27

拷贝config配置文件

[root@master ~]# mkdir .kube 
[root@master ~]# cp /etc/kubernetes/admin.conf .kube/config

用提供的kube-flannel.yml文件安装网络

[root@master ~]# kubectl apply -f yaml/kube-flannel.yaml

在两个node节点上加入集群

[root@node ~]#  kubeadm join 192.168.117.14:6443 --token w8ybfg.qhsi4tvrrhk2h2xz \
    --discovery-token-ca-cert-hash sha256:575ddcd924f82ace0768212c175a593a198cfde5e23f972d06cec875eded0d27

创建Pod

[root@master ~]# vim test.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: pod-test
  labels:
    os: centos
spec:
  containers:
  - name: hello
    image: nginx:latest
    imagePullPolicy: Never
    env:
    - name: Test
      value: "123456"
    command: ["bash","-c","while true;do date;sleep 1;done"]
[root@master ~]# kubectl create -f test.yaml

查看Pod状态

[root@master ~]# kubectl get pods
NAME       READY   STATUS    RESTARTS   AGE
pod-test   1/1     Running   0          27m

新建一个命名空间以及Deployment对象

[root@master ~]# kubectl create namespace blog
[root@master ~]# vim wordpress-db.yaml
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: mysql-deploy
  namespace: blog
  labels:
    app: mysql
spec:
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: mysql:5.6
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 3306
          name: dbport
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: rootPassWord
        - name: MYSQL_DATABASE
          value: wordpress
        - name: MYSQL_USER
          value: wordpress
        - name: MYSQL_PASSWORD
          value: wordpress
        volumeMounts:
        - name: db
          mountPath: /var/lib/mysql
      volumes:
      - name: db
        hostPath:
          path: /var/lib/mysql
---
apiVersion: v1
kind: Service
metadata:
  name: mysql
  namespace: blog
spec:
  selector:
    app: mysql
  ports:
  - name: mysqlport
    protocol: TCP
    port: 3306
    targetPort: dbport
[root@master ~]# kubectl create -f wordpress-db.yaml

查看Service的详细情况

[root@master ~]# kubectl describe svc mysql -n blog
Name:              mysql
Namespace:         blog
Labels:            <none>
Annotations:       <none>
Selector:          app=mysql
Type:              ClusterIP
IP:                10.106.156.104
Port:              mysqlport  3306/TCP
TargetPort:        dbport/TCP
Endpoints:         192.188.1.3:3306
Session Affinity:  None
Events:            <none>

创建Wordpress服务

[root@master ~]# vim wordpress.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: wordpress-deploy
  namespace: blog
  labels:
    app: wordpress
spec:
  template:
    metadata:
      labels:
        app: wordpress
    spec:
      containers:
      - name: wordpress
        image: wordpress
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
          name: wdport
        env:
        - name: WORDPRESS_DB_HOST
          value: 10.106.156.104:3306    //对应mysql服务的clusterIP
        - name: WORDPRESS_DB_USER
          value: wordpress
        - name: WORDPRESS_DB_PASSWORD
          value: wordpress
---
apiVersion: v1
kind: Service
metadata:
  name: wordpress
  namespace: blog
spec:
  type: NodePort
  selector:
    app: wordpress
  ports:
  - name: wordpressport
    protocol: TCP
    port: 80
    targetPort: wdport
[root@master ~]# kubectl create -f wordpress.yaml

创建wordpress-pod

[root@master ~]# vim wordpress-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: wordpress
  namespace: blog
spec:
  containers:
  - name: wordpress
    image: wordpress
    imagePullPolicy: IfNotPresent
    ports:
    - containerPort: 80
      name: wdport
    env:
    - name: WORDPRESS_DB_HOST
      value: localhost:3306
    - name: WORDPRESS_DB_USER
      value: wordpress
    - name: WORDPRESS_DB_PASSWORD
      value: wordpress
  - name: mysql
    image: mysql:5.6
    imagePullPolicy: IfNotPresent
    ports:
    - containerPort: 3306
      name: dbport
    env:
    - name: MYSQL_ROOT_PASSWORD
      value: rootPassWord
    - name: MYSQL_DATABASE
      value: wordpress
    - name: MYSQL_USER
      value: wordpress
    - name: MYSQL_PASSWORD
      value: wordpress
    volumeMounts:
    - name: db
      mountPath: /var/lib/mysql
  volumes:
  - name: db
    hostPath:
      path: /var/lib/mysql
[root@master ~]# kubectl create -f wordpress-pod.yaml

查看服务信息

[root@master ~]# kubectl get svc -n blog
NAME        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
mysql       ClusterIP   10.106.156.104   <none>        3306/TCP       6m8s
wordpress   NodePort    10.98.162.136    <none>        80:30861/TCP   34s

浏览器访问wordpress应用

Node的隔离与恢复

隔离node节点/恢复node节点调度

[root@master ~]# kubectl cordon node
[root@master ~]# kubectl get nodes
NAME     STATUS                          ROLES    AGE    VERSION
master   Ready                              master   5d3h   v1.14.1
node     Ready,SchedulingDisabled   <none>   5d3h   v1.14.1
node2    Ready    <none>   14h     v1.14.1

[root@master ~]# kubectl uncordon node
[root@master ~]# kubectl get nodes
NAME     STATUS   ROLES    AGE    VERSION
master   Ready    master   5d3h   v1.14.1
node     Ready    <none>   5d3h   v1.14.1
node2    Ready    <none>   14h     v1.14.1

Pod动态扩容和缩放

运行Deployment

[root@master ~]# kubectl run nginx --image=nginx:latest
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/nginx created

Pod扩容，将Nginx Deployment控制的Pod副本数量从初始的1扩容至5

[root@master ~]# kubectl scale deployment nginx --replicas=5
deployment.extensions/nginx scaled

Pod缩容，将Nginx Deployment控制的Pod副本数量从5缩容至2

[root@master ~]# kubectl scale deployment nginx --replicas=2
deployment.extensions/nginx scaled

将Pod调度到指定的Node

添加/删除标签

[root@master ~]# kubectl label nodes node project=gcxt
node/node labeled
[root@master ~]# kubectl label node node project-
node/node labeled

调度Pod到指定Node节点

[root@master ~]# kubectl label nodes node project=gcxt
[root@master ~]# vim nginx.yaml
apiVersion: v1
kind: ReplicationController
metadata:
  name: memcached-gcxt
  labels:
    name: memcached-gcxt
spec:
  replicas: 1
  selector:
    name: memcached-gcxt
  template:
    metadata:
      labels:
        name: memcached-gcxt
    spec:
      containers:
      - name: memcached-gcxt
        image: memcached
        command:
        - memcached
        - -m 64
        ports:
        - containerPort: 11211
      nodeSelector:
        project: gcxt
[root@master ~]# kubectl create -f nginx.yaml

查看Pod，可以看到Pod被调度到node节点

[root@master ~]# kubectl get pods -owide
NAME                    READY   STATUS    RESTARTS   AGE     IP             NODE    NOMINATED NODE   READINESS GATES
memcached-gcxt-d5r4r    1/1     Running   0          23m   192.188.2.2    node    <none>           <none>

应用滚动升级

启动Deployment

[root@master ~]# vim httpd.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: httpd
spec:
  replicas: 3
  template:
    metadata:
      labels:
        run: httpd
    spec:
      containers:
        - name: httpd
          image: httpd:2.2.31
          ports:
            - containerPort: 80
[root@master ~]# kubectl create -f httpd.yaml

查看Pod和Deployment状态

[root@master ~]# kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
httpd-5ddb558f47-cg6rc   1/1     Running   0          62s
httpd-5ddb558f47-tpzl5   1/1     Running   0          62s
httpd-5ddb558f47-ts7mf   1/1     Running   0          62s
pod-test                 1/1     Running   6          2d16h
[root@master ~]# kubectl get deployments httpd -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES         SELECTOR
httpd   3/3     3            3           80s   httpd        httpd:2.2.31   run=httpd

滚动升级，将httpd.yaml配置文件中的httpd版本改为2.2.32，再次启动

[root@master ~]# vim httpd.yaml
          image: httpd:2.2.32
[root@master ~]# kubectl apply -f httpd.yaml

查看Deployment的详细信息，其中的日志信息描述了滚动升级的过程

[root@master ~]# kubectl describe deployment httpd
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  5m5s  deployment-controller  Scaled up replica set httpd-5ddb558f47 to 3
  Normal  ScalingReplicaSet  66s   deployment-controller  Scaled up replica set httpd-8bdffc6d8 to 1
  Normal  ScalingReplicaSet  64s   deployment-controller  Scaled down replica set httpd-5ddb558f47 to 2
  Normal  ScalingReplicaSet  64s   deployment-controller  Scaled up replica set httpd-8bdffc6d8 to 2
  Normal  ScalingReplicaSet  63s   deployment-controller  Scaled down replica set httpd-5ddb558f47 to 1
  Normal  ScalingReplicaSet  63s   deployment-controller  Scaled up replica set httpd-8bdffc6d8 to 3
  Normal  ScalingReplicaSet  61s   deployment-controller  Scaled down replica set httpd-5ddb558f47 to 0

创建3个配置文件，唯一不同之处是镜像的版本号

[root@master ~]# vim httpd.v1.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: httpd
spec:
  revisionHistoryLimit: 10
  replicas: 3
  template:
    metadata:
      labels:
        run: httpd
    spec:
      containers:
        - name: httpd
          image: httpd:2.2.16
          ports:
            - containerPort: 80
[root@master ~]# vim httpd.v2.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: httpd
spec:
  revisionHistoryLimit: 10
  replicas: 3
  template:
    metadata:
      labels:
        run: httpd
    spec:
      containers:
        - name: httpd
          image: httpd:2.2.17
          ports:
            - containerPort: 80
[root@master ~]# vim httpd.v3.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: httpd
spec:
  revisionHistoryLimit: 10
  replicas: 3
  template:
    metadata:
      labels:
        run: httpd
    spec:
      containers:
        - name: httpd
          image: httpd:2.2.18
          ports:
            - containerPort: 80

部署Deployment

[root@master ~]# kubectl apply -f httpd.v1.yaml --record
deployment.apps/httpd configured
[root@master ~]# kubectl apply -f httpd.v2.yaml --record
deployment.apps/httpd configured
[root@master ~]# kubectl apply -f httpd.v3.yaml --record
deployment.apps/httpd configured

查看Deployment

[root@master ~]# kubectl get deployments -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES         SELECTOR
httpd   3/3     1            3           8m28s   httpd        httpd:2.2.18   run=httpd

查看revision历史记录

[root@master ~]# kubectl rollout history deployment httpd
deployment.extensions/httpd 
REVISION  CHANGE-CAUSE
1         <none>
2         <none>
3         kubectl apply --filename=httpd.v1.yaml --record=true
4         kubectl apply --filename=httpd.v2.yaml --record=true
5         kubectl apply --filename=httpd.v3.yaml --record=true

回滚到指定版本revision 1

[root@master ~]# kubectl rollout undo deployment httpd --to-revision=1
deployment.extensions/httpd rolled back
[root@master ~]# kubectl get deployments -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES         SELECTOR
httpd   3/3     3            3           10m   httpd        httpd:2.2.31   run=httpd

再次查看revision历史记录，可以看到revision记录也相应增加

[root@master ~]# kubectl rollout history deployment httpd
deployment.extensions/httpd 
REVISION  CHANGE-CAUSE
2         <none>
3         kubectl apply --filename=httpd.v1.yaml --record=true
4         kubectl apply --filename=httpd.v2.yaml --record=true
5         kubectl apply --filename=httpd.v3.yaml --record=true
6         <none>

Cloudpods

开源、云原生的融合云平台

更多推荐

面向未来的 IT 基础设施管理架构——融合云（Unified IaaS）

随着数字化时代的到来，IT系统已成为人类社会正常运转不可或缺的组成部分。不远的未来，智能制造，5G和人工智能等技术将成为推动生产力发展的重要引擎，人类社会将面临前所未有的全面彻底的数字化浪潮。IT基础设施作为IT系统运行的平台和载体，是实现数字化的基石。在这场数字化浪潮中，企业必须积极拥抱云计算技术，采用符合技术发展趋势、面向未来的IT基础构架，才能在未来的竞争中赢得先机。一、云计算历经十余年

Cloudpods

Cloudpods负载均衡的功能介绍

作者:周有松今天的内容会从以下几个方面展开：负载均衡产品简介。主要介绍负载均衡作为一个云上产品，它的功能模型是怎样的，日常使用中会遇到的业务词汇负载均衡的功能与典型应用场景。这部分主要结合业务词汇，对负载均衡服务中常见的一些功能选项进行介绍，并举例介绍一些典型的应用场景最后，我们做一下总结，讨论一下负载均衡产品相比传统方式的优点一、产品简介 1. 以NGINX为例提到负载均衡，我们以

Cloudpods

使用Linux vfio将Nvidia GPU透传给QEMU虚拟机

Linux 上虚拟机 GPU 透传需要使用 vfio 的方式。主要是因为在 vfio 方式下对虚拟设备的权限和 DMA 隔离上做的更好。但是这么做也有个缺点，这个物理设备在主机和其他虚拟机都不能使用了。 qemu 直接使用物理设备本身命令行是很简单的，关键在于事先在主机上对系统、内核和物理设备的一些配置。单纯从 qemu 的命令行来看，其实和普通虚拟机启动就差了最后那个-device的选项。这

Cloudpods

所有评论(0)

查看更多评论

cruize3

@un_locked

已为社区贡献2条内容