Kubernetes CKA认证运维工程师笔记-Kubernetes项目部署案例

1. 容器交付流程

在这里插入图片描述

本地开发阶段持续集成/持续交付部署应用运维
代码编写代码编译持续交付监控告警
Dockerfile编写构建镜像规划部署拓扑日志收集
编排模板编写推送镜像环境配置故障排查
容器化测试部署编排模板扩缩容配置持续优化

2. 在K8s平台部署项目流程

在这里插入图片描述

制作镜像使用控制器部署镜像对外暴露应用日志、监控日常运维
dockerfileDeployment、StatefulSet、DaemonSetService、IngressPrometheus+Grafana、ELK Stack

3. 在K8s平台部署Java网站项目

熟悉项目：

  1. 项目代码构成
  2. 依赖的服务
  3. 提供服务的端口
  4. 配置文件
  5. 程序在工作中涉及持久化的文件

java:jdk、maven
mysql
Tomcat
src/main/resources/application.yml
pv、pvc

镜像分类：

  1. 基础镜像，例如：centos、ubuntu
  2. 环境镜像，例如：jdk、nginx、tomcat
  3. 项目镜像，例如：dashboard

第一步：制作镜像

yum install java-1.8.0-openjdk maven git -y
git clone https://github.com/lizhenliang/tomcat-java-demo
mvn clean package -Dmaven.test.skip=true # 代码编译构建
unzip target/*.war -d target/ROOT # 解压构建文件

FROM tomcat
LABEL maintainer lizhenliang
COPY target/ROOT /usr/local/tomcat/webapps/ROOT

docker build -t image:tag .
docker push <镜像仓库地址>/<项目名>/image:tag     #demo:1.0\demo:1.1\demo:1.2

docker run -d image:tag

第一步：制作镜像
使用镜像仓库（私有仓库、公共仓库）：
1、配置可信任（如果仓库是HTTPS访问不用配置）

# vi /etc/docker/daemon.json
{
"insecure-registries": ["10.0.0.65"]
}

2、将镜像仓库认证凭据保存在K8s Secret中

kubectl create secret docker-registry registry-auth --docker-username=admin --docker-password=Harbor12345 --docker-server=10.0.0.65

3、在yaml中使用这个认证凭据
imagePullSecrets:
-name: registry-auth

第二步：使用控制器部署镜像

# 控制器定义
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
spec:
replicas: 1
selector:
matchLabels:
app: web

# 被控制对象
template:
metadata:
labels:
app: web
spec:
containers:
-image: lizhenliang/java-demo
name: java-demo

Pod主要配置启动容器属性：

  • 变量
  • 资源配额
  • 健康检查
  • 卷挂载点

使用configmap保存项目配置文件

第三步：对外暴露应用

示例：Nginx作为负载均衡器

upstream java-demo {
server 192.168.31.62:80;
server 192.168.31.63:80;
}
server {
listen 80;
server_name example.ctnrs.com;
location / {
proxy_pass http://java-demo;
proxy_set_header Host $Host;
}
}

应用访问架构图
在这里插入图片描述

[root@k8s-master 项目部署]# cd tomcat-java-demo-master/
[root@k8s-master tomcat-java-demo-master]# ls
db  LICENSE  pom.xml  README.md  src
[root@k8s-master tomcat-java-demo-master]# cd src/
[root@k8s-master src]# ls
main
[root@k8s-master src]# cd main/
[root@k8s-master main]# ls
java  resources
[root@k8s-master main]# ls java/
com
[root@k8s-master main]# ld resources/
resources/: file not recognized: Is a directory
[root@k8s-master main]# ls resources/
application.yml  log4j.properties  static  templates
[root@k8s-master main]# ls db
ls: cannot access db: No such file or directory
[root@k8s-master main]# ls db/
ls: cannot access db/: No such file or directory
[root@k8s-master main]# cd ..
[root@k8s-master src]# ls db/
ls: cannot access db/: No such file or directory
[root@k8s-master src]# cd ..
[root@k8s-master tomcat-java-demo-master]# ls db/
tables_ly_tomcat.sql
[root@k8s-master tomcat-java-demo-master]# cat src/main/resources/application.yml 
server:
  port: 8080
spring:
  datasource:
    url: jdbc:mysql://localhost:3306/test?characterEncoding=utf-8
    username: root
    password: 12345
    driver-class-name: com.mysql.jdbc.Driver
  freemarker:
    allow-request-override: false
    cache: true
    check-template-location: true
    charset: UTF-8
    content-type: text/html; charset=utf-8
    expose-request-attributes: false
    expose-session-attributes: false
    expose-spring-macro-helpers: false
    suffix: .ftl
    template-loader-path:
      - classpath:/templates/
[root@k8s-master tomcat-java-demo-master]# yum install java-1.8.0-openjdkit -y
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
Package 1:java-1.8.0-openjdk-1.8.0.312.b07-1.el7_9.x86_64 already installatest version
Package maven-3.0.5-17.el7.noarch already installed and latest version
Package git-1.8.3.1-23.el7_8.x86_64 already installed and latest version
Nothing to do
[root@k8s-master tomcat-java-demo-master]# mvn clean package -Dmaven.test.skip=true
...
[INFO] Building war: /root/项目部署/tomcat-java-demo-master/target/ly-simple-tomcat-0.0.1-SNAPSHOT.war
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4:57.598s
[INFO] Finished at: Mon Dec 27 10:28:54 CST 2021
[INFO] Final Memory: 26M/140M
[INFO] ------------------------------------------------------------------------

[root@k8s-master tomcat-java-demo-master]# ls
db  LICENSE  pom.xml  README.md  src  target
[root@k8s-master tomcat-java-demo-master]# ls target/
classes            ly-simple-tomcat-0.0.1-SNAPSHOT      maven-archiver
generated-sources  ly-simple-tomcat-0.0.1-SNAPSHOT.war  maven-status
[root@k8s-master tomcat-java-demo-master]# unzip target/ly-simple-tomcat-0.0.1-SNAPSHOT.war -d target/ROOT
Archive:  target/ly-simple-tomcat-0.0.1-SNAPSHOT.war
  inflating: target/ROOT/META-INF/MANIFEST.MF
 ...
 [root@k8s-master tomcat-java-demo-master]# ls target/
classes                              maven-archiver
generated-sources                    maven-status
ly-simple-tomcat-0.0.1-SNAPSHOT      ROOT
ly-simple-tomcat-0.0.1-SNAPSHOT.war
[root@k8s-master tomcat-java-demo-master]# ls target/ROOT/
META-INF  WEB-INF
[root@k8s-master tomcat-java-demo-master]# docker build -t java-demo .
Sending build context to Docker daemon  65.45MB
Step 1/4 : FROM lizhenliang/tomcat
latest: Pulling from lizhenliang/tomcat
a02a4930cb5d: Pull complete 
12ac6c660ca4: Pull complete 
675ab56ce773: Pull complete 
Digest: sha256:3bd83236a4b6cf4152d0384981ce9ed55063bc5fd5259d206bfdfdb877651db0
Status: Downloaded newer image for lizhenliang/tomcat:latest
 ---> 143035d83fdc
Step 2/4 : LABEL maintainer www.ctnrs.com
 ---> Running in cced391ddde7
Removing intermediate container cced391ddde7
 ---> f645047acc86
Step 3/4 : RUN rm -rf /usr/local/tomcat/webapps/*
 ---> Running in 2343188f5767
Removing intermediate container 2343188f5767
 ---> 4ca7cd219431
Step 4/4 : ADD target/ROOT /usr/local/tomcat/webapps/ROOT
 ---> 844893cae228
Successfully built 844893cae228
Successfully tagged java-demo:latest
[root@k8s-master tomcat-java-demo-master]# docker run -d -p 8888:8080 java-demo
5a787c6a6eb49d303e07dd2769a52287933daf5c6573c8b15fda3769ab7fd09f

在这里插入图片描述

[root@k8s-master tomcat-java-demo-master]# vi /usr/lib/systemd/system/docker.service
...
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --insecure-registry 10.0.0.65 --insecure-registry harbor.lpk.net
...
[root@k8s-master tomcat-java-demo-master]# systemctl daemon-reload 
[root@k8s-master tomcat-java-demo-master]# systemctl restart docker
[root@k8s-master tomcat-java-demo-master]# docker login 10.0.0.65
Username: admin
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[root@k8s-master tomcat-java-demo-master]# docker push 10.0.0.65/demo/java-demo:v1 
The push refers to repository [10.0.0.65/demo/java-demo]
9e7a9c6b2bd4: Pushed 
e02e78c5e1a2: Pushed 
ceead5ca823f: Pushed 
2353c173a26a: Pushed 
071d8bd76517: Pushed 
v1: digest: sha256:62288c888c740eaeb7786da327d05ea7bda3d14d6669c0e18c822ff69b62f643 size: 1371
[root@k8s-master ~]# vi java-demo.yaml
[root@k8s-master ~]# cat java-demo.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: java-demo
  labels:
    app: java-demo
spec:
  replicas: 3
  selector:
    matchLabels:
      project: www
      app: java-demo
  template:
    metadata:
      labels:
        project: www
        app: java-demo
    spec:
      containers:
      - name: web
        image: 10.0.0.65/demo/java-demo:v1
        ports:
        - containerPort: 8080
[root@k8s-master ~]# kubectl apply -f java-demo.yaml 
deployment.apps/java-demo created
[root@k8s-master ~]# kubectl get pods
NAME                                     READY   STATUS             RESTARTS   AGE
client1                                  1/1     Running            9          4d23h
client2                                  1/1     Running            9          4d23h
configmap-demo-pod                       1/1     Running            4          6d9h
java-demo-7d7d8cd565-dsllq               0/1     ImagePullBackOff   0          7h13m
java-demo-7d7d8cd565-h7c84               0/1     ImagePullBackOff   0          7h13m
java-demo-7d7d8cd565-zr465               0/1     ImagePullBackOff   0          7h13m
my-pod2                                  1/1     Running            15         6d16h
nfs-client-provisioner-58d675cd5-dx7n4   1/1     Running            10         6d11h
pod-taint                                1/1     Running            9          12d
secret-demo-pod                          1/1     Running            4          6d8h
sh                                       1/1     Running            6          6d10h
test-76846b5956-gftn9                    1/1     Running            2          6d10h
test-76846b5956-r7s9k                    1/1     Running            2          6d10h
test-76846b5956-trpbn                    1/1     Running            2          6d10h
test2-78c4694588-87b9r                   1/1     Running            5          6d11h
web-0                                    1/1     Running            4          6d10h
web-1                                    1/1     Running            4          6d10h
web-2                                    1/1     Running            4          6d10h
web-96d5df5c8-vc9kf                      1/1     Running            3          4d23h
[root@k8s-master ~]# kubectl describe pod java-demo-7d7d8cd565-dsllq
Name:         java-demo-7d7d8cd565-dsllq
Namespace:    default
Priority:     0
Node:         k8s-node2/10.0.0.63
Start Time:   Mon, 27 Dec 2021 14:19:33 +0800
Labels:       app=java-demo
              pod-template-hash=7d7d8cd565
              project=www
Annotations:  cni.projectcalico.org/podIP: 10.244.169.186/32
              cni.projectcalico.org/podIPs: 10.244.169.186/32
Status:       Pending
IP:           10.244.169.186
IPs:
  IP:           10.244.169.186
Controlled By:  ReplicaSet/java-demo-7d7d8cd565
Containers:
  web:
    Container ID:   
    Image:          10.0.0.65/demo/java-demo:v1
    Image ID:       
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-8grtj (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  default-token-8grtj:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-8grtj
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason   Age                       From     Message
  ----     ------   ----                      ----     -------
  Normal   BackOff  12m (x1850 over 7h12m)    kubelet  Back-off pulling image "10.0.0.65/demo/java-demo:v1"
  Warning  Failed   2m54s (x1894 over 7h12m)  kubelet  Error: ImagePullBackOff

[root@k8s-node1 ~]# docker run -d 10.0.0.65/demo/java-demo:v1
Unable to find image '10.0.0.65/demo/java-demo:v1' locally
docker: Error response from daemon: unauthorized: unauthorized to access repository: demo/java-demo, action: pull: unauthorized to access repository: demo/java-demo, action: pull.
See 'docker run --help'.
[root@k8s-node1 ~]# docker login 10.0.0.65
Username: admin
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[root@k8s-node1 ~]# docker run -d 10.0.0.65/demo/java-demo:v1
Unable to find image '10.0.0.65/demo/java-demo:v1' locally
v1: Pulling from demo/java-demo
a02a4930cb5d: Already exists 
12ac6c660ca4: Already exists 
675ab56ce773: Already exists 
4c1eea01bd69: Pull complete 
26dff194b2a4: Pull complete 
Digest: sha256:62288c888c740eaeb7786da327d05ea7bda3d14d6669c0e18c822ff69b62f643
Status: Downloaded newer image for 10.0.0.65/demo/java-demo:v1
9c0b4ea12bb78c0b9984aa4669e9951401b84682514a723d6ce9a3d0d0e754cf

[root@k8s-master ~]# vi java-demo.yaml 
[root@k8s-master ~]# cat java-demo.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: java-demo
  labels:
    app: java-demo
spec:
  replicas: 3
  selector:
    matchLabels:
      project: www
      app: java-demo
  template:
    metadata:
      labels:
        project: www
        app: java-demo
    spec:
      imagePullSecrets:
      - name: registry-auth
      containers:
      - name: web
        image: 10.0.0.65/demo/java-demo:v1
        ports:
        - containerPort: 8080
[root@k8s-master ~]# kubectl delete -f java-demo.yaml 
deployment.apps "java-demo" deleted
[root@k8s-master ~]# kubectl apply -f java-demo.yaml 
deployment.apps/java-demo created
[root@k8s-master ~]# kubectl get pods
NAME                                     READY   STATUS              RESTARTS   AGE
java-demo-bcbfffcb9-f7nfk                0/1     ContainerCreating   0          12s
java-demo-bcbfffcb9-mbxtq                0/1     ContainerCreating   0          12s
java-demo-bcbfffcb9-v4bqj                0/1     ContainerCreating   0          12s
[root@k8s-master ~]# kubectl get pods
NAME                                     READY   STATUS         RESTARTS   AGE
java-demo-bcbfffcb9-f7nfk                1/1     Running        0          19s
java-demo-bcbfffcb9-mbxtq                1/1     Running        0          19s
java-demo-bcbfffcb9-v4bqj                0/1     ErrImagePull   0          19s
[root@k8s-master ~]# kubectl get pods
NAME                                     READY   STATUS    RESTARTS   AGE
java-demo-bcbfffcb9-f7nfk                1/1     Running   2          19m
java-demo-bcbfffcb9-k5g89                1/1     Running   1          8m45s
java-demo-bcbfffcb9-mbxtq                1/1     Running   2          19m


[root@k8s-master tomcat-java-demo-master]# cat src/main/resources/application.yml 
server:
  port: 8080
spring:
  datasource:
    url: jdbc:mysql://localhost:3306/test?characterEncoding=utf-8
    username: root
    password: 12345
    driver-class-name: com.mysql.jdbc.Driver
  freemarker:
    allow-request-override: false
    cache: true
    check-template-location: true
    charset: UTF-8
    content-type: text/html; charset=utf-8
    expose-request-attributes: false
    expose-session-attributes: false
    expose-spring-macro-helpers: false
    suffix: .ftl
    template-loader-path:
      - classpath:/templates/

[root@k8s-master tomcat-java-demo-master]# vi configmap.yaml
[root@k8s-master tomcat-java-demo-master]# cat configmap.yaml 
apiVersion: v1
kind: ConfigMap
metadata:
  name: java-demo-config
data:
  application.yml: |
    server:
      port: 8080
    spring:
      datasource:
        url: jdbc:mysql://localhost:3306/test?characterEncoding=utf-8
        username: root
        password: 12345
        driver-class-name: com.mysql.jdbc.Driver
      freemarker:
        allow-request-override: false
        cache: true
        check-template-location: true
        charset: UTF-8
        content-type: text/html; charset=utf-8
        expose-request-attributes: false
        expose-session-attributes: false
        expose-spring-macro-helpers: false
        suffix: .ftl
        template-loader-path:
          - classpath:/templates/
[root@k8s-master tomcat-java-demo-master]# kubectl apply -f configmap.yaml 
configmap/java-demo-config created

[root@java-demo-757f9dcc46-wx9cq tomcat]# cd webapps/ROOT/WEB-INF/classes/
[root@java-demo-757f9dcc46-wx9cq classes]# ls
application.yml
[root@java-demo-757f9dcc46-wx9cq classes]# command terminated with exit code 137
[root@k8s-master ~]# kubectl get pods
NAME                                     READY   STATUS    RESTARTS   AGE
java-demo-757f9dcc46-vrdzz               0/1     Running   0          84s
java-demo-757f9dcc46-w9sgc               0/1     Running   0          84s
java-demo-757f9dcc46-wx9cq               0/1     Running   1          84s
[root@k8s-master ~]# vi java-demo.yaml 
[root@k8s-master ~]# kubectl delete -f java-demo.yaml 
deployment.apps "java-demo" deleted
[root@k8s-master ~]# kubectl apply -f java-demo.yaml 
deployment.apps/java-demo created
[root@k8s-master ~]# kubectl get pods
NAME                                     READY   STATUS              RESTARTS   AGE
configmap-demo-pod                       1/1     Running             5          6d10h
java-demo-6bf4cd9644-4vxmp               0/1     Running             0          5s
java-demo-6bf4cd9644-wqdqf               0/1     Pending             0          4s
java-demo-6bf4cd9644-zdlvw               0/1     ContainerCreating   0          4s
java-demo-757f9dcc46-vrdzz               0/1     Terminating         1          3m21s
java-demo-757f9dcc46-w9sgc               0/1     Terminating         1          3m21s
[root@k8s-master ~]# kubectl get pods
NAME                                     READY   STATUS        RESTARTS   AGE
configmap-demo-pod                       1/1     Running       5          6d10h
java-demo-6bf4cd9644-4vxmp               0/1     Running       0          38s
java-demo-6bf4cd9644-wqdqf               0/1     Pending       0          37s
java-demo-6bf4cd9644-zdlvw               0/1     Running       0          37s
java-demo-757f9dcc46-vrdzz               0/1     Terminating   1          3m54s
java-demo-757f9dcc46-w9sgc               0/1     Terminating   1          3m54s
[root@k8s-master ~]# kubectl exec -it java-demo-6bf4cd9644-4vxmp -- bash
[root@java-demo-6bf4cd9644-4vxmp tomcat]# cd webapps/ROOT/WEB-INF/classes/
[root@java-demo-6bf4cd9644-4vxmp classes]# ls
application.yml  com  log4j.properties  static  templates
[root@java-demo-6bf4cd9644-4vxmp classes]# exit
exit
[root@k8s-master ~]# vi service.yaml
[root@k8s-master ~]# vi java-demo-service.yaml
[root@k8s-master ~]# kubectl apply -f java-demo-service.yaml 
service/java-demo created
[root@k8s-master ~]# kubectl get ep
NAME             ENDPOINTS                                                   AGE
fuseim.pri-ifs   <none>                                                      6d12h
java-demo        10.244.169.150:8080,10.244.169.184:8080,10.244.36.84:8080   42s
kubernetes       10.0.0.61:6443                                              35d
my-dep           <none>                                                      32d
my-service       10.244.36.68:80,10.244.36.71:80,10.244.36.79:80             26d
nginx            10.244.36.68:80,10.244.36.71:80,10.244.36.79:80             6d12h
[root@k8s-master ~]# vi java-demo-ingress.yaml
[root@k8s-master ~]# cat java-demo-ingress.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: java-demo
spec:
  rules:
  - host: example.ctnrs.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: java-demo
            port:
              number: 80
[root@k8s-master ~]# kubectl apply -f java-demo-ingress.yaml 
ingress.networking.k8s.io/java-demo created
[root@k8s-master ~]# kubectl get ingress
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
NAME              CLASS    HOSTS               ADDRESS   PORTS   AGE
example-ingress   <none>   example.ctnrs.com             80      11d
java-demo         <none>   example.ctnrs.com             80      11s
[root@k8s-master ~]# kubectl get pods -n ongress
No resources found in ongress namespace.
[root@k8s-master ~]# kubectl get pods -n ingress
No resources found in ingress namespace.
[root@k8s-master ~]# kubectl get pods -n ingress-nginx
NAME                             READY   STATUS    RESTARTS   AGE
nginx-ingress-controller-7cjvq   1/1     Running   12         11d
nginx-ingress-controller-pk4sq   1/1     Running   11         11d
# 运维 # kubernetes # java
Logo
K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 |  韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号​前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。​长夜有穷，真水无香。领略过了 K8s“身在江

avatar K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub

avatar K8S/Kubernetes

k8s 火了！

2020，上云之年，产品云端化成为一种趋势。在一线城市，很多公司都已经构建了自己的私有云环境，比如阿里云、网易云、华为云等。而Kubernetes 作为基于容器编排领域的王者，具备扩展...

avatar K8S/Kubernetes