摘要:本篇通过一个在线书店微服务应用的完整部署流程,串联 Namespace、ConfigMap、Secret、StatefulSet、Deployment、Service、Ingress、HPA 等资源,演示在 K8s 上部署真实项目的标准做法。

一、项目架构

在线书店应用包含前端、API 网关、业务服务(书籍、订单、用户)及数据层(MySQL、Redis)。Ingress 作为七层入口,将流量路由到各服务;业务服务通过 ClusterIP Service 互相发现与调用。

架构说明:外部请求经 Ingress 按 path 分流;/ 访问前端 SPA,/api/books/api/orders/api/users 分别路由到对应业务 Service;业务服务通过内部 DNS(如 mysql-svcredis-svc)访问数据层;StatefulSet 保证 MySQL 有稳定网络标识与持久化存储。

Ingress 入口

Frontend SPA

API Gateway

Admin

Book Service

Order Service

User Service

MySQL

MySQL

Redis

组件类型说明
FrontendDeploymentReact SPA,静态资源
API GatewayDeploymentNginx 反向代理,路由到后端
Book/Order/User ServiceDeployment业务微服务,多副本
MySQLStatefulSet书籍、订单数据存储
RedisDeployment缓存与会话

二、准备工作

2.1 创建 Namespace

使用独立 Namespace 隔离 bookstore 相关资源,便于权限与配额管理。

# 00-namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: bookstore
  labels:
    app: bookstore
    env: production
kubectl apply -f 00-namespace.yaml

关键字段metadata.name 指定 Namespace 名称;labels 可用于多租户隔离与资源选择。

2.2 创建 Secret(数据库密码)

敏感信息通过 Secret 管理,避免明文写入 YAML。生产环境建议使用外部密钥管理(如 Vault)。

kubectl create secret generic db-credentials \
  -n bookstore \
  --from-literal=MYSQL_ROOT_PASSWORD=rootpass123 \
  --from-literal=MYSQL_PASSWORD=apppass456 \
  --from-literal=REDIS_PASSWORD=redispass789

2.3 创建 ConfigMap(应用配置)

非敏感配置(数据库名、服务地址、端口等)放入 ConfigMap,支持通过 envFrom 批量注入。

# 01-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: bookstore
data:
  MYSQL_DATABASE: "bookstore"
  MYSQL_USER: "bookstore_app"
  BOOK_SERVICE_URL: "http://book-svc:8080"
  ORDER_SERVICE_URL: "http://order-svc:8080"
  USER_SERVICE_URL: "http://user-svc:8080"
  REDIS_HOST: "redis-svc"
  REDIS_PORT: "6379"

三、部署数据库层

3.1 MySQL(使用 StatefulSet)

有状态数据库使用 StatefulSet 保证 Pod 名称稳定、有序部署与 PVC 绑定。Headless Service 提供稳定的 DNS 记录,便于主从或分片场景扩展。

mysql-svc Headless

mysql-0

mysql-data PVC

# 02-mysql.yaml
apiVersion: v1
kind: Service
metadata:
  name: mysql-svc
  namespace: bookstore
spec:
  clusterIP: None                    # Headless Service
  selector:
    app: mysql
  ports:
  - port: 3306
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
  namespace: bookstore
spec:
  serviceName: mysql-svc
  replicas: 1
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: mysql:8.0
        ports:
        - containerPort: 3306
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: MYSQL_ROOT_PASSWORD
        - name: MYSQL_DATABASE
          valueFrom:
            configMapKeyRef:
              name: app-config
              key: MYSQL_DATABASE
        - name: MYSQL_USER
          valueFrom:
            configMapKeyRef:
              name: app-config
              key: MYSQL_USER
        - name: MYSQL_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: MYSQL_PASSWORD
        resources:
          requests:
            cpu: "250m"
            memory: "512Mi"
          limits:
            cpu: "500m"
            memory: "1Gi"
        volumeMounts:
        - name: mysql-data
          mountPath: /var/lib/mysql
        livenessProbe:
          exec:
            command:
            - mysqladmin
            - ping
            - -h
            - localhost
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          exec:
            command:
            - mysql
            - -h
            - localhost
            - -u
            - root
            - -p$(MYSQL_ROOT_PASSWORD)
            - -e
            - "SELECT 1"
          initialDelaySeconds: 10
          periodSeconds: 5
  volumeClaimTemplates:
  - metadata:
      name: mysql-data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 10Gi

3.2 Redis

Redis 作为无状态缓存,使用 Deployment 部署,通过 Secret 注入密码。

# 03-redis.yaml
apiVersion: v1
kind: Service
metadata:
  name: redis-svc
  namespace: bookstore
spec:
  selector:
    app: redis
  ports:
  - port: 6379
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: bookstore
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:7-alpine
        ports:
        - containerPort: 6379
        command: ["redis-server", "--requirepass", "$(REDIS_PASSWORD)"]
        env:
        - name: REDIS_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: REDIS_PASSWORD
        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "250m"
            memory: "256Mi"
        livenessProbe:
          tcpSocket:
            port: 6379
          initialDelaySeconds: 10
          periodSeconds: 5

四、部署业务服务

4.1 Book Service

业务服务通过 Deployment 多副本部署,配合 livenessProbe 与 readinessProbe 保障可用性。Pod 反亲和性(preferred)尽量将副本分散到不同节点,降低单点故障影响。

# 04-book-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: book-svc
  namespace: bookstore
spec:
  selector:
    app: book-service
  ports:
  - port: 8080
    targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: book-service
  namespace: bookstore
spec:
  replicas: 2
  selector:
    matchLabels:
      app: book-service
  template:
    metadata:
      labels:
        app: book-service
        version: v1
    spec:
      containers:
      - name: book-service
        image: bookstore/book-service:v1
        ports:
        - containerPort: 8080
        envFrom:
        - configMapRef:
            name: app-config
        env:
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: MYSQL_PASSWORD
        - name: DB_HOST
          value: "mysql-svc"
        resources:
          requests:
            cpu: "100m"
            memory: "256Mi"
          limits:
            cpu: "500m"
            memory: "512Mi"
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 5
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchLabels:
                  app: book-service
              topologyKey: kubernetes.io/hostname

关键字段envFrom 批量注入 ConfigMap;podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution 软反亲和,尽量将副本分散到不同节点;topologyKey: kubernetes.io/hostname 按节点分散。

4.2 Order Service 和 User Service

结构类似 Book Service,Order 依赖 MySQL,User 依赖 Redis,通过 ConfigMap 与 Secret 注入配置。

# 05-order-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: order-svc
  namespace: bookstore
spec:
  selector:
    app: order-service
  ports:
  - port: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service
  namespace: bookstore
spec:
  replicas: 2
  selector:
    matchLabels:
      app: order-service
  template:
    metadata:
      labels:
        app: order-service
    spec:
      containers:
      - name: order-service
        image: bookstore/order-service:v1
        ports:
        - containerPort: 8080
        envFrom:
        - configMapRef:
            name: app-config
        env:
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: MYSQL_PASSWORD
        resources:
          requests:
            cpu: "100m"
            memory: "256Mi"
          limits:
            cpu: "500m"
            memory: "512Mi"
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 5
---
# 06-user-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: user-svc
  namespace: bookstore
spec:
  selector:
    app: user-service
  ports:
  - port: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service
  namespace: bookstore
spec:
  replicas: 2
  selector:
    matchLabels:
      app: user-service
  template:
    metadata:
      labels:
        app: user-service
    spec:
      containers:
      - name: user-service
        image: bookstore/user-service:v1
        ports:
        - containerPort: 8080
        envFrom:
        - configMapRef:
            name: app-config
        env:
        - name: REDIS_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: REDIS_PASSWORD
        resources:
          requests:
            cpu: "100m"
            memory: "256Mi"
          limits:
            cpu: "500m"
            memory: "512Mi"
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10

五、部署前端和 Ingress

5.1 Frontend

前端为静态资源,通过 Deployment 多副本部署,经 Ingress 对外暴露。

# 07-frontend.yaml
apiVersion: v1
kind: Service
metadata:
  name: frontend-svc
  namespace: bookstore
spec:
  selector:
    app: frontend
  ports:
  - port: 80
    targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  namespace: bookstore
spec:
  replicas: 2
  selector:
    matchLabels:
      app: frontend
  template:
    metadata:
      labels:
        app: frontend
    spec:
      containers:
      - name: frontend
        image: bookstore/frontend:v1
        ports:
        - containerPort: 80
        resources:
          requests:
            cpu: "50m"
            memory: "64Mi"
          limits:
            cpu: "200m"
            memory: "128Mi"
        readinessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 5
          periodSeconds: 5

5.2 Ingress

Ingress 按 path 将流量路由到不同 Service。生产环境需配置 TLS 证书,此处示例使用 bookstore-tls Secret。路径匹配顺序由上而下,更长、更具体的 path 应放在前面。

/api/books

/api/orders

/api/users

/

Ingress

book-svc

order-svc

user-svc

frontend-svc

# 08-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: bookstore-ingress
  namespace: bookstore
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - bookstore.example.com
    secretName: bookstore-tls
  rules:
  - host: bookstore.example.com
    http:
      paths:
      - path: /api/books
        pathType: Prefix
        backend:
          service:
            name: book-svc
            port:
              number: 8080
      - path: /api/orders
        pathType: Prefix
        backend:
          service:
            name: order-svc
            port:
              number: 8080
      - path: /api/users
        pathType: Prefix
        backend:
          service:
            name: user-svc
            port:
              number: 8080
      - path: /
        pathType: Prefix
        backend:
          service:
            name: frontend-svc
            port:
              number: 80

关键字段ingressClassName: nginx 指定 Ingress Controller;pathType: Prefix 表示前缀匹配;tls.secretName 引用 TLS 证书 Secret;annotations 可配置 body 大小、超时等。

六、配置 HPA 自动扩缩容

HPA 根据 CPU、内存等指标自动伸缩 Deployment 副本数。以下示例在 CPU 平均使用率超过 70% 或内存超过 80% 时扩容,最多 10 副本。

# 09-hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: book-service-hpa
  namespace: bookstore
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: book-service
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

关键字段scaleTargetRef 指定要扩缩的 Deployment;minReplicas/maxReplicas 限制副本范围;HPA 依赖 Metrics Server 获取 CPU/内存使用率,需预先配置 resources.requests

七、部署和验证

7.1 部署顺序

按依赖顺序部署:先 Namespace、ConfigMap、Secret,再数据库,等数据库 Ready 后部署业务服务,最后部署前端与 Ingress、HPA。

Namespace + ConfigMap + Secret

MySQL + Redis

业务服务

Frontend + Ingress + HPA

# 1. 基础资源
kubectl apply -f 00-namespace.yaml
kubectl apply -f 01-configmap.yaml
kubectl create secret generic db-credentials -n bookstore \
  --from-literal=MYSQL_ROOT_PASSWORD=rootpass123 \
  --from-literal=MYSQL_PASSWORD=apppass456 \
  --from-literal=REDIS_PASSWORD=redispass789

# 2. 数据库
kubectl apply -f 02-mysql.yaml
kubectl apply -f 03-redis.yaml

# 等待数据库就绪
kubectl wait --for=condition=ready pod -l app=mysql -n bookstore --timeout=120s
kubectl wait --for=condition=ready pod -l app=redis -n bookstore --timeout=60s

# 3. 业务服务
kubectl apply -f 04-book-service.yaml
kubectl apply -f 05-order-service.yaml
kubectl apply -f 06-user-service.yaml
kubectl apply -f 07-frontend.yaml
kubectl apply -f 08-ingress.yaml
kubectl apply -f 09-hpa.yaml

7.2 验证与测试

# 查看 Pod 状态
kubectl get pods -n bookstore -o wide

# 查看所有资源
kubectl get all -n bookstore

# 查看 Ingress
kubectl get ingress -n bookstore

# 集群内测试连通性
kubectl run test --rm -it --image=curlimages/curl -n bookstore -- \
  curl http://book-svc:8080/health

7.3 分步验证各服务

# 1. 数据库就绪
kubectl exec -it mysql-0 -n bookstore -- mysqladmin -uroot -p$MYSQL_ROOT_PASSWORD ping

# 2. Redis 连通(容器内环境变量已注入)
kubectl exec -it deploy/redis -n bookstore -- sh -c 'redis-cli -a "$REDIS_PASSWORD" ping'

# 3. 业务服务健康检查
kubectl exec -it deploy/book-service -n bookstore -c book-service -- wget -qO- http://localhost:8080/health

# 4. 端到端:从 Ingress 访问(需配置 hosts 或使用端口转发)
kubectl port-forward -n bookstore svc/frontend-svc 8080:80
# 访问 http://localhost:8080 验证前端

7.4 监控与日志接入要点

  • Prometheus:为业务 Service 添加 prometheus.io/scrape: "true"prometheus.io/port: "8080"prometheus.io/path: "/metrics" 注解,配合 ServiceMonitor 自动发现。
  • 日志:应用输出到 stdout/stderr,由集群内 Fluent Bit 或 Promtail DaemonSet 采集;建议使用 JSON 结构化日志,便于在 Loki/ELK 中检索。
  • Grafana:导入 Kubernetes 与业务相关 Dashboard,配置 namespace 变量为 bookstore

八、部署资源全景

下图展示 bookstore 命名空间内各资源的依赖关系:Ingress 按 path 路由到不同 Service,Service 背后是 Deployment 或 StatefulSet,数据库使用 PVC 持久化。

Workloads

Services

Ingress

bookstore-ingress

book-svc

order-svc

user-svc

frontend-svc

mysql-svc

redis-svc

book-service x2

order-service x2

user-service x2

frontend x2

mysql StatefulSet

redis Deployment

九、常见问题(FAQ)

9.1 数据库连接失败

  • 确认 ConfigMap、Secret 已创建且 key 正确
  • 使用 kubectl exec 进入业务 Pod 测试 mysql -h mysql-svc -u ... 连通性
  • 检查 MySQL readinessProbe 是否通过
  • 确认业务服务启动顺序:先 kubectl wait 等数据库 Ready 再部署业务服务

9.2 Ingress 404 或 502

  • 确认 Ingress Controller 已安装(如 ingress-nginx)
  • 检查 pathType 与 path 匹配规则,长 path 应放在前面
  • 确认 backend Service 和 port 正确,且 Pod 有 Ready 端点
  • 检查 kubectl get endpoints 确认 Service 有 Endpoints

9.3 HPA 不生效

  • 部署 Metrics Server:kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
  • 确认 Deployment 已设置 resources.requests,HPA 依赖其计算使用率
  • 执行 kubectl top pods -n bookstore 验证 Metrics Server 是否正常

9.4 Pod 启动后频繁重启

  • 查看 kubectl logs <pod> --previous 排查崩溃原因
  • 检查 ConfigMap/Secret 引用是否存在、key 是否正确
  • 确认 livenessProbe 的 initialDelaySeconds 足够,避免启动慢被误杀

十、总结

部署顺序:Namespace + ConfigMap + Secret → 数据库 → 业务服务 → 前端 + Ingress + HPA。生产检查清单:设置 requests/limits、配置 liveness/readiness、使用 Pod 反亲和性、HPA 自动扩缩容、敏感信息用 Secret、数据用 PVC 持久化。

部署检查清单

requests/limits

健康检查

Pod 反亲和性

HPA

Secret 敏感信息

PVC 持久化

更多推荐