《Docker容器化部署:多阶段构建优化》

📋 本文概览

学习目标

  • 掌握Dockerfile多阶段构建技术,将镜像体积减少70%以上
  • 理解Docker Compose编排多服务应用的最佳实践
  • 学会使用环境变量和配置文件管理不同部署环境
  • 实现完善的健康检查和容器监控机制
  • 构建可直接用于生产环境的容器化部署方案

技术栈

  • Docker 24.0+
  • Docker Compose v2.20+
  • Python 3.11 (后端)
  • Node.js 20 (前端构建)
  • PostgreSQL 15
  • Redis 7
  • Nginx (反向代理)

预计阅读时间: 45分钟

前置知识要求

  • 基本的Linux命令行操作
  • Docker基础概念(镜像、容器、卷)
  • 了解QuantumFlow项目架构(参考前24篇文章)

🎯 业务场景:为什么需要容器化?

传统部署的痛点

在没有容器化之前,QuantumFlow的部署面临诸多挑战:

环境不一致问题

# 开发环境 (开发者本地机器)
Python 3.11.2 + PostgreSQL 14 + Redis 6
✅ 运行正常

# 测试环境 (测试服务器)
Python 3.10.8 + PostgreSQL 15 + Redis 7
⚠️ 部分功能异常

# 生产环境 (云服务器)
Python 3.9.16 + PostgreSQL 13 + Redis 6
❌ 启动失败

依赖管理混乱

# 手动安装依赖,容易出错
$ sudo apt-get install python3-dev postgresql libpq-dev redis-server
$ pip install -r requirements.txt
# 版本冲突、缺少系统库、权限问题...

# 每次更新都是噩梦
$ git pull
$ pip install -r requirements.txt  # 可能破坏现有环境
$ sudo systemctl restart quantumflow  # 祈祷能正常启动

扩展困难

# 需要手动在多台服务器上重复部署
Server1: 手动配置 2小时
Server2: 手动配置 2小时
Server3: 手动配置失败,调试1小时...

容器化带来的价值

一致性保证

# 开发、测试、生产环境使用完全相同的镜像
FROM python:3.11-slim
# 精确锁定所有依赖版本
COPY requirements.lock .
RUN pip install -r requirements.lock
# ✅ "在我机器上能跑" = "在所有机器上都能跑"

快速部署与回滚

# 5秒内启动整个应用栈
$ docker-compose up -d

# 1秒内回滚到上一版本
$ docker-compose down
$ docker-compose -f docker-compose.v1.2.3.yml up -d

资源隔离与安全

# 每个服务独立运行,互不干扰
services:
  backend:
    cpus: '2.0'      # CPU限制
    memory: 4G       # 内存限制
    read_only: true  # 只读文件系统

水平扩展

# 轻松扩展到10个后端实例
$ docker-compose up -d --scale backend=10

🏗️ 整体架构设计

容器化架构图

graph TB
    subgraph "Docker Host"
        subgraph "Frontend Layer"
            Nginx[Nginx<br/>反向代理]
            Static[静态文件卷]
        end
        
        subgraph "Application Layer"
            Backend1[Backend-1<br/>FastAPI]
            Backend2[Backend-2<br/>FastAPI]
            Backend3[Backend-3<br/>FastAPI]
            Worker1[Celery Worker-1]
            Worker2[Celery Worker-2]
        end
        
        subgraph "Data Layer"
            PostgreSQL[(PostgreSQL<br/>主数据库)]
            Redis[(Redis<br/>缓存+队列)]
            PGData[pg_data卷]
            RedisData[redis_data卷]
        end
        
        subgraph "Monitoring"
            Prometheus[Prometheus]
            Grafana[Grafana]
        end
    end
    
    Internet((互联网)) --> Nginx
    Nginx --> Backend1
    Nginx --> Backend2
    Nginx --> Backend3
    Backend1 --> PostgreSQL
    Backend2 --> PostgreSQL
    Backend3 --> PostgreSQL
    Backend1 --> Redis
    Worker1 --> Redis
    Worker2 --> Redis
    PostgreSQL --> PGData
    Redis --> RedisData
    Nginx --> Static
    
    Prometheus -.监控.-> Backend1
    Prometheus -.监控.-> Backend2
    Prometheus -.监控.-> PostgreSQL
    Grafana -.展示.-> Prometheus

服务依赖关系

services:
  # 基础设施层(最先启动)
  postgres:
    depends_on: []
  redis:
    depends_on: []
  
  # 应用层(依赖基础设施)
  backend:
    depends_on:
      postgres:
        condition: service_healthy  # 等待数据库就绪
      redis:
        condition: service_healthy
  
  celery-worker:
    depends_on:
      - backend  # 复用backend镜像
      - redis
  
  # 前端层(依赖应用层)
  nginx:
    depends_on:
      - backend

💻 核心实现

1. 后端多阶段Dockerfile优化

传统单阶段构建的问题

# ❌ 传统方式:镜像体积 1.2GB
FROM python:3.11
WORKDIR /app

# 安装所有构建依赖(编译器、开发库等)
RUN apt-get update && apt-get install -y \
    gcc g++ make \
    libpq-dev \
    git \
    curl \
    vim \
    && rm -rf /var/lib/apt/lists/*

# 安装Python依赖
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# 复制源代码
COPY . .

# 问题:
# 1. 包含大量构建工具(gcc、make),运行时不需要
# 2. 包含开发工具(vim、git),存在安全风险
# 3. 包含测试文件、文档等无关文件
# 4. 基础镜像过大(python:3.11 = 1GB)

✅ 优化后的多阶段构建

# ==================== 阶段1: 构建依赖 ====================
FROM python:3.11-slim AS builder

# 设置构建参数
ARG DEBIAN_FRONTEND=noninteractive
ARG PIP_NO_CACHE_DIR=1
ARG PIP_DISABLE_PIP_VERSION_CHECK=1

# 安装构建依赖(仅此阶段需要)
RUN apt-get update && apt-get install -y --no-install-recommends \
    gcc \
    g++ \
    libpq-dev \
    && rm -rf /var/lib/apt/lists/*

# 创建虚拟环境(便于后续复制)
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

# 安装Python依赖
COPY requirements.txt requirements-lock.txt ./
RUN pip install --upgrade pip setuptools wheel && \
    pip install -r requirements-lock.txt

# 编译Python字节码(加速启动)
COPY src/ /app/src/
RUN python -m compileall -b /app/src && \
    find /app/src -name "*.py" -delete


# ==================== 阶段2: 运行环境 ====================
FROM python:3.11-slim AS runtime

# 创建非root用户(安全最佳实践)
RUN groupadd -r quantumflow && \
    useradd -r -g quantumflow -u 1000 quantumflow

# 仅安装运行时依赖
RUN apt-get update && apt-get install -y --no-install-recommends \
    libpq5 \
    curl \
    && rm -rf /var/lib/apt/lists/*

# 从builder阶段复制虚拟环境
COPY --from=builder /opt/venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

# 设置工作目录
WORKDIR /app

# 复制应用代码(使用.dockerignore过滤)
COPY --chown=quantumflow:quantumflow --from=builder /app/src /app/src
COPY --chown=quantumflow:quantumflow alembic/ /app/alembic/
COPY --chown=quantumflow:quantumflow alembic.ini /app/
COPY --chown=quantumflow:quantumflow config/ /app/config/

# 创建必要目录
RUN mkdir -p /app/logs /app/tmp && \
    chown -R quantumflow:quantumflow /app

# 切换到非root用户
USER quantumflow

# 健康检查
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
    CMD curl -f http://localhost:8000/health || exit 1

# 暴露端口
EXPOSE 8000

# 启动命令
CMD ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]

优化效果对比

指标传统方式优化后改进
镜像体积1.2 GB350 MB↓ 70%
构建时间8分钟3分钟↓ 62%
启动时间15秒5秒↓ 67%
安全漏洞87个12个↓ 86%
层数23层12层↓ 48%

关键优化技术解析

# 技术1: 使用slim基础镜像
FROM python:3.11-slim  # 150MB vs python:3.11 (1GB)

# 技术2: 合并RUN指令减少层数
RUN apt-get update && \
    apt-get install -y pkg1 pkg2 && \
    rm -rf /var/lib/apt/lists/*  # ✅ 1层

# vs 分开写(❌ 3层)
RUN apt-get update
RUN apt-get install -y pkg1 pkg2
RUN rm -rf /var/lib/apt/lists/*

# 技术3: 利用构建缓存
COPY requirements.txt .     # 依赖文件先复制(不常变)
RUN pip install -r requirements.txt
COPY src/ .                 # 源代码后复制(经常变)

# 技术4: 清理缓存
RUN pip install --no-cache-dir -r requirements.txt  # 不保存pip缓存
RUN apt-get clean && rm -rf /var/lib/apt/lists/*     # 清理apt缓存

# 技术5: 字节码预编译
RUN python -m compileall -b /app && \  # 生成.pyc文件
    find /app -name "*.py" -delete      # 删除.py源文件(可选)

2. 前端多阶段构建

# ==================== 阶段1: 依赖安装 ====================
FROM node:20-alpine AS deps

WORKDIR /app

# 复制依赖文件
COPY package.json package-lock.json ./

# 安装依赖(使用npm ci保证一致性)
RUN npm ci --only=production && \
    npm cache clean --force


# ==================== 阶段2: 构建 ====================
FROM node:20-alpine AS builder

WORKDIR /app

# 复制依赖
COPY --from=deps /app/node_modules ./node_modules
COPY package.json package-lock.json ./

# 安装开发依赖
RUN npm ci

# 复制源代码
COPY public/ ./public/
COPY src/ ./src/
COPY index.html tsconfig.json vite.config.ts ./

# 构建生产版本
ARG VITE_API_URL
ENV VITE_API_URL=$VITE_API_URL
RUN npm run build


# ==================== 阶段3: 生产运行 ====================
FROM nginx:1.25-alpine AS runtime

# 复制构建产物
COPY --from=builder /app/dist /usr/share/nginx/html

# 复制Nginx配置
COPY nginx.conf /etc/nginx/conf.d/default.conf

# 健康检查
HEALTHCHECK --interval=30s --timeout=3s \
    CMD wget --quiet --tries=1 --spider http://localhost:80/health || exit 1

EXPOSE 80

CMD ["nginx", "-g", "daemon off;"]

前端优化效果

指标传统方式优化后改进
镜像体积450 MB25 MB↓ 94%
启动时间8秒1秒↓ 87%

3. .dockerignore配置(关键优化)

.dockerignore(后端)

# Python缓存
__pycache__/
*.py[cod]
*$py.class
*.so
.Python

# 虚拟环境
venv/
env/
ENV/

# 测试和覆盖率
.pytest_cache/
.coverage
htmlcov/
*.cover

# IDE配置
.vscode/
.idea/
*.swp
*.swo

# 文档
docs/
*.md
LICENSE

# Git
.git/
.gitignore

# 日志
logs/
*.log

# 临时文件
tmp/
*.tmp

# 数据库
*.db
*.sqlite

# 环境变量(敏感信息)
.env
.env.*

# 大文件
*.zip
*.tar.gz

.dockerignore(前端)

node_modules/
dist/
.git/
.vscode/
*.md
.env.local
coverage/
.cache/

效果: 减少构建上下文 85%(从 500MB → 75MB)


🎼 Docker Compose编排完整方案

docker-compose.yml(开发环境)

version: '3.9'

# ==================== 网络定义 ====================
networks:
  quantumflow-net:
    driver: bridge
    ipam:
      config:
        - subnet: 172.28.0.0/16


# ==================== 卷定义 ====================
volumes:
  postgres_data:
    driver: local
  redis_data:
    driver: local
  static_files:
    driver: local


# ==================== 服务定义 ====================
services:
  
  # ========== 数据库服务 ==========
  postgres:
    image: postgres:15-alpine
    container_name: quantumflow-postgres
    restart: unless-stopped
    
    environment:
      POSTGRES_DB: ${POSTGRES_DB:-quantumflow}
      POSTGRES_USER: ${POSTGRES_USER:-quantumflow}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-changeme}
      POSTGRES_INITDB_ARGS: "--encoding=UTF8 --locale=C"
      PGDATA: /var/lib/postgresql/data/pgdata
    
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./scripts/init-db.sh:/docker-entrypoint-initdb.d/init-db.sh:ro
    
    ports:
      - "${POSTGRES_PORT:-5432}:5432"
    
    networks:
      quantumflow-net:
        ipv4_address: 172.28.0.10
    
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-quantumflow}"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 10s
    
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 4G
        reservations:
          cpus: '1.0'
          memory: 2G
    
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"


  # ========== Redis服务 ==========
  redis:
    image: redis:7-alpine
    container_name: quantumflow-redis
    restart: unless-stopped
    
    command: >
      redis-server
      --appendonly yes
      --appendfsync everysec
      --maxmemory 2gb
      --maxmemory-policy allkeys-lru
      --requirepass ${REDIS_PASSWORD:-changeme}
    
    volumes:
      - redis_data:/data
    
    ports:
      - "${REDIS_PORT:-6379}:6379"
    
    networks:
      quantumflow-net:
        ipv4_address: 172.28.0.11
    
    healthcheck:
      test: ["CMD", "redis-cli", "--raw", "incr", "ping"]
      interval: 10s
      timeout: 3s
      retries: 5
    
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 2G


  # ========== 后端API服务 ==========
  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
      args:
        - ENVIRONMENT=development
      target: runtime
      cache_from:
        - quantumflow/backend:latest
    
    image: quantumflow/backend:${VERSION:-latest}
    container_name: quantumflow-backend
    restart: unless-stopped
    
    environment:
      # 应用配置
      ENVIRONMENT: ${ENVIRONMENT:-development}
      DEBUG: ${DEBUG:-True}
      SECRET_KEY: ${SECRET_KEY}
      
      # 数据库配置
      DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}
      DB_POOL_SIZE: 20
      DB_MAX_OVERFLOW: 10
      
      # Redis配置
      REDIS_URL: redis://:${REDIS_PASSWORD}@redis:6379/0
      CELERY_BROKER_URL: redis://:${REDIS_PASSWORD}@redis:6379/1
      CELERY_RESULT_BACKEND: redis://:${REDIS_PASSWORD}@redis:6379/2
      
      # JWT配置
      JWT_SECRET_KEY: ${JWT_SECRET_KEY}
      JWT_ALGORITHM: HS256
      JWT_EXPIRE_MINUTES: 60
      
      # CORS配置
      CORS_ORIGINS: ${CORS_ORIGINS:-http://localhost:3000,http://localhost:80}
      
      # 日志配置
      LOG_LEVEL: ${LOG_LEVEL:-INFO}
      
    volumes:
      - ./backend/src:/app/src:ro  # 开发时代码热重载
      - ./backend/logs:/app/logs
      - static_files:/app/static
    
    ports:
      - "${BACKEND_PORT:-8000}:8000"
    
    networks:
      - quantumflow-net
    
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    
    deploy:
      replicas: ${BACKEND_REPLICAS:-3}
      resources:
        limits:
          cpus: '2.0'
          memory: 4G
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3


  # ========== Celery Worker服务 ==========
  celery-worker:
    image: quantumflow/backend:${VERSION:-latest}
    container_name: quantumflow-celery-worker
    restart: unless-stopped
    
    command: celery -A src.celery_app worker --loglevel=info --concurrency=4
    
    environment:
      ENVIRONMENT: ${ENVIRONMENT:-development}
      DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}
      REDIS_URL: redis://:${REDIS_PASSWORD}@redis:6379/0
      CELERY_BROKER_URL: redis://:${REDIS_PASSWORD}@redis:6379/1
      CELERY_RESULT_BACKEND: redis://:${REDIS_PASSWORD}@redis:6379/2
    
    volumes:
      - ./backend/src:/app/src:ro
      - ./backend/logs:/app/logs
    
    networks:
      - quantumflow-net
    
    depends_on:
      - backend
      - redis
    
    deploy:
      replicas: ${WORKER_REPLICAS:-2}
      resources:
        limits:
          cpus: '2.0'
          memory: 4G


  # ========== Celery Beat调度器 ==========
  celery-beat:
    image: quantumflow/backend:${VERSION:-latest}
    container_name: quantumflow-celery-beat
    restart: unless-stopped
    
    command: celery -A src.celery_app beat --loglevel=info
    
    environment:
      ENVIRONMENT: ${ENVIRONMENT:-development}
      DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}
      CELERY_BROKER_URL: redis://:${REDIS_PASSWORD}@redis:6379/1
    
    volumes:
      - ./backend/src:/app/src:ro
    
    networks:
      - quantumflow-net
    
    depends_on:
      - redis


  # ========== Nginx反向代理 ==========
  nginx:
    build:
      context: ./frontend
      dockerfile: Dockerfile
      args:
        - VITE_API_URL=${VITE_API_URL:-http://localhost:8000}
    
    image: quantumflow/frontend:${VERSION:-latest}
    container_name: quantumflow-nginx
    restart: unless-stopped
    
    ports:
      - "${NGINX_PORT:-80}:80"
      - "${NGINX_SSL_PORT:-443}:443"
    
    volumes:
      - ./nginx/conf.d:/etc/nginx/conf.d:ro
      - ./nginx/ssl:/etc/nginx/ssl:ro
      - static_files:/usr/share/nginx/html/static:ro
      - ./nginx/logs:/var/log/nginx
    
    networks:
      - quantumflow-net
    
    depends_on:
      - backend
    
    healthcheck:
      test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost/health"]
      interval: 30s
      timeout: 3s
      retries: 3


  # ========== Flower监控 (开发环境) ==========
  flower:
    image: mher/flower:2.0
    container_name: quantumflow-flower
    restart: unless-stopped
    
    command: celery --broker=redis://:${REDIS_PASSWORD}@redis:6379/1 flower --port=5555
    
    environment:
      CELERY_BROKER_URL: redis://:${REDIS_PASSWORD}@redis:6379/1
      CELERY_RESULT_BACKEND: redis://:${REDIS_PASSWORD}@redis:6379/2
      FLOWER_BASIC_AUTH: ${FLOWER_USER:-admin}:${FLOWER_PASSWORD:-changeme}
    
    ports:
      - "${FLOWER_PORT:-5555}:5555"
    
    networks:
      - quantumflow-net
    
    depends_on:
      - redis
      - celery-worker
    
    profiles:
      - dev  # 仅在开发环境启动


  # ========== Adminer数据库管理 (开发环境) ==========
  adminer:
    image: adminer:4.8.1
    container_name: quantumflow-adminer
    restart: unless-stopped
    
    environment:
      ADMINER_DEFAULT_SERVER: postgres
      ADMINER_DESIGN: nette
    
    ports:
      - "${ADMINER_PORT:-8080}:8080"
    
    networks:
      - quantumflow-net
    
    depends_on:
      - postgres
    
    profiles:
      - dev

docker-compose.prod.yml(生产环境覆盖)

version: '3.9'

services:
  
  backend:
    build:
      target: runtime  # 使用多阶段构建的最终阶段
    
    environment:
      ENVIRONMENT: production
      DEBUG: "False"
      LOG_LEVEL: WARNING
    
    volumes:
      - ./backend/logs:/app/logs  # 移除代码卷(不需要热重载)
      - static_files:/app/static
    
    deploy:
      replicas: 5  # 生产环境更多副本
      resources:
        limits:
          cpus: '4.0'
          memory: 8G
        reservations:
          cpus: '2.0'
          memory: 4G
      update_config:
        parallelism: 2
        delay: 10s
        failure_action: rollback
      rollback_config:
        parallelism: 1
        delay: 5s
  
  
  celery-worker:
    environment:
      ENVIRONMENT: production
      LOG_LEVEL: WARNING
    
    deploy:
      replicas: 4
      resources:
        limits:
          cpus: '4.0'
          memory: 8G
  
  
  postgres:
    volumes:
      - /data/postgres:/var/lib/postgresql/data  # 使用宿主机路径
    
    deploy:
      resources:
        limits:
          cpus: '4.0'
          memory: 16G
        reservations:
          cpus: '2.0'
          memory: 8G
  
  
  nginx:
    ports:
      - "80:80"
      - "443:443"
    
    volumes:
      - /etc/letsencrypt:/etc/nginx/ssl:ro  # 使用真实SSL证书

启动命令

# 开发环境
docker-compose up -d

# 开发环境+监控工具
docker-compose --profile dev up -d

# 生产环境
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d

# 生产环境扩展
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d --scale backend=10

🔧 环境变量管理

.env.example(模板文件)

# ==================== 应用配置 ====================
ENVIRONMENT=development
DEBUG=True
VERSION=1.0.0

# ==================== 数据库配置 ====================
POSTGRES_DB=quantumflow
POSTGRES_USER=quantumflow
POSTGRES_PASSWORD=changeme_in_production
POSTGRES_PORT=5432

# ==================== Redis配置 ====================
REDIS_PASSWORD=changeme_in_production
REDIS_PORT=6379

# ==================== 后端配置 ====================
BACKEND_PORT=8000
BACKEND_REPLICAS=3
SECRET_KEY=changeme_in_production_use_openssl_rand_hex_32
JWT_SECRET_KEY=changeme_in_production_use_openssl_rand_hex_32

# ==================== Celery配置 ====================
WORKER_REPLICAS=2

# ==================== 前端配置 ====================
NGINX_PORT=80
NGINX_SSL_PORT=443
VITE_API_URL=http://localhost:8000

# ==================== CORS配置 ====================
CORS_ORIGINS=http://localhost:3000,http://localhost:80

# ==================== 监控工具 (仅开发环境) ====================
FLOWER_PORT=5555
FLOWER_USER=admin
FLOWER_PASSWORD=changeme
ADMINER_PORT=8080

# ==================== 日志配置 ====================
LOG_LEVEL=INFO

环境变量加载优先级

# src/config.py
import os
from pydantic_settings import BaseSettings
from functools import lru_cache

class Settings(BaseSettings):
    """
    配置加载优先级(从高到低):
    1. 环境变量
    2. .env文件
    3. 默认值
    """
    
    # 应用配置
    environment: str = "development"
    debug: bool = False
    secret_key: str
    
    # 数据库配置
    database_url: str
    db_pool_size: int = 20
    db_max_overflow: int = 10
    db_echo: bool = False
    
    # Redis配置
    redis_url: str
    celery_broker_url: str
    celery_result_backend: str
    
    # JWT配置
    jwt_secret_key: str
    jwt_algorithm: str = "HS256"
    jwt_expire_minutes: int = 60
    
    # CORS配置
    cors_origins: list[str] = []
    
    # 日志配置
    log_level: str = "INFO"
    
    class Config:
        env_file = ".env"
        env_file_encoding = "utf-8"
        case_sensitive = False
        
        # 支持列表类型的环境变量
        @classmethod
        def parse_env_var(cls, field_name: str, raw_val: str):
            if field_name == "cors_origins":
                return [origin.strip() for origin in raw_val.split(",")]
            return raw_val


@lru_cache()
def get_settings() -> Settings:
    """单例模式获取配置"""
    return Settings()


# 使用示例
settings = get_settings()
print(f"Environment: {settings.environment}")
print(f"Database: {settings.database_url}")

敏感信息管理(Docker Secrets)

# docker-compose.secrets.yml
version: '3.9'

secrets:
  postgres_password:
    file: ./secrets/postgres_password.txt
  jwt_secret:
    file: ./secrets/jwt_secret.txt

services:
  backend:
    secrets:
      - postgres_password
      - jwt_secret
    
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
      JWT_SECRET_KEY_FILE: /run/secrets/jwt_secret
# src/utils/secrets.py
import os

def read_secret(secret_name: str, default: str = None) -> str:
    """从Docker Secret或环境变量读取敏感信息"""
    secret_file = os.getenv(f"{secret_name.upper()}_FILE")
    
    if secret_file and os.path.exists(secret_file):
        with open(secret_file, "r") as f:
            return f.read().strip()
    
    return os.getenv(secret_name.upper(), default)


# 使用
JWT_SECRET = read_secret("jwt_secret_key", "default-dev-key")

🏥 健康检查完整方案

后端健康检查端点

# src/api/health.py
from fastapi import APIRouter, Depends, status
from sqlalchemy.ext.asyncio import AsyncSession
from redis import Redis
from typing import Dict, Any
import time
import psutil

from src.database import get_db
from src.cache import get_redis

router = APIRouter(prefix="/health", tags=["Health"])


@router.get("", status_code=status.HTTP_200_OK)
async def health_check() -> Dict[str, str]:
    """
    简单健康检查(用于Docker healthcheck)
    
    返回:
        {"status": "healthy"}
    """
    return {"status": "healthy"}


@router.get("/detailed", status_code=status.HTTP_200_OK)
async def detailed_health_check(
    db: AsyncSession = Depends(get_db),
    redis: Redis = Depends(get_redis)
) -> Dict[str, Any]:
    """
    详细健康检查(监控系统调用)
    
    检查项目:
    - 数据库连接
    - Redis连接
    - 磁盘空间
    - 内存使用
    - CPU使用率
    """
    start_time = time.time()
    health_status = {
        "status": "healthy",
        "timestamp": time.time(),
        "checks": {}
    }
    
    # 1. 检查数据库
    try:
        await db.execute("SELECT 1")
        health_status["checks"]["database"] = {
            "status": "healthy",
            "response_time_ms": round((time.time() - start_time) * 1000, 2)
        }
    except Exception as e:
        health_status["status"] = "unhealthy"
        health_status["checks"]["database"] = {
            "status": "unhealthy",
            "error": str(e)
        }
    
    # 2. 检查Redis
    try:
        redis.ping()
        health_status["checks"]["redis"] = {
            "status": "healthy",
            "response_time_ms": round((time.time() - start_time) * 1000, 2)
        }
    except Exception as e:
        health_status["status"] = "unhealthy"
        health_status["checks"]["redis"] = {
            "status": "unhealthy",
            "error": str(e)
        }
    
    # 3. 检查磁盘空间
    disk = psutil.disk_usage('/')
    disk_usage_percent = disk.percent
    health_status["checks"]["disk"] = {
        "status": "healthy" if disk_usage_percent < 90 else "warning",
        "usage_percent": disk_usage_percent,
        "free_gb": round(disk.free / (1024**3), 2)
    }
    
    # 4. 检查内存
    memory = psutil.virtual_memory()
    health_status["checks"]["memory"] = {
        "status": "healthy" if memory.percent < 90 else "warning",
        "usage_percent": memory.percent,
        "available_gb": round(memory.available / (1024**3), 2)
    }
    
    # 5. 检查CPU
    cpu_percent = psutil.cpu_percent(interval=1)
    health_status["checks"]["cpu"] = {
        "status": "healthy" if cpu_percent < 80 else "warning",
        "usage_percent": cpu_percent
    }
    
    health_status["response_time_ms"] = round((time.time() - start_time) * 1000, 2)
    
    return health_status


@router.get("/ready", status_code=status.HTTP_200_OK)
async def readiness_check(
    db: AsyncSession = Depends(get_db),
    redis: Redis = Depends(get_redis)
) -> Dict[str, str]:
    """
    就绪检查(Kubernetes使用)
    
    仅当所有依赖服务可用时返回200
    """
    try:
        # 检查数据库
        await db.execute("SELECT 1")
        
        # 检查Redis
        redis.ping()
        
        return {"status": "ready"}
    
    except Exception as e:
        raise HTTPException(
            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
            detail=f"Service not ready: {str(e)}"
        )


@router.get("/live", status_code=status.HTTP_200_OK)
async def liveness_check() -> Dict[str, str]:
    """
    存活检查(Kubernetes使用)
    
    仅检查进程是否响应,不检查依赖
    """
    return {"status": "alive"}

Dockerfile健康检查

# 方式1: 使用curl
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
    CMD curl -f http://localhost:8000/health || exit 1

# 方式2: 使用wget(Alpine镜像)
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
    CMD wget --quiet --tries=1 --spider http://localhost:8000/health || exit 1

# 方式3: 使用Python脚本(无需额外工具)
COPY healthcheck.py /app/
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
    CMD python /app/healthcheck.py || exit 1
# healthcheck.py
import sys
import urllib.request

try:
    response = urllib.request.urlopen("http://localhost:8000/health", timeout=10)
    if response.status == 200:
        sys.exit(0)
    else:
        sys.exit(1)
except Exception:
    sys.exit(1)

Docker Compose健康检查

services:
  postgres:
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U quantumflow"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 10s
  
  redis:
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 5
  
  backend:
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    
    depends_on:
      postgres:
        condition: service_healthy  # 等待数据库健康
      redis:
        condition: service_healthy

监控健康状态脚本

#!/bin/bash
# scripts/monitor-health.sh

while true; do
    echo "=== $(date) ==="
    
    # 检查所有容器健康状态
    docker ps --format "table {{.Names}}\t{{.Status}}" | grep -E "(healthy|unhealthy)"
    
    # 检查详细健康信息
    curl -s http://localhost:8000/health/detailed | jq '.'
    
    echo ""
    sleep 60
done

📊 性能优化与最佳实践

1. 镜像构建缓存策略

# ❌ 错误示例:频繁变化的文件先复制
COPY . /app
RUN pip install -r requirements.txt  # 代码变化会导致重新安装依赖

# ✅ 正确示例:依赖文件先复制
COPY requirements.txt .
RUN pip install -r requirements.txt  # 缓存层
COPY . /app  # 仅此层需要重建

2. BuildKit加速

# 启用BuildKit(更快的构建)
export DOCKER_BUILDKIT=1

# 构建时显示进度
docker build --progress=plain -t myapp .

# 使用缓存
docker build --cache-from myapp:latest -t myapp:new .

3. 多平台构建

# 构建支持AMD64和ARM64的镜像
docker buildx create --use
docker buildx build --platform linux/amd64,linux/arm64 -t quantumflow/backend:latest --push .

4. 资源限制最佳实践

services:
  backend:
    deploy:
      resources:
        limits:
          cpus: '2.0'        # 最大2核
          memory: 4G         # 最大4GB内存
          pids: 100          # 最大进程数
        reservations:
          cpus: '1.0'        # 保证1核
          memory: 2G         # 保证2GB

5. 网络优化

networks:
  quantumflow-net:
    driver: bridge
    driver_opts:
      com.docker.network.driver.mtu: 1450  # 调整MTU避免分片
    ipam:
      config:
        - subnet: 172.28.0.0/16
          gateway: 172.28.0.1

6. 日志管理

services:
  backend:
    logging:
      driver: "json-file"
      options:
        max-size: "10m"      # 单个日志文件最大10MB
        max-file: "3"        # 最多保留3个文件
        compress: "true"     # 压缩旧日志
# 查看日志
docker-compose logs -f backend

# 清理日志
docker-compose logs --no-log-prefix backend > /dev/null

🧪 测试验证

1. 本地测试流程

# 步骤1: 清理环境
docker-compose down -v
docker system prune -f

# 步骤2: 构建镜像
docker-compose build --no-cache

# 步骤3: 启动服务
docker-compose up -d

# 步骤4: 检查健康状态
docker-compose ps
# 等待所有服务显示 "healthy"

# 步骤5: 运行集成测试
docker-compose exec backend pytest tests/integration/

# 步骤6: 检查日志
docker-compose logs backend | tail -100

# 步骤7: 性能测试
docker-compose exec backend locust -f tests/performance/locustfile.py

2. 自动化测试脚本

#!/bin/bash
# scripts/test-containers.sh

set -e

echo "🧪 开始容器化测试..."

# 清理环境
echo "📦 清理旧容器..."
docker-compose down -v

# 构建镜像
echo "🔨 构建镜像..."
docker-compose build

# 启动服务
echo "🚀 启动服务..."
docker-compose up -d

# 等待服务就绪
echo "⏳ 等待服务就绪..."
for i in {1..30}; do
    if curl -f http://localhost:8000/health > /dev/null 2>&1; then
        echo "✅ 后端服务已就绪"
        break
    fi
    echo "等待中... ($i/30)"
    sleep 2
done

# 运行测试
echo "🧪 运行集成测试..."
docker-compose exec -T backend pytest tests/integration/ -v

# 运行性能测试
echo "⚡ 运行性能测试..."
docker-compose exec -T backend locust -f tests/performance/locustfile.py --headless -u 100 -r 10 -t 1m

# 检查健康状态
echo "🏥 检查健康状态..."
health_status=$(curl -s http://localhost:8000/health/detailed | jq -r '.status')
if [ "$health_status" != "healthy" ]; then
    echo "❌ 健康检查失败"
    docker-compose logs backend
    exit 1
fi

echo "✅ 所有测试通过!"

3. 镜像安全扫描

# 使用Trivy扫描镜像漏洞
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
    aquasec/trivy image quantumflow/backend:latest

# 使用Docker Scout
docker scout cves quantumflow/backend:latest

# 使用Snyk
snyk container test quantumflow/backend:latest

📦 附件资源

完整的Dockerfile(后端)

# 文件: backend/Dockerfile
# 构建命令: docker build -t quantumflow/backend:latest .

# ==================== 阶段1: 基础环境 ====================
FROM python:3.11-slim AS base

# 设置环境变量
ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    PIP_NO_CACHE_DIR=1 \
    PIP_DISABLE_PIP_VERSION_CHECK=1 \
    DEBIAN_FRONTEND=noninteractive

# 更新系统包
RUN apt-get update && apt-get upgrade -y


# ==================== 阶段2: 依赖构建 ====================
FROM base AS builder

# 安装构建依赖
RUN apt-get install -y --no-install-recommends \
    gcc \
    g++ \
    make \
    libpq-dev \
    && rm -rf /var/lib/apt/lists/*

# 创建虚拟环境
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

# 升级pip
RUN pip install --upgrade pip setuptools wheel

# 安装Python依赖
WORKDIR /build
COPY requirements.txt requirements-lock.txt ./
RUN pip install -r requirements-lock.txt

# 编译Python字节码
COPY src/ /build/src/
RUN python -m compileall -b /build/src && \
    find /build/src -name "*.py" -delete


# ==================== 阶段3: 运行环境 ====================
FROM base AS runtime

# 创建应用用户
RUN groupadd -r quantumflow && \
    useradd -r -g quantumflow -u 1000 -d /app -s /sbin/nologin quantumflow

# 安装运行时依赖
RUN apt-get install -y --no-install-recommends \
    libpq5 \
    curl \
    && rm -rf /var/lib/apt/lists/*

# 从builder复制虚拟环境
COPY --from=builder /opt/venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

# 设置工作目录
WORKDIR /app

# 复制应用代码
COPY --chown=quantumflow:quantumflow --from=builder /build/src /app/src
COPY --chown=quantumflow:quantumflow alembic/ /app/alembic/
COPY --chown=quantumflow:quantumflow alembic.ini /app/
COPY --chown=quantumflow:quantumflow config/ /app/config/

# 创建必要目录
RUN mkdir -p /app/logs /app/tmp /app/static && \
    chown -R quantumflow:quantumflow /app

# 切换到应用用户
USER quantumflow

# 健康检查
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
    CMD curl -f http://localhost:8000/health || exit 1

# 暴露端口
EXPOSE 8000

# 启动命令
CMD ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]

完整的docker-compose.yml

(见前文"Docker Compose编排完整方案"部分)

.dockerignore

# 文件: backend/.dockerignore

# Python缓存
__pycache__/
*.py[cod]
*$py.class
*.so
.Python

# 虚拟环境
venv/
env/
ENV/
.venv/

# 测试
.pytest_cache/
.coverage
htmlcov/
*.cover
.tox/
.nox/

# IDE
.vscode/
.idea/
*.swp
*.swo
.DS_Store

# 文档
docs/
*.md
README*
LICENSE

# Git
.git/
.gitignore
.gitattributes

# CI/CD
.github/
.gitlab-ci.yml
Jenkinsfile

# 日志
logs/
*.log

# 临时文件
tmp/
temp/
*.tmp
*.bak

# 数据库
*.db
*.sqlite
*.sqlite3

# 环境变量
.env
.env.*
!.env.example

# 大文件
*.zip
*.tar.gz
*.rar

# OS文件
Thumbs.db

Nginx配置

# 文件: nginx/conf.d/default.conf

# 上游后端服务器
upstream backend {
    least_conn;  # 最少连接负载均衡
    
    server backend:8000 max_fails=3 fail_timeout=30s;
    
    keepalive 32;  # 保持连接池
}

# HTTP服务器(重定向到HTTPS)
server {
    listen 80;
    server_name quantumflow.example.com;
    
    # ACME挑战(Let's Encrypt)
    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }
    
    # 重定向到HTTPS
    location / {
        return 301 https://$server_name$request_uri;
    }
}

# HTTPS服务器
server {
    listen 443 ssl http2;
    server_name quantumflow.example.com;
    
    # SSL配置
    ssl_certificate /etc/nginx/ssl/fullchain.pem;
    ssl_certificate_key /etc/nginx/ssl/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 10m;
    
    # 安全头
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;
    
    # Gzip压缩
    gzip on;
    gzip_vary on;
    gzip_min_length 1024;
    gzip_types text/plain text/css text/xml text/javascript application/json application/javascript application/xml+rss;
    
    # 前端静态文件
    location / {
        root /usr/share/nginx/html;
        try_files $uri $uri/ /index.html;
        
        expires 1h;
        add_header Cache-Control "public, immutable";
    }
    
    # 静态资源(长期缓存)
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
        root /usr/share/nginx/html;
        expires 1y;
        add_header Cache-Control "public, immutable";
    }
    
    # API代理
    location /api/ {
        proxy_pass http://backend;
        proxy_http_version 1.1;
        
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Connection "";
        
        # 超时配置
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
        
        # 缓冲配置
        proxy_buffering on;
        proxy_buffer_size 4k;
        proxy_buffers 8 4k;
    }
    
    # WebSocket代理
    location /ws/ {
        proxy_pass http://backend;
        proxy_http_version 1.1;
        
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        
        proxy_read_timeout 86400;  # 24小时
    }
    
    # 健康检查
    location /health {
        access_log off;
        return 200 "healthy\n";
        add_header Content-Type text/plain;
    }
}

部署脚本

#!/bin/bash
# 文件: scripts/deploy.sh

set -e

# 颜色输出
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'

echo -e "${GREEN}🚀 开始部署 QuantumFlow${NC}"

# 读取版本号
VERSION=${1:-latest}
ENVIRONMENT=${2:-production}

echo -e "${YELLOW}版本: $VERSION${NC}"
echo -e "${YELLOW}环境: $ENVIRONMENT${NC}"

# 1. 拉取最新代码
echo -e "${GREEN}📥 拉取最新代码...${NC}"
git pull origin main

# 2. 备份当前版本
echo -e "${GREEN}💾 备份当前版本...${NC}"
docker-compose ps > backup_$(date +%Y%m%d_%H%M%S).txt

# 3. 构建新镜像
echo -e "${GREEN}🔨 构建镜像...${NC}"
docker-compose build --pull

# 4. 运行数据库迁移
echo -e "${GREEN}🗄️ 运行数据库迁移...${NC}"
docker-compose run --rm backend alembic upgrade head

# 5. 滚动更新
echo -e "${GREEN}♻️ 滚动更新服务...${NC}"
docker-compose up -d --no-deps --scale backend=5 backend
sleep 10
docker-compose up -d --no-deps --scale backend=3 --remove-orphans backend

# 6. 检查健康状态
echo -e "${GREEN}🏥 检查健康状态...${NC}"
for i in {1..30}; do
    health=$(curl -s http://localhost/health/detailed | jq -r '.status' 2>/dev/null || echo "unhealthy")
    if [ "$health" == "healthy" ]; then
        echo -e "${GREEN}✅ 服务健康${NC}"
        break
    fi
    echo "等待服务就绪... ($i/30)"
    sleep 2
done

if [ "$health" != "healthy" ]; then
    echo -e "${RED}❌ 部署失败,开始回滚...${NC}"
    docker-compose down
    # 这里应该恢复到之前的版本
    exit 1
fi

# 7. 清理旧镜像
echo -e "${GREEN}🧹 清理旧镜像...${NC}"
docker image prune -f

echo -e "${GREEN}✅ 部署完成!${NC}"

💡 小结

本文深入讲解了QuantumFlow的Docker容器化部署方案,核心要点包括:

1. 多阶段构建优化

  • 镜像体积从1.2GB降至350MB(↓70%)
  • 构建时间从8分钟降至3分钟(↓62%)
  • 安全漏洞从87个降至12个(↓86%)

2. Docker Compose编排

  • 7个服务的完整编排方案
  • 开发/生产环境配置分离
  • 健康检查与依赖管理

3. 生产级特性

  • 非root用户运行
  • 资源限制与监控
  • 滚动更新与回滚
  • 日志管理与清理

4. 最佳实践

  • .dockerignore优化构建上下文
  • 环境变量分层管理
  • 网络隔离与安全加固
  • 自动化测试脚本

下一篇预告:《Kubernetes集群部署:生产级配置》

  • K8s核心概念(Pod/Deployment/Service)
  • Helm Chart编写
  • 水平扩展(HPA)
  • 滚动更新与金丝雀发布
  • Ingress配置与TLS证书

思考题

  1. 如何在不停机的情况下更新数据库Schema?(提示:蓝绿部署)
  2. 如何实现跨主机的容器通信?(提示:Overlay网络)
  3. 如何监控容器资源使用情况?(提示:cAdvisor + Prometheus)

📚 参考资料

官方文档

安全指南

开源项目参考

git clone https://github.com/quantumflow/deployment.git
cd deployment/docker

包含文件

  • ✅ backend/Dockerfile
  • ✅ frontend/Dockerfile
  • ✅ docker-compose.yml
  • ✅ docker-compose.prod.yml
  • ✅ .dockerignore(前后端)
  • ✅ nginx/conf.d/default.conf
  • ✅ scripts/deploy.sh
  • ✅ scripts/test-containers.sh
  • ✅ .env.example

📝 本文为《QuantumFlow工作流自动化从入门到精通》专栏第25篇,全文约2.7万字,配套代码已开源。

更多推荐