人工智能 / 机器学习 Python 学习笔记

适用方向：人工智能 / 机器学习
核心库：scikit-learn, PyTorch, TensorFlow
学习路径：基础理论 → 传统机器学习 → 深度学习框架 → 项目实战

目录

  1. 机器学习基础概念
  2. scikit-learn 传统机器学习
  3. PyTorch 深度学习框架
  4. TensorFlow / Keras 深度学习框架
  5. 三大框架对比与选型
  6. 经典项目实战
  7. 学习资源推荐

1. 机器学习基础概念

1.1 机器学习分类

类型说明典型算法
监督学习有标签数据，学习输入到输出的映射线性回归、逻辑回归、SVM、决策树、神经网络
无监督学习无标签数据，发现数据内在结构K-Means、PCA、DBSCAN、自编码器
强化学习通过与环境交互学习最优策略Q-Learning、DQN、PPO
半监督学习少量标注 + 大量未标注数据标签传播、伪标签

1.2 模型评估指标

分类任务
# 混淆矩阵基础概念
# TP (True Positive)   真正例：预测为正，实际为正
# TN (True Negative)   真负例：预测为负，实际为负
# FP (False Positive)  假正例：预测为正，实际为负
# FN (False Negative)  假负例：预测为负，实际为正

# 精确率 Precision = TP / (TP + FP)
# 召回率 Recall = TP / (TP + FN)
# F1-score = 2 * Precision * Recall / (Precision + Recall)
# 准确率 Accuracy = (TP + TN) / (TP + TN + FP + FN)
# AUC-ROC：受试者工作特征曲线下面积
回归任务
# MAE (平均绝对误差) = (1/n) * Σ|y_true - y_pred|
# MSE (均方误差) = (1/n) * Σ(y_true - y_pred)²
# RMSE (均方根误差) = √MSE
# R² (决定系数) = 1 - SS_res / SS_tot

1.3 过拟合与欠拟合

现象表现解决方法
欠拟合训练集和测试集效果都差增加模型复杂度、增加特征、减少正则化
过拟合训练集效果好，测试集效果差增加数据、正则化(L1/L2)、Dropout、早停

2. scikit-learn 传统机器学习

2.1 核心模块概览

sklearn
├── datasets          # 内置数据集
├── preprocessing     # 数据预处理
├── model_selection   # 模型选择与评估
├── linear_model      # 线性模型
├── svm               # 支持向量机
├── tree              # 决策树
├── ensemble          # 集成学习
├── neighbors         # K近邻
├── cluster           # 聚类
├── decomposition     # 降维
├── metrics           # 评估指标
└── pipeline          # 流水线

2.2 数据预处理

2.2.1 数据标准化与归一化
from sklearn.preprocessing import StandardScaler, MinMaxScaler, LabelEncoder, OneHotEncoder
import numpy as np

# StandardScaler：标准化 (均值0，方差1)
# 适用：对特征尺度敏感的算法（SVM、逻辑回归、神经网络）
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)  # 注意：测试集只用transform

# MinMaxScaler：归一化 [0, 1]
# 适用：图像数据、需要 bounded output 的场景
minmax = MinMaxScaler(feature_range=(0, 1))
X_scaled = minmax.fit_transform(X)

# 标签编码（分类特征）
le = LabelEncoder()
y_encoded = le.fit_transform(y_categorical)

# 独热编码
onehot = OneHotEncoder(sparse=False)
X_onehot = onehot.fit_transform(X_categorical.reshape(-1, 1))
2.2.2 缺失值处理
from sklearn.impute import SimpleImputer

# 均值填充（数值型）
imputer = SimpleImputer(strategy='mean')  # 'mean', 'median', 'most_frequent', 'constant'
X_imputed = imputer.fit_transform(X)
2.2.3 特征选择
from sklearn.feature_selection import SelectKBest, f_classif, RFE
from sklearn.linear_model import LogisticRegression

# 方法1：单变量特征选择
selector = SelectKBest(score_func=f_classif, k=10)
X_selected = selector.fit_transform(X, y)

# 方法2：递归特征消除 (RFE)
estimator = LogisticRegression()
selector = RFE(estimator, n_features_to_select=5, step=1)
X_selected = selector.fit_transform(X, y)

2.3 经典算法实现

2.3.1 线性回归
from sklearn.linear_model import LinearRegression, Ridge, Lasso
from sklearn.metrics import mean_squared_error, r2_score

# 普通线性回归
lr = LinearRegression()
lr.fit(X_train, y_train)
y_pred = lr.predict(X_test)

# 评估
print(f"系数: {lr.coef_}")
print(f"截距: {lr.intercept_}")
print(f"MSE: {mean_squared_error(y_test, y_pred)}")
print(f"R²: {r2_score(y_test, y_pred)}")

# Ridge回归 (L2正则化)
ridge = Ridge(alpha=1.0)  # alpha越大，正则化越强
ridge.fit(X_train, y_train)

# Lasso回归 (L1正则化，可做特征选择)
lasso = Lasso(alpha=0.1)
lasso.fit(X_train, y_train)
2.3.2 逻辑回归（分类）
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

# 多分类默认使用 'multinomial' 或 'ovr'
log_reg = LogisticRegression(
    penalty='l2',        # 正则化类型: 'l1', 'l2', 'elasticnet', None
    C=1.0,               # 正则化强度的倒数，越小正则化越强
    solver='lbfgs',      # 优化算法
    max_iter=1000,
    multi_class='auto'
)
log_reg.fit(X_train, y_train)

y_pred = log_reg.predict(X_test)
y_prob = log_reg.predict_proba(X_test)  # 概率输出

print(f"准确率: {accuracy_score(y_test, y_pred)}")
print(classification_report(y_test, y_pred))
print(confusion_matrix(y_test, y_pred))
2.3.3 支持向量机 (SVM)
from sklearn.svm import SVC, SVR

# 分类 SVM
svm = SVC(
    kernel='rbf',        # 'linear', 'poly', 'rbf', 'sigmoid'
    C=1.0,               # 惩罚参数
    gamma='scale',       # RBF核的gamma参数
    probability=True     # 启用概率估计
)
svm.fit(X_train, y_train)

# 回归 SVM
svr = SVR(kernel='rbf', C=1.0, epsilon=0.1)
svr.fit(X_train, y_train)
2.3.4 决策树与随机森林
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier

# 决策树
dt = DecisionTreeClassifier(
    max_depth=5,              # 最大深度
    min_samples_split=2,      # 分裂最小样本数
    min_samples_leaf=1,       # 叶子节点最小样本数
    criterion='gini'          # 'gini' 或 'entropy'
)
dt.fit(X_train, y_train)

# 随机森林（Bagging集成）
rf = RandomForestClassifier(
    n_estimators=100,         # 树的数量
    max_depth=10,
    min_samples_split=2,
    max_features='sqrt',      # 每棵树考虑的特征数
    random_state=42,
    n_jobs=-1                 # 使用所有CPU核心
)
rf.fit(X_train, y_train)

# 特征重要性
importances = rf.feature_importances_

# GBDT（Boosting集成）
gbdt = GradientBoostingClassifier(
    n_estimators=100,
    learning_rate=0.1,
    max_depth=3,
    random_state=42
)
gbdt.fit(X_train, y_train)
2.3.5 K近邻 (KNN)
from sklearn.neighbors import KNeighborsClassifier

knn = KNeighborsClassifier(
    n_neighbors=5,       # K值
    weights='uniform',   # 'uniform' 或 'distance'
    metric='minkowski'   # 距离度量
)
knn.fit(X_train, y_train)

2.4 无监督学习

2.4.1 K-Means 聚类
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

kmeans = KMeans(
    n_clusters=3,        # 聚类数量
    init='k-means++',    # 初始化方法
    n_init=10,           # 运行次数
    random_state=42
)
kmeans.fit(X)

labels = kmeans.labels_          # 每个样本的聚类标签
centroids = kmeans.cluster_centers_  # 聚类中心
inertia = kmeans.inertia_        # 簇内平方和

# 轮廓系数评估（越接近1越好）
score = silhouette_score(X, labels)
print(f"轮廓系数: {score}")

# 选择最佳K值：肘部法则
inertias = []
for k in range(1, 11):
    kmeans = KMeans(n_clusters=k, random_state=42)
    kmeans.fit(X)
    inertias.append(kmeans.inertia_)
2.4.2 PCA 降维
from sklearn.decomposition import PCA

pca = PCA(n_components=2)  # 保留的维度数，也可以是比例如 0.95
X_pca = pca.fit_transform(X)

print(f"解释方差比: {pca.explained_variance_ratio_}")
print(f"累计解释方差: {sum(pca.explained_variance_ratio_)}")

2.5 模型选择与调优

2.5.1 数据集划分
from sklearn.model_selection import train_test_split, cross_val_score

# 简单划分
X_train, X_test, y_train, y_test = train_test_split(
    X, y,
    test_size=0.2,      # 测试集比例
    random_state=42,    # 随机种子，保证可复现
    stratify=y          # 分层抽样（分类任务推荐）
)

# 交叉验证
scores = cross_val_score(
    model, X, y,
    cv=5,               # 5折交叉验证
    scoring='accuracy'  # 评估指标
)
print(f"交叉验证得分: {scores.mean():.4f} ± {scores.std():.4f}")
2.5.2 网格搜索与随机搜索
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV

# 网格搜索
param_grid = {
    'n_estimators': [50, 100, 200],
    'max_depth': [None, 10, 20],
    'min_samples_split': [2, 5, 10]
}

grid_search = GridSearchCV(
    estimator=RandomForestClassifier(random_state=42),
    param_grid=param_grid,
    cv=5,
    scoring='accuracy',
    n_jobs=-1,
    verbose=1
)
grid_search.fit(X_train, y_train)

print(f"最佳参数: {grid_search.best_params_}")
print(f"最佳得分: {grid_search.best_score_}")
best_model = grid_search.best_estimator_

# 随机搜索（更快，适合大参数空间）
random_search = RandomizedSearchCV(
    estimator=model,
    param_distributions=param_dist,
    n_iter=50,          # 随机采样次数
    cv=5,
    random_state=42
)

2.6 Pipeline 流水线

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.svm import SVC

# 构建流水线：数据依次经过每一步
pipeline = Pipeline([
    ('scaler', StandardScaler()),     # 第一步：标准化
    ('pca', PCA(n_components=0.95)),  # 第二步：PCA降维
    ('svm', SVC())                    # 第三步：SVM分类
])

pipeline.fit(X_train, y_train)
y_pred = pipeline.predict(X_test)

# 流水线 + 网格搜索
param_grid = {
    'pca__n_components': [0.9, 0.95, 0.99],  # 注意双下划线
    'svm__C': [0.1, 1, 10],
    'svm__gamma': ['scale', 'auto']
}

grid = GridSearchCV(pipeline, param_grid, cv=5)
grid.fit(X_train, y_train)

3. PyTorch 深度学习框架

3.1 PyTorch 基础

3.1.1 张量 (Tensor) 操作
import torch
import numpy as np

# 创建张量
x = torch.tensor([1, 2, 3])               # 从列表创建
x = torch.randn(3, 4)                     # 标准正态分布
x = torch.rand(3, 4)                      # [0,1)均匀分布
x = torch.zeros(2, 3)                     # 全零
x = torch.ones(2, 3)                      # 全一
x = torch.eye(3)                          # 单位矩阵
x = torch.arange(0, 10, 2)                # 序列

# 张量属性
print(x.shape)      # 形状
print(x.dtype)      # 数据类型
print(x.device)     # 设备 (cpu/cuda)

# 张量运算
a = torch.randn(3, 4)
b = torch.randn(4, 5)

# 矩阵乘法
c = torch.matmul(a, b)  # 或 a @ b

# 逐元素运算
c = a + b
c = a * b
c = torch.exp(a)
c = torch.sigmoid(a)
c = torch.relu(a)

# 形状操作
x = torch.randn(4, 4)
x_reshaped = x.reshape(2, 8)   # 重塑形状
x_flat = x.flatten()           # 展平
x_T = x.t()                    # 转置
x_unsqueezed = x.unsqueeze(0)  # 增加维度
x_squeezed = x.squeeze()       # 压缩维度

# GPU 运算
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
x = x.to(device)  # 移动到GPU

# NumPy 互转
np_array = x.numpy()
tensor = torch.from_numpy(np_array)
3.1.2 自动求导 (Autograd)
import torch

# 启用梯度追踪
x = torch.tensor(2.0, requires_grad=True)
w = torch.tensor(3.0, requires_grad=True)
b = torch.tensor(1.0, requires_grad=True)

# 前向传播
y = w * x + b  # y = 3*2 + 1 = 7

# 反向传播：计算梯度
y.backward()

# 查看梯度
print(x.grad)  # dy/dx = w = 3.0
print(w.grad)  # dy/dw = x = 2.0
print(b.grad)  # dy/db = 1.0

# 清零梯度（防止累积）
x.grad.zero_()
w.grad.zero_()
b.grad.zero_()

# 关闭梯度计算（推理时使用，节省内存）
with torch.no_grad():
    y = w * x + b
    # 这里不会计算梯度

3.2 神经网络构建

3.2.1 自定义模型
import torch
import torch.nn as nn
import torch.nn.functional as F

# 方式1：继承 nn.Module
class SimpleNN(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super(SimpleNN, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.relu = nn.ReLU()
        self.fc2 = nn.Linear(hidden_size, num_classes)
    
    def forward(self, x):
        out = self.fc1(x)
        out = self.relu(out)
        out = self.fc2(out)
        return out

# 方式2：使用 nn.Sequential（简单模型）
model = nn.Sequential(
    nn.Linear(784, 256),
    nn.ReLU(),
    nn.Linear(256, 128),
    nn.ReLU(),
    nn.Linear(128, 10)
)

# 实例化模型
model = SimpleNN(input_size=784, hidden_size=128, num_classes=10)
print(model)

# 查看参数数量
total_params = sum(p.numel() for p in model.parameters())
trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
print(f"总参数: {total_params}, 可训练参数: {trainable_params}")
3.2.2 常用层
import torch.nn as nn

# 全连接层
nn.Linear(in_features, out_features, bias=True)

# 卷积层
nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0)
nn.ConvTranspose2d()  # 转置卷积（上采样）

# 池化层
nn.MaxPool2d(kernel_size, stride=None)
nn.AvgPool2d(kernel_size)
nn.AdaptiveAvgPool2d(output_size)  # 自适应池化

# 归一化层
nn.BatchNorm1d(num_features)
nn.BatchNorm2d(num_features)
nn.LayerNorm(normalized_shape)

# Dropout
nn.Dropout(p=0.5)
nn.Dropout2d(p=0.5)

# 激活函数
nn.ReLU()
nn.LeakyReLU(negative_slope=0.01)
nn.Sigmoid()
nn.Tanh()
nn.Softmax(dim=None)
nn.GELU()

# 损失函数
nn.CrossEntropyLoss()      # 多分类（包含Softmax）
nn.BCELoss()               # 二分类（需先Sigmoid）
nn.BCEWithLogitsLoss()     # 二分类（包含Sigmoid）
nn.MSELoss()               # 回归
nn.NLLLoss()               # 负对数似然

3.3 数据加载

from torch.utils.data import Dataset, DataLoader
from torchvision import datasets, transforms

# 内置数据集 + 变换
transform = transforms.Compose([
    transforms.ToTensor(),                     # 转为张量 [0,1]
    transforms.Normalize((0.1307,), (0.3081,)) # 标准化
])

train_dataset = datasets.MNIST(
    root='./data',
    train=True,
    transform=transform,
    download=True
)

# 自定义数据集
class CustomDataset(Dataset):
    def __init__(self, data, labels, transform=None):
        self.data = data
        self.labels = labels
        self.transform = transform
    
    def __len__(self):
        return len(self.data)
    
    def __getitem__(self, idx):
        sample = self.data[idx]
        label = self.labels[idx]
        
        if self.transform:
            sample = self.transform(sample)
        
        return sample, label

# 数据加载器
train_loader = DataLoader(
    dataset=train_dataset,
    batch_size=64,      # 批次大小
    shuffle=True,       # 打乱顺序
    num_workers=4,      # 多进程加载
    pin_memory=True     # 锁页内存（GPU训练时更快）
)

# 遍历数据
for images, labels in train_loader:
    print(f"批次形状: {images.shape}, 标签形状: {labels.shape}")
    break

3.4 训练流程

3.4.1 完整训练模板
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader

def train_model(model, train_loader, val_loader, num_epochs=10, learning_rate=0.001):
    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    model = model.to(device)
    
    # 损失函数和优化器
    criterion = nn.CrossEntropyLoss()
    optimizer = optim.Adam(model.parameters(), lr=learning_rate)
    
    # 学习率调度器
    scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=5, gamma=0.1)
    
    best_acc = 0.0
    
    for epoch in range(num_epochs):
        # 训练阶段
        model.train()
        train_loss = 0.0
        train_correct = 0
        train_total = 0
        
        for batch_idx, (images, labels) in enumerate(train_loader):
            images = images.to(device)
            labels = labels.to(device)
            
            # 前向传播
            outputs = model(images)
            loss = criterion(outputs, labels)
            
            # 反向传播和优化
            optimizer.zero_grad()  # 清零梯度
            loss.backward()        # 反向传播
            optimizer.step()       # 更新参数
            
            # 统计
            train_loss += loss.item()
            _, predicted = torch.max(outputs.data, 1)
            train_total += labels.size(0)
            train_correct += (predicted == labels).sum().item()
        
        # 验证阶段
        model.eval()  # 切换到评估模式（关闭Dropout、BatchNorm训练模式）
        val_loss = 0.0
        val_correct = 0
        val_total = 0
        
        with torch.no_grad():  # 不计算梯度
            for images, labels in val_loader:
                images = images.to(device)
                labels = labels.to(device)
                
                outputs = model(images)
                loss = criterion(outputs, labels)
                
                val_loss += loss.item()
                _, predicted = torch.max(outputs.data, 1)
                val_total += labels.size(0)
                val_correct += (predicted == labels).sum().item()
        
        # 更新学习率
        scheduler.step()
        
        # 打印结果
        train_acc = 100 * train_correct / train_total
        val_acc = 100 * val_correct / val_total
        
        print(f'Epoch [{epoch+1}/{num_epochs}]')
        print(f'  Train Loss: {train_loss/len(train_loader):.4f}, Acc: {train_acc:.2f}%')
        print(f'  Val Loss: {val_loss/len(val_loader):.4f}, Acc: {val_acc:.2f}%')
        
        # 保存最佳模型
        if val_acc > best_acc:
            best_acc = val_acc
            torch.save(model.state_dict(), 'best_model.pth')
    
    print(f'最佳验证准确率: {best_acc:.2f}%')
    return model
3.4.2 优化器
import torch.optim as optim

# SGD
optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.9, weight_decay=1e-4)

# Adam（最常用）
optimizer = optim.Adam(model.parameters(), lr=0.001, betas=(0.9, 0.999), weight_decay=0)

# AdamW（带权重衰减修正，Transformer常用）
optimizer = optim.AdamW(model.parameters(), lr=0.001, weight_decay=0.01)

# RMSprop
optimizer = optim.RMSprop(model.parameters(), lr=0.01, alpha=0.99)

3.5 模型保存与加载

# 保存模型参数（推荐）
torch.save(model.state_dict(), 'model_weights.pth')

# 加载模型参数
model = SimpleNN(input_size, hidden_size, num_classes)
model.load_state_dict(torch.load('model_weights.pth'))
model.eval()  # 切换到评估模式

# 保存整个模型（不推荐，文件大，兼容性差）
torch.save(model, 'full_model.pth')
model = torch.load('full_model.pth')

# 保存检查点（包含优化器等信息）
checkpoint = {
    'epoch': epoch,
    'model_state_dict': model.state_dict(),
    'optimizer_state_dict': optimizer.state_dict(),
    'loss': loss,
}
torch.save(checkpoint, 'checkpoint.pth')

# 加载检查点
checkpoint = torch.load('checkpoint.pth')
model.load_state_dict(checkpoint['model_state_dict'])
optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
epoch = checkpoint['epoch']

3.6 卷积神经网络 (CNN)

import torch
import torch.nn as nn

class CNN(nn.Module):
    def __init__(self, num_classes=10):
        super(CNN, self).__init__()
        # 输入: (batch, 1, 28, 28) - MNIST
        self.conv1 = nn.Conv2d(1, 32, kernel_size=3, stride=1, padding=1)
        self.bn1 = nn.BatchNorm2d(32)
        self.relu = nn.ReLU()
        self.pool = nn.MaxPool2d(kernel_size=2, stride=2)  # 28x28 -> 14x14
        
        self.conv2 = nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1)
        self.bn2 = nn.BatchNorm2d(64)
        # 再池化: 14x14 -> 7x7
        
        self.fc1 = nn.Linear(64 * 7 * 7, 128)
        self.dropout = nn.Dropout(0.5)
        self.fc2 = nn.Linear(128, num_classes)
    
    def forward(self, x):
        # 卷积层1
        out = self.conv1(x)
        out = self.bn1(out)
        out = self.relu(out)
        out = self.pool(out)
        
        # 卷积层2
        out = self.conv2(out)
        out = self.bn2(out)
        out = self.relu(out)
        out = self.pool(out)
        
        # 展平
        out = out.view(out.size(0), -1)
        
        # 全连接层
        out = self.fc1(out)
        out = self.relu(out)
        out = self.dropout(out)
        out = self.fc2(out)
        
        return out

# 测试
model = CNN()
x = torch.randn(1, 1, 28, 28)  # batch=1, channel=1, 28x28
output = model(x)
print(f"输出形状: {output.shape}")  # torch.Size([1, 10])

3.7 迁移学习

import torch
import torch.nn as nn
from torchvision import models

# 加载预训练模型
model = models.resnet18(pretrained=True)

# 方式1：微调所有层
for param in model.parameters():
    param.requires_grad = True  # 全部可训练

# 方式2：冻结特征提取层，只训练分类头
for param in model.parameters():
    param.requires_grad = False  # 冻结所有层

# 替换最后的全连接层
num_ftrs = model.fc.in_features
model.fc = nn.Linear(num_ftrs, num_classes)  # 新层默认 requires_grad=True

# 训练时只更新fc层的参数
optimizer = optim.Adam(model.fc.parameters(), lr=0.001)

# 常用预训练模型
models.resnet18(pretrained=True)
models.resnet50(pretrained=True)
models.vgg16(pretrained=True)
models.mobilenet_v2(pretrained=True)
models.efficientnet_b0(pretrained=True)

4. TensorFlow / Keras 深度学习框架

4.1 TensorFlow 基础

4.1.1 张量操作
import tensorflow as tf
import numpy as np

# 创建张量
x = tf.constant([1, 2, 3])                  # 常量张量
x = tf.Variable([1.0, 2.0, 3.0])            # 变量张量（可训练）
x = tf.random.normal(shape=(3, 4))          # 标准正态
x = tf.random.uniform(shape=(3, 4))         # 均匀分布
x = tf.zeros(shape=(2, 3))                  # 全零
x = tf.ones(shape=(2, 3))                   # 全一
x = tf.eye(3)                               # 单位矩阵

# 张量属性
print(x.shape)      # 形状
print(x.dtype)      # 数据类型
print(x.device)     # 设备

# 张量运算
a = tf.random.normal((3, 4))
b = tf.random.normal((4, 5))

# 矩阵乘法
c = tf.matmul(a, b)

# 逐元素运算
c = a + b
c = a * b
c = tf.exp(a)
c = tf.sigmoid(a)
c = tf.nn.relu(a)

# 形状操作
x = tf.random.normal((4, 4))
x_reshaped = tf.reshape(x, (2, 8))
x_flat = tf.reshape(x, [-1])  # 展平
x_T = tf.transpose(x)
x_expanded = tf.expand_dims(x, axis=0)
x_squeezed = tf.squeeze(x_expanded)

# GPU 运算
print("GPU可用:", tf.config.list_physical_devices('GPU'))

# NumPy 互转
np_array = x.numpy()
tensor = tf.convert_to_tensor(np_array)
4.1.2 自动求导
import tensorflow as tf

x = tf.Variable(2.0)
w = tf.Variable(3.0)
b = tf.Variable(1.0)

# 使用 GradientTape 记录计算过程
with tf.GradientTape() as tape:
    y = w * x + b  # 前向传播

# 计算梯度
gradients = tape.gradient(y, [w, x, b])
print(f"dy/dw = {gradients[0].numpy()}")  # 3.0
print(f"dy/dx = {gradients[1].numpy()}")  # 2.0
print(f"dy/db = {gradients[2].numpy()}")  # 1.0

# 高阶导数
with tf.GradientTape() as tape2:
    with tf.GradientTape() as tape1:
        y = x ** 3
    dy_dx = tape1.gradient(y, x)
d2y_dx2 = tape2.gradient(dy_dx, x)
print(f"二阶导数: {d2y_dx2.numpy()}")  # 12.0 (x=2时，6x=12)

4.2 Keras 模型构建

4.2.1 Sequential API（顺序模型）
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

# 方式1：Sequential
model = keras.Sequential([
    layers.Dense(256, activation='relu', input_shape=(784,)),
    layers.Dropout(0.3),
    layers.Dense(128, activation='relu'),
    layers.Dropout(0.3),
    layers.Dense(10, activation='softmax')
])

# 方式2：add方法
model = keras.Sequential()
model.add(layers.Dense(256, activation='relu', input_shape=(784,)))
model.add(layers.Dropout(0.3))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dense(10, activation='softmax'))

model.summary()  # 查看模型结构
4.2.2 Functional API（函数式模型）
# 适合多输入、多输出、残差连接等复杂结构
inputs = keras.Input(shape=(784,))

x = layers.Dense(256, activation='relu')(inputs)
x = layers.Dropout(0.3)(x)
x = layers.Dense(128, activation='relu')(x)
x = layers.Dropout(0.3)(x)
outputs = layers.Dense(10, activation='softmax')(x)

model = keras.Model(inputs=inputs, outputs=outputs)
model.summary()
4.2.3 子类化模型
class CustomModel(keras.Model):
    def __init__(self, num_classes=10):
        super(CustomModel, self).__init__()
        self.dense1 = layers.Dense(256, activation='relu')
        self.dropout1 = layers.Dropout(0.3)
        self.dense2 = layers.Dense(128, activation='relu')
        self.dropout2 = layers.Dropout(0.3)
        self.dense3 = layers.Dense(num_classes, activation='softmax')
    
    def call(self, inputs, training=False):
        x = self.dense1(inputs)
        x = self.dropout1(x, training=training)
        x = self.dense2(x)
        x = self.dropout2(x, training=training)
        return self.dense3(x)

model = CustomModel(num_classes=10)

4.3 常用层与激活函数

from tensorflow.keras import layers

# 全连接层
layers.Dense(units, activation=None, use_bias=True)

# 卷积层
layers.Conv2D(filters, kernel_size, strides=(1,1), padding='valid', activation=None)
layers.Conv2DTranspose(filters, kernel_size, strides=(1,1), padding='valid')

# 池化层
layers.MaxPooling2D(pool_size=(2,2), strides=None)
layers.AveragePooling2D(pool_size=(2,2))
layers.GlobalAveragePooling2D()

# 归一化层
layers.BatchNormalization()
layers.LayerNormalization()

# Dropout
layers.Dropout(rate=0.5)
layers.SpatialDropout2D(rate=0.5)

# 循环层
layers.LSTM(units)
layers.GRU(units)
layers.SimpleRNN(units)

# 嵌入层
layers.Embedding(input_dim, output_dim, input_length=None)

# 激活函数（可作为层使用）
layers.ReLU()
layers.LeakyReLU(alpha=0.3)
layers.Softmax()
layers.Activation('sigmoid')

4.4 数据加载

import tensorflow as tf
from tensorflow.keras.datasets import mnist

# 内置数据集
(x_train, y_train), (x_test, y_test) = mnist.load_data()

# 预处理
x_train = x_train.astype('float32') / 255.0
x_test = x_test.astype('float32') / 255.0
x_train = x_train.reshape(-1, 784)
x_test = x_test.reshape(-1, 784)

# 标签独热编码（可选，sparse_categorical_crossentropy不需要）
y_train_onehot = tf.keras.utils.to_categorical(y_train, 10)
y_test_onehot = tf.keras.utils.to_categorical(y_test, 10)

# tf.data.Dataset
train_dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))
train_dataset = train_dataset.shuffle(buffer_size=1024).batch(64).prefetch(tf.data.AUTOTUNE)

# 图像数据生成器（数据增强）
from tensorflow.keras.preprocessing.image import ImageDataGenerator

datagen = ImageDataGenerator(
    rotation_range=15,
    width_shift_range=0.1,
    height_shift_range=0.1,
    horizontal_flip=True,
    rescale=1./255,
    validation_split=0.2
)

train_generator = datagen.flow_from_directory(
    'data/train',
    target_size=(224, 224),
    batch_size=32,
    class_mode='categorical',
    subset='training'
)

4.5 模型编译与训练

# 编译模型
model.compile(
    optimizer='adam',                    # 优化器
    loss='sparse_categorical_crossentropy',  # 损失函数
    metrics=['accuracy']                 # 评估指标
)

# 自定义优化器和学习率
optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)
model.compile(
    optimizer=optimizer,
    loss=tf.keras.losses.SparseCategoricalCrossentropy(),
    metrics=[tf.keras.metrics.SparseCategoricalAccuracy()]
)

# 训练
history = model.fit(
    x_train, y_train,
    epochs=10,
    batch_size=64,
    validation_split=0.1,    # 从训练集划分验证集
    # validation_data=(x_val, y_val),  # 或使用指定验证集
    verbose=1
)

# 评估
test_loss, test_acc = model.evaluate(x_test, y_test, verbose=0)
print(f'测试准确率: {test_acc:.4f}')

# 预测
predictions = model.predict(x_test)
predicted_classes = tf.argmax(predictions, axis=1)

4.6 回调函数

from tensorflow.keras.callbacks import (
    ModelCheckpoint, EarlyStopping, ReduceLROnPlateau, TensorBoard
)

callbacks = [
    # 保存最佳模型
    ModelCheckpoint(
        'best_model.h5',
        monitor='val_accuracy',
        save_best_only=True,
        mode='max',
        verbose=1
    ),
    
    # 早停
    EarlyStopping(
        monitor='val_loss',
        patience=5,           # 5个epoch没改善就停止
        restore_best_weights=True,
        verbose=1
    ),
    
    # 学习率衰减
    ReduceLROnPlateau(
        monitor='val_loss',
        factor=0.1,           # 学习率乘以0.1
        patience=3,           # 3个epoch没改善就降低
        min_lr=1e-7,
        verbose=1
    ),
    
    # TensorBoard 可视化
    TensorBoard(log_dir='./logs')
]

# 训练时传入 callbacks
history = model.fit(
    x_train, y_train,
    epochs=50,
    batch_size=64,
    validation_split=0.1,
    callbacks=callbacks
)

4.7 CNN 示例

import tensorflow as tf
from tensorflow.keras import layers, models

def create_cnn(input_shape=(28, 28, 1), num_classes=10):
    model = models.Sequential([
        # 卷积块1
        layers.Conv2D(32, (3, 3), padding='same', input_shape=input_shape),
        layers.BatchNormalization(),
        layers.Activation('relu'),
        layers.MaxPooling2D((2, 2)),
        
        # 卷积块2
        layers.Conv2D(64, (3, 3), padding='same'),
        layers.BatchNormalization(),
        layers.Activation('relu'),
        layers.MaxPooling2D((2, 2)),
        
        # 卷积块3
        layers.Conv2D(128, (3, 3), padding='same'),
        layers.BatchNormalization(),
        layers.Activation('relu'),
        layers.GlobalAveragePooling2D(),
        
        # 分类头
        layers.Dense(128, activation='relu'),
        layers.Dropout(0.5),
        layers.Dense(num_classes, activation='softmax')
    ])
    
    return model

model = create_cnn()
model.summary()

4.8 迁移学习

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.applications import ResNet50

# 加载预训练模型（不包含顶部分类层）
base_model = ResNet50(
    weights='imagenet',
    include_top=False,
    input_shape=(224, 224, 3)
)

# 冻结基础模型
base_model.trainable = False

# 构建完整模型
model = models.Sequential([
    base_model,
    layers.GlobalAveragePooling2D(),
    layers.Dense(256, activation='relu'),
    layers.Dropout(0.5),
    layers.Dense(num_classes, activation='softmax')
])

# 先训练顶部分类层
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(train_data, epochs=10, validation_data=val_data)

# 微调：解冻部分层
base_model.trainable = True
fine_tune_at = 100  # 解冻最后100层

for layer in base_model.layers[:fine_tune_at]:
    layer.trainable = False

# 使用更小的学习率微调
model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=1e-5),
    loss='categorical_crossentropy',
    metrics=['accuracy']
)

model.fit(train_data, epochs=20, initial_epoch=10, validation_data=val_data)

4.9 模型保存与加载

# 保存整个模型（架构+权重+优化器状态）
model.save('my_model.h5')

# 加载模型
model = tf.keras.models.load_model('my_model.h5')

# 只保存权重
model.save_weights('my_weights.h5')

# 加载权重（需要先构建相同结构的模型）
model.load_weights('my_weights.h5')

# SavedModel 格式（推荐，TensorFlow原生格式）
model.save('saved_model/my_model')
model = tf.keras.models.load_model('saved_model/my_model')

5. 三大框架对比与选型

5.1 框架对比

维度scikit-learnPyTorchTensorFlow/Keras
定位传统机器学习深度学习研究/生产深度学习生产/部署
学习曲线简单中等较陡（Keras简单）
灵活性低（封装好）高（动态图）中（静态图，2.0后有eager）
调试难度简单简单（Python原生调试）较难
生产部署简单中等（TorchScript/ONNX）强（TFLite/Serving）
社区生态传统ML最丰富学术圈最火工业界最成熟
数据量需求小到中大大
适用场景表格数据、传统ML研究、NLP、CV生产部署、移动端

5.2 选型建议

选择决策树：
├── 数据是表格型（结构化数据）
│   ├── 特征工程为主 → scikit-learn
│   └── 需要深度模型 → PyTorch / TensorFlow
│
├── 图像 / 计算机视觉
│   ├── 研究创新 → PyTorch
│   └── 生产部署 → TensorFlow
│
├── 自然语言处理
│   └── 绝大多数情况 → PyTorch（HuggingFace生态）
│
├── 移动端 / 嵌入式
│   └── TensorFlow Lite
│
└── 快速原型验证
    └── Keras (TensorFlow)

5.3 学习路径建议

第一阶段：基础
├── Python 基础（numpy, pandas, matplotlib）
├── 机器学习基础理论
└── scikit-learn 实践

第二阶段：深度学习入门
├── 神经网络基础（BP、CNN、RNN）
├── PyTorch 或 TensorFlow 二选一
└── MNIST / CIFAR 分类实战

第三阶段：进阶
├── 选择主攻框架深入
├── 学习经典网络架构
├── 迁移学习、数据增强
└── 一个完整项目实战

第四阶段：专项深入
├── CV方向：目标检测、分割、生成模型
├── NLP方向：Transformer、BERT、大模型
└── 工程化：部署、优化、MLOps

6. 经典项目实战

6.1 项目1：房价预测（回归任务）

# 使用 scikit-learn
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_squared_error, r2_score

# 1. 数据加载与探索
# data = pd.read_csv('house_prices.csv')
# print(data.describe())

# 2. 特征工程（略）
# 处理缺失值、编码分类特征、特征组合...

# 3. 数据划分
# X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# 4. 模型训练
# scaler = StandardScaler()
# X_train_scaled = scaler.fit_transform(X_train)
# X_test_scaled = scaler.transform(X_test)
# 
# model = RandomForestRegressor(n_estimators=100, random_state=42)
# model.fit(X_train_scaled, y_train)

# 5. 评估
# y_pred = model.predict(X_test_scaled)
# print(f'RMSE: {np.sqrt(mean_squared_error(y_test, y_pred))}')
# print(f'R²: {r2_score(y_test, y_pred)}')

6.2 项目2：手写数字识别（CNN分类）

# 使用 PyTorch
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

# 数据准备
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,))
])

train_dataset = datasets.MNIST('./data', train=True, download=True, transform=transform)
test_dataset = datasets.MNIST('./data', train=False, transform=transform)

train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True)
test_loader = DataLoader(test_dataset, batch_size=1000, shuffle=False)

# 模型定义（CNN）
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 32, 3, 1)
        self.conv2 = nn.Conv2d(32, 64, 3, 1)
        self.dropout1 = nn.Dropout2d(0.25)
        self.dropout2 = nn.Dropout(0.5)
        self.fc1 = nn.Linear(9216, 128)
        self.fc2 = nn.Linear(128, 10)

    def forward(self, x):
        x = self.conv1(x)
        x = torch.relu(x)
        x = self.conv2(x)
        x = torch.relu(x)
        x = torch.max_pool2d(x, 2)
        x = self.dropout1(x)
        x = torch.flatten(x, 1)
        x = self.fc1(x)
        x = torch.relu(x)
        x = self.dropout2(x)
        x = self.fc2(x)
        return x

# 训练
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = Net().to(device)
optimizer = optim.Adam(model.parameters(), lr=0.001)
criterion = nn.CrossEntropyLoss()

# 训练循环（参考3.4.1完整训练模板）

6.3 项目3：图像分类（迁移学习）

# 使用 TensorFlow/Keras
import tensorflow as tf
from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras import layers, models

# 数据加载（ImageDataGenerator 或 tf.data）
# train_ds = tf.keras.utils.image_dataset_from_directory(
#     'data/train',
#     image_size=(224, 224),
#     batch_size=32
# )

# 基础模型
base_model = MobileNetV2(
    weights='imagenet',
    include_top=False,
    input_shape=(224, 224, 3)
)
base_model.trainable = False

# 构建模型
model = models.Sequential([
    base_model,
    layers.GlobalAveragePooling2D(),
    layers.Dense(128, activation='relu'),
    layers.Dropout(0.5),
    layers.Dense(num_classes, activation='softmax')
])

# 编译训练
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
# model.fit(train_ds, epochs=10, validation_data=val_ds)

# 微调
base_model.trainable = True
fine_tune_at = 100
for layer in base_model.layers[:fine_tune_at]:
    layer.trainable = False

model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=1e-5),
    loss='categorical_crossentropy',
    metrics=['accuracy']
)
# model.fit(train_ds, epochs=20, initial_epoch=10, validation_data=val_ds)

7. 学习资源推荐

7.1 在线课程

课程平台特点
吴恩达机器学习Coursera经典入门，理论扎实
吴恩达深度学习专项Coursera深度学习入门首选
CS231n 计算机视觉StanfordCV方向经典
CS224n NLPStanfordNLP方向经典
fast.aifast.ai实战导向，PyTorch

7.2 书籍

  • 《统计学习方法》 - 李航：传统机器学习经典，中文
  • 《深度学习》 - Ian Goodfellow：深度学习圣经（花书）
  • 《动手学深度学习》 - 李沐：实战导向，有PyTorch/TensorFlow版
  • 《Python机器学习》 - Sebastian Raschka：scikit-learn实战

7.3 文档与教程

  • scikit-learn 官方文档：https://scikit-learn.org/stable/
  • PyTorch 官方教程：https://pytorch.org/tutorials/
  • TensorFlow 官方教程：https://www.tensorflow.org/tutorials
  • HuggingFace：https://huggingface.co/（NLP必备）

7.4 练习平台

  • Kaggle：数据科学竞赛平台
  • LeetCode：算法练习
  • Papers With Code：论文+代码
  • GitHub Awesome系列：各方向资源汇总

附录：常用代码片段速查

A.1 环境配置

# 创建虚拟环境
conda create -n ml python=3.9
conda activate ml

# 安装基础库
pip install numpy pandas matplotlib seaborn scikit-learn

# 安装 PyTorch (CUDA 11.8)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

# 安装 TensorFlow
pip install tensorflow

# 验证安装
python -c "import torch; print('PyTorch:', torch.__version__); print('CUDA:', torch.cuda.is_available())"
python -c "import tensorflow as tf; print('TensorFlow:', tf.__version__)"

A.2 设置随机种子

# PyTorch
import torch
import numpy as np
import random

def set_seed(seed=42):
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)
    torch.cuda.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)
    torch.backends.cudnn.deterministic = True
    torch.backends.cudnn.benchmark = False

set_seed(42)

# TensorFlow
import tensorflow as tf
tf.random.set_seed(42)
np.random.seed(42)

A.3 训练可视化

# 使用 matplotlib 绘制训练曲线
import matplotlib.pyplot as plt

def plot_history(history):
    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))
    
    # 损失曲线
    ax1.plot(history['loss'], label='train')
    ax1.plot(history['val_loss'], label='val')
    ax1.set_title('Loss')
    ax1.set_xlabel('Epoch')
    ax1.legend()
    
    # 准确率曲线
    ax2.plot(history['accuracy'], label='train')
    ax2.plot(history['val_accuracy'], label='val')
    ax2.set_title('Accuracy')
    ax2.set_xlabel('Epoch')
    ax2.legend()
    
    plt.tight_layout()
    plt.show()

学习建议：理论 + 实践相结合，每学一个算法都动手实现一遍，然后在真实数据集上验证。从简单的项目开始，逐步增加复杂度。遇到问题先查官方文档，再搜社区解决方案。坚持做项目，边做边学是最快的成长方式。

# python # 人工智能 # 机器学习
Logo
亚马逊云科技技术品牌专区

更多推荐

cover

基于STM32的全屋环境智能监测与预警终端设计

avatar 亚马逊云科技技术品牌专区

Keras：面向人类的深度学习库

Keras3支持多后端（TensorFlow、PyTorch、JAX），通过设置环境变量切换，兼容指定版本。pip安装后可用Sequential API快速构建CNN模型，如MNIST分类示例，实

avatar 亚马逊云科技技术品牌专区

AI行业日报｜2026-09-25：5个热点事件

本期行业动态聚焦于端侧算力重构、自主智能体工作流、开发工具经济模型、网络隔离架构设计与公共部门合规审查。以下结合一线技术场景进行结构化梳理，供 AI 研究员与软件工程团队参考。 PrismML 正将其轻量级语言模型适配至搭载高通处理器的智能眼镜终端 1 。该平台的核心战略指向开源权重模型的本地化运行，旨在最大化挖掘穿戴设备现有算力的利用效率 1 。对于嵌入式与边缘计算开发者而言，此类微型模型向低功

avatar 亚马逊云科技技术品牌专区