随机森林特征重要性分析与K8s资源配置实战2026版:机器学习模型部署完整指南
## 前言
本文结合随机森林特征重要性分析和K8s资源配置两个主题,展示从模型训练到生产部署的完整流程。
## 一、随机森林特征重要性
### 1.1 原理
随机森林通过两种方式计算特征重要性:
- 基尼不纯度下降:每个特征在树分裂时减少的不纯度
- 排列重要性:打乱特征值后模型性能的下降程度
### 1.2 代码实现
```python
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification
import numpy as np
import matplotlib.pyplot as plt
# 生成数据
X, y = make_classification(n_samples=1000, n_features=20,
n_informative=10, random_state=42)
feature_names = [f'feature_{i}' for i in range(20)]
# 训练模型
rf = RandomForestClassifier(n_estimators=100, random_state=42)
rf.fit(X, y)
# 特征重要性
importances = rf.feature_importances_
indices = np.argsort(importances)[::-1]
# 可视化
plt.figure(figsize=(10, 6))
plt.bar(range(len(importances)), importances[indices])
plt.xticks(range(len(importances)),
[feature_names[i] for i in indices], rotation=45)
plt.title('Feature Importances')
plt.tight_layout()
plt.savefig('feature_importance.png')
```
### 1.3 排列重要性
```python
from sklearn.inspection import permutation_importance
result = permutation_importance(rf, X, y, n_repeats=10, random_state=42)
for i in result.importances_mean.argsort()[::-1]:
if result.importances_mean[i] - 2 * result.importances_std[i] > 0:
print(f"{feature_names[i]:20s} "
f"{result.importances_mean[i]:.4f} "
f"+/- {result.importances_std[i]:.4f}")
```
## 二、模型优化
### 2.1 特征选择
```python
from sklearn.feature_selection import SelectFromModel
# 自动选择重要特征
selector = SelectFromModel(rf, threshold='median')
X_selected = selector.fit_transform(X, y)
print(f"Selected {X_selected.shape[1]} features")
print(f"Selected features: {[feature_names[i] for i in selector.get_support(indices=True)]}")
```
### 2.2 超参数调优
```python
from sklearn.model_selection import GridSearchCV
param_grid = {
'n_estimators': [50, 100, 200],
'max_depth': [10, 20, None],
'min_samples_split': [2, 5, 10],
}
grid_search = GridSearchCV(rf, param_grid, cv=5, scoring='accuracy')
grid_search.fit(X, y)
print(f"Best params: {grid_search.best_params_}")
print(f"Best score: {grid_search.best_score_:.4f}")
```
## 三、K8s部署
### 3.1 模型服务化
```python
# app.py - Flask API
from flask import Flask, request, jsonify
import pickle
import numpy as np
app = Flask(__name__)
with open('model.pkl', 'rb') as f:
model = pickle.load(f)
@app.route('/predict', methods=['POST'])
def predict():
data = request.json
features = np.array(data['features']).reshape(1, -1)
prediction = model.predict(features)
probability = model.predict_proba(features)
return jsonify({
'prediction': int(prediction[0]),
'probability': float(probability[0][1])
})
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5000)
```
### 3.2 Docker化
```dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 5000
CMD ["gunicorn", "-w", "4", "-b", "0.0.0.0:5000", "app:app"]
```
### 3.3 K8s部署
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ml-model
spec:
replicas: 3
selector:
matchLabels:
app: ml-model
template:
metadata:
labels:
app: ml-model
spec:
containers:
- name: model
image: my-registry/ml-model:v1
resources:
requests:
memory: "512Mi"
cpu: "250m"
limits:
memory: "1Gi"
cpu: "500m"
livenessProbe:
httpGet:
path: /health
port: 5000
initialDelaySeconds: 30
readinessProbe:
httpGet:
path: /ready
port: 5000
initialDelaySeconds: 5
---
apiVersion: v1
kind: Service
metadata:
name: ml-model-svc
spec:
selector:
app: ml-model
ports:
- port: 80
targetPort: 5000
type: LoadBalancer
```
## 四、监控与运维
### 4.1 资源监控
- CPU使用率:kubectl top pods
- 内存使用率:kubectl top nodes
- 请求延迟:Prometheus + Grafana
- 预测准确率:定期评估线上模型
### 4.2 自动扩缩容
```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: ml-model-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: ml-model
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
```
## 五、总结
1. 特征重要性分析帮助理解模型决策依据
2. 排列重要性比基尼重要性更可靠
3. 模型服务化需要考虑并发和延迟
4. K8s提供弹性伸缩和自愈能力
5. 生产环境需要完善的监控和告警体系
更多推荐
所有评论(0)