From KNN Basics to Practice: Why StandardScaler and OneHotEncoder Matter

KNN（K-Nearest Neighbors，K近邻算法）是机器学习中最容易理解的算法之一：

KNN (K-Nearest Neighbors) is one of the easiest machine learning algorithms to understand.

它不像神经网络那样需要复杂训练，也不像决策树那样有大量规则，它的核心逻辑只有一句话

Unlike neural networks, it does not require complex training, and unlike decision trees, it does not rely on many rules. Its core idea can be summarized in one sentence:

一个样本属于什么类别，取决于它周围最近的 K 个样本。

A sample’s label is determined by the K nearest samples around it.

但也正因为它的逻辑完全依赖“距离”，所以 KNN 对数据预处理极其敏感。

However, because KNN relies entirely on distance, it is extremely sensitive to data preprocessing.

本文用 4 个 Python 文件循序渐进讲清楚：

This article uses 4 Python scripts to explain step by step:

• KNN 分类是什么
  What KNN classification is

• KNN 回归是什么
  What KNN regression is

• 标准化/归一化是什么
  What scaling/normalization is

• 为什么在真实数据中必须使用 StandardScaler + OneHotEncoder
  Why StandardScaler + OneHotEncoder are required in real-world datasets

1. KNN 分类：最近的邻居投票决定类别

1. KNN Classification: Nearest Neighbors Vote for the Label

先看一个最小的 KNN 分类例子：
Let’s start with a minimal KNN classification example:

from sklearn.neighbors import KNeighborsClassifier

knn = KNeighborsClassifier(n_neighbors=2, weights='distance')
X = [[2, 1], [3, 1], [1, 4], [2, 6]]
y = [0, 1, 0, 1]
knn.fit(X, y)

x = [[4, 9]]
x_class = knn.predict(x)
print(x_class)

1.1 代码里的核心概念

1.1 Core Concepts in the Code

• X：特征（Feature），每一行是一个样本
  X: Features, each row is one sample

• y：标签（Label），每个样本的类别
  y: Labels, the class for each sample

• fit(X, y)：训练（对 KNN 来说，本质就是“存数据”）
  fit(X, y): Training (for KNN, it essentially stores the data)

• predict(x)：预测新点属于哪个类别
  predict(x): Predicts the class of a new point

1.2 n_neighbors=2 是什么？

1.2 What Does n_neighbors=2 Mean?

它表示：预测新样本时，只看最近的 2 个样本。
It means: when predicting a new sample, KNN only looks at the 2 nearest samples.

1.3 weights='distance' 是什么？

1.3 What Does weights='distance' Mean?

默认 KNN 投票是“人人一票”。而 weights='distance' 的意思是：
By default, KNN uses uniform voting. But weights='distance' means:

距离越近的邻居，投票权重越大。
The closer a neighbor is, the higher its voting weight.

这通常更合理，因为离得更近的点更有参考价值。
This is usually more reasonable because closer points should have more influence.

2. KNN 回归：最近的邻居加权平均

2. KNN Regression: Weighted Average of Nearest Neighbors

KNN 不仅能分类，还能回归。
KNN can be used not only for classification but also for regression.

from sklearn.neighbors import KNeighborsRegressor

knn = KNeighborsRegressor(n_neighbors=2, weights='distance')
X = [[2, 1], [3, 1], [1, 4], [2, 6]]
y = [0.5, 0.33, 4, 3]
knn.fit(X, y)

x = [[4, 9]]
x_pred = knn.predict(x)
print(x_pred)

2.1 KNN 回归的输出是什么？

2.1 What Is the Output of KNN Regression?

• 分类输出：0 或 1
  Classification output: 0 or 1

• 回归输出：连续值，比如 2.3、4.8
  Regression output: continuous values like 2.3 or 4.8

2.2 KNN 回归的核心逻辑

2.2 Core Logic of KNN Regression

找到最近 K 个邻居，把它们的 y 值做加权平均。
Find the K nearest neighbors and compute the weighted average of their y values.

3. 缩放器（Scaler）：为什么需要把特征拉到同一尺度？

3. Scalers: Why Features Must Be on the Same Scale

KNN 最大的问题就是：
The biggest issue with KNN is:

KNN 完全依赖距离，而距离完全依赖特征的数值大小。
KNN relies entirely on distance, and distance depends heavily on feature scales.

3.1 MinMaxScaler（归一化）

3.1 MinMaxScaler (Normalization)

from sklearn.preprocessing import MinMaxScaler
X = [[2, 1], [3, 1], [1, 4], [2, 6]]

scaler = MinMaxScaler(feature_range=(0,1))
X_scaled = scaler.fit_transform(X)
print(X_scaled)

归一化会把每一列缩放到 [0, 1] 区间。
Normalization scales each feature column into the [0, 1] range.

3.2 StandardScaler（标准化）

3.2 StandardScaler (Standardization)

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
print(X_scaled)

标准化的目标是：
The goal of standardization is:

• 均值变成 0
  Mean becomes 0

• 标准差变成 1
  Standard deviation becomes 1

#3_scaler_test.py
from sklearn.preprocessing import MinMaxScaler
import numpy as np
X = [[2, 1], [3, 1], [1, 4], [2, 6]]
print(np.array(X))
sclar = MinMaxScaler(feature_range=(0,1))#缩放到目标范围
X_scaled = sclar.fit_transform(X)
print(X_scaled)


#定义标准化缩放器
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
print(X_scaled)

#公式验证
X=np.array(X)
mean=np.mean(X,axis=0)
std=np.std(X,axis=0)
print(mean)
print(std)
X_scaled=(X-mean)/std
print(X_scaled)

3.3 为什么 KNN 特别依赖 StandardScaler？

3.3 Why KNN Strongly Depends on StandardScaler?

因为 KNN 的“距离”计算类似于：
Because KNN computes distances like:

如果某个特征数值范围特别大，比如胆固醇（100~500），它会主导距离。
If a feature has a much larger range (e.g., cholesterol 100–500), it will dominate the distance.

结果是：
As a result:

KNN 会认为胆固醇是最重要的特征（即使它不一定最重要）。
KNN will treat cholesterol as the most important feature (even if it is not).

所以在 KNN 中：标准化几乎是必做的。
Therefore, for KNN: standardization is almost mandatory.

4. 实战案例：心脏病预测里为什么必须 StandardScaler + OneHotEncoder？

4. Practical Case: Why StandardScaler + OneHotEncoder Are Required for Heart Disease Prediction

4.1 数据特征并不全是数字：有三种类型

数值型特征（numerical）

numerical_features = ["年龄", "静息血压", "胆固醇", "最大心率", "运动后的ST下降", "主血管数量"]

特点：可以直接参与数学运算。
These can be used directly in mathematical operations.

类别型特征（categorical）

categorical_features = ["胸痛类型", "静息心电图结果", "峰值ST段的斜率", "地中海贫血"]

特点：是类别，不存在“大小顺序”。
These are categories with no inherent order.

二元特征（binary）

binary_features = ["性别", "空腹血糖", "运动性心绞痛"]

特点：只有 0/1。
These contain only 0/1 values.

4.2 第一重点：StandardScaler（标准化缩放器）

4.2 Key Point #1: StandardScaler

("numeric", StandardScaler(), numerical_features)

意义是：
The purpose is:

把不同量纲的数值特征拉到同一尺度，让距离计算公平。
Bring features with different units into the same scale, making distance calculation fair.

4.3 第二重点：OneHotEncoder（独热编码器）

4.3 Key Point #2: OneHotEncoder

Applied OneHot encoding to categorical features:

("categorical", OneHotEncoder(drop="first"), categorical_features)

4.3.1 为什么类别特征不能直接用数字？

4.3.1 Why Can’t We Use Raw Numbers for Categories?

如果你把胸痛类型直接编码为 0/1/2/3，KNN 会误以为 0 和 1 更接近，0 和 3 更远。
If you encode chest pain types as 0/1/2/3, KNN will assume 0 is closer to 1 than to 3.

但类别之间没有这种距离关系。
But categorical values do not have such distance relationships.

4.3.2 OneHotEncoder 做了什么？

它把一个类别列拆成多个 0/1 列。
It converts one categorical column into multiple 0/1 columns.

这样 KNN 计算距离时：
Then KNN distance behaves correctly:

• 同类别 → 差值为 0
  Same category → difference 0

• 不同类别 → 差值为 1
  Different category → difference 1

4.4 ColumnTransformer：把预处理流程工程化

4.4 ColumnTransformer: Engineering the Whole Preprocessing Pipeline

column_transformer = ColumnTransformer(transformers=[
    ("numeric", StandardScaler(), numerical_features),
    ("categorical", OneHotEncoder(drop="first"), categorical_features),
    ("binary", StandardScaler(), binary_features)
])

Its purpose is:

对不同类型特征使用不同处理方式，并自动拼接输出。
Apply different transformations to different feature types and automatically concatenate outputs.

4.5 为什么特征列数从 13 变成 19？

4.5 Why Did the Feature Count Change from 13 to 19?

原因是：
Because:

• 数值特征列数不变
  Numerical features keep their count

• 二元特征列数不变
  Binary features keep their count

• 类别特征 OneHot 后列数增加
  Categorical features expand after OneHot encoding

StandardScaler 解决的问题

What StandardScaler Solves

数值特征量纲不同导致距离失真。
Different feature scales distort distances.

OneHotEncoder 解决的问题

What OneHotEncoder Solves

类别特征没有大小顺序，不能直接当数字算距离。
Categorical features have no order, so they cannot be treated as numeric distances.

5. 超参数调优：用 GridSearchCV 找到最优 K 值

6. Hyperparameter Tuning: Finding the Optimal K with GridSearchCV

knn = KNeighborsClassifier(n_neighbors=3)

在前面的步骤中，我们手动设置 n_neighbors=3，但这个值并不一定是最优的。
In previous steps, we manually set n_neighbors=3, but this value is not necessarily optimal.

在 KNN 中，n_neighbors 是最核心的超参数，它直接影响模型的复杂度。
In KNN, n_neighbors is the most important hyperparameter because it directly controls model complexity.

5.1 什么是超参数（Hyperparameter）？

What Is a Hyperparameter?

超参数是训练开始前由人为设定的参数，而不是模型从数据中学习得到的参数。
A hyperparameter is a parameter set before training and is not learned from the data.

在 KNN 中，n_neighbors、weights、metric 都属于超参数。
In KNN, n_neighbors, weights, and metric are all hyperparameters.

5.2 使用 GridSearchCV 搜索最优参数

5.2 Using GridSearchCV to Search for the Best Parameter

# 定义参数网格
param_grid = {"n_neighbors": list(range(1,16))}

# 创建 GridSearchCV 对象
gridSearch_CV = GridSearchCV(estimator=knn, param_grid=param_grid, cv=10)

# 训练模型（包含交叉验证）
gridSearch_CV.fit(x_train, y_train)

# 输出结果
print(gridSearch_CV.best_params_)
print(gridSearch_CV.best_score_)
print(gridSearch_CV.best_estimator_)
print(pd.DataFrame(gridSearch_CV.cv_results_).to_string())

5.3 参数网格的含义

5.3 Meaning of the Parameter Grid

range(1,16) 表示尝试 K=1 到 K=15 的所有可能值。
range(1,16) means trying all possible K values from 1 to 15.

模型会对每一个 K 值进行评估。
The model will evaluate each K value.

cv=10 表示使用 10 折交叉验证。
cv=10 means using 10-fold cross validation.

数据会被分成 10 份，每次用 9 份训练，1 份验证，重复 10 次。
The data is split into 10 parts; each time 9 parts are used for training and 1 part for validation, repeated 10 times.

最终取 10 次结果的平均值作为模型评分。
The final score is the average of the 10 validation results.

这样做可以减少单次随机划分带来的偶然性。
This reduces randomness caused by a single train-test split.

5.4 如何验证是否过拟合？

5.4 How to verify overfitting?

可以画 k vs accuracy 曲线：

You can plot k vs accuracy curve:

import matplotlib.pyplot as plt

k_range = range(1,16)
scores = []

for k in k_range:
    knn = KNeighborsClassifier(n_neighbors=k)
    knn.fit(x_train,y_train)
    scores.append(knn.score(x_test,y_test))

plt.plot(k_range,scores)
plt.xlabel("k")
plt.ylabel("Accuracy")
plt.show()

GridSearchCV 选 k=1，是因为在 10 折交叉验证中，它的平均准确率最高。
GridSearchCV selected k=1 because it achieved the highest mean cross-validation accuracy under 10-fold CV.