一、逻辑回归

逻辑回归（Logistic Regression）是一种用于解决分类问题的统计方法，尤其适用于二分
类问题。尽管名称中有“回归”，但它主要用于分类任务。

1、损失函数

对数损失Log Loss，又叫二元交叉熵损失（Binary Cross-Entropy Loss        BCE），用于衡量模型输出的概率分布与真实标签之间的差距。

2、案例——心脏病预测

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler,OneHotEncoder
from sklearn.linear_model import LogisticRegression

dataset = pd.read_csv('E:\\Py_project\\numpy_pandas\\heart_disease\\data\\heart_disease.csv')
dataset.dropna(inplace=True)

x = dataset.drop('是否患有心脏病',axis=1)
y = dataset['是否患有心脏病']
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.3,random_state=42)

numerical_features = ["年龄", "静息血压", "胆固醇", "最大心率", "运动后的ST下降", "主血管数量"]
categorical_features = ["胸痛类型", "静息心电图结果", "峰值ST段的斜率", "地中海贫血"]
binary_features = ["性别", "空腹血糖", "运动性心绞痛"]

column_transformer = ColumnTransformer(transformers=[
    ("num",StandardScaler(),numerical_features),
    ('cat',OneHotEncoder(drop = 'first'),categorical_features),
    ('bin','passthrough',binary_features)
])
x_train = column_transformer.fit_transform(x_train)
x_test = column_transformer.transform(x_test)

model = LogisticRegression(
    solver='saga',
    max_iter=1000,
    penalty='l1',
    C=0.5,
    class_weight='balanced',
    random_state=42,
)
model.fit(x_train,y_train)
score = model.score(x_test,y_test)

print(score)

0.8181818181818182

3、多分类任务

一对多OVR（One-vs-Rest）和Softmax回归（Multinomial Logistic Regression，多项逻辑回归）

OVR是当作K个二分类器模型，概率之和不一定就等于1，比较适合类别比较少的情况

Softmax是一个函数，取代之前的sigmoid二分函数，当作一个多分类模型，概率之和等于1

以下是OVR模型创建代码，因为版本问题推荐用model_ovr2的方式创建

from sklearn.linear_model import LogisticRegression
from sklearn.multiclass import OneVsRestClassifier
model_ovr1 = LogisticRegression(multi_class="ovr")
model_ovr2 = OneVsRestClassifier(LogisticRegression())

以下是多项逻辑回归Softmax，softmax2是最新版本，默认用multinomial，现在只要是分类任务大于等于3个的时候，就是默认multinomial，不用传参数了，2分类就是用之前的逻辑回归，一般不用OVR，如果实在想用，看上面的代码。

from sklearn.linear_model import LogisticRegression
model_softmax = LogisticRegression(multi_class='multinomial')
model_softmax2 = LogisticRegression()

4、案例——手写数字识别

imshow是一种把数字转换成颜色的画图翻译器

import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import MinMaxScaler


dataset = pd.read_csv('E:\\Py_project\\numpy_pandas\\day01\\data\\train.csv')

X = dataset.drop(columns=['label'],axis=1)
y = dataset['label']
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
digit = x_train.iloc[3,:].values.reshape(28,28)
label = y_train.iloc[3]
print(label)
plt.imshow(digit,cmap = 'gray')
plt.show()

scalar = MinMaxScaler()
x_train = scalar.fit_transform(x_train)
x_test = scalar.transform(x_test)

model = LogisticRegression(
    max_iter=500,
)
model.fit(x_train,y_train)
score = model.score(x_test,y_test)
print(score)

9
0.9186507936507936

二、感知机perceptron

1、逻辑门代码实现

与门为例

import numpy as np

def AND0(x1,x2):
    w1,w2,theta = 0.5,0.5,0.7
    result = w1 * x1 + w2 * x2
    if result <= theta:
        return 0
    else:
        return 1
def AND1(x1,x2):
    x = np.array([x1,x2])
    w = np.array([0.5,0.5])
    b = -0.7
    result = w @ x + b
    if result <= 0:
        return 0
    else:
        return 1

2、感知机的局限

异或门

3、多层感知机实现异或门

组合构成异或门

代码实现的话就是调用三个函数，传入对应结果