目录

Mamba优势

1. 本人环境配置

1.1 傻瓜式安装Cuda

1.2 上流环境变量法

2. 安装Causal-Conv1d 

2.1 直接安装(亲测不是很行)

2.2 傻瓜式安装 

3. 安装mamba-ssm

4. 参考

---

安装完之后发现自己真是大聪明（bushi

最近MambaVision也开始火了，老师那颗热爱科研的心也是有一次被激起了，我那个悬着的心也终于在配置Mamba的时候Die了.......

---

Mamba优势

1. Selection mechanism（选择机制）引入了Gate，类似RNN的门控机制
2. Linearly in sequence length（线性计算），降低train和inference计算量
3. Hardware-aware Algorithm 降低硬件开销
4. Architecture was composed with H3 and Gated MLP

---

1. 本人环境配置

python==3.10.14
causal-conv1d==1.1.1
triton==2.0.0
mamba-ssm==1.2.0.post1
torch==2.1.1+cu118
torchaudio==2.1.1+cu118
torchvision==0.16.1+cu118

由于之前在网站上面搜索的时候都是基于cu118+python3.10的安装包（triton的whl包也是），因此以cuda11.8进行安装，主打一个听劝^.^

1.1 傻瓜式安装Cuda

直接去Nvidia官网进行下载cu11.8cu11.8https://developer.nvidia.com/cuda-11-8-0-download-archive?target_os=Windows

最好是进行下载local文件进行本地安装，exe可能需要科学，安装完了之后可以直接进环境变量path看看是否有cu11.8（nvidia-ssm中的cuda version只是安装过版本问题，只需要查看nvcc -V查看是否为

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:41:10_Pacific_Daylight_Time_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

1.2 上流环境变量法

创建虚拟环境并且安装cudatoolkit==11.8，这样在虚拟变量中会额外有一个cuda版本（比较便捷），后面的cuda-nvcc一定要加，不然当前虚拟环境可能找不到对应的cuda版本

conda create -n your_env_name python=3.10.13
conda activate your_env_name
conda install cudatoolkit==11.8 -c nvidia
pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu118
conda install -c "nvidia/label/cuda-11.8.0" cuda-nvcc

2. 安装Causal-Conv1d 

安装Causal-Conv1d在checkout的时候一定要找对应cuda的版本

首先安装一下 packaging

conda install packaging

2.1 直接安装(亲测不是很行)

pip install causal-conv1d==1.1.1

2.2 傻瓜式安装 

首先安装一下triton（里面包含了cmake）对后续编译提供基础

triton-2.0.0-cp10https://hf-mirror.com/r4ziel/xformers_pre_built/blob/main/triton-2.0.0-cp310-cp310-win_amd64.whl

后续就是开始在github上面下载安装本地文件

git clone https://github.com/Dao-AILab/causal-conv1d.git
cd causal-conv1d
git checkout v1.1.1

******为什么要checkout v1.1.1 因为这是支持cu118的最高版本******

然后就是进行install causal-conv1d的过程了

要在git（没有的话在git_download中进行下载）进行安装，在windows terminal中识别不到命令行

CAUSAL_CONV1D_FORCE_BUILD=TRUE pip install .

 按照这个步骤安装causal-conv1d应该不是什么大问题^.^

3. 安装mamba-ssm

不说别的歪门邪道了（直接傻瓜式安装）离线安装

git clone https://github.com/state-spaces/mamba.git
cd mamba
git checkout v1.1.1

 先下载mamba的文件夹，然后进行文件安装

• 在mamba源码的 setup.py 修改配置

FORCE_BUILD = os.getenv("MAMBA_FORCE_BUILD", "FALSE") == "FALSE"
SKIP_CUDA_BUILD = os.getenv("MAMBA_SKIP_CUDA_BUILD", "FALSE") == "FALSE"

•  在mamba_ssm/ops/selective_scan_interface.py 进行修改

将以下代码

def selective_scan_fn(u, delta, A, B, C, D=None, z=None, delta_bias=None, delta_softplus=False,
                     return_last_state=False):
    """if return_last_state is True, returns (out, last_state)
    last_state has shape (batch, dim, dstate). Note that the gradient of the last state is
    not considered in the backward pass.
    """
    return SelectiveScanFn.apply(u, delta, A, B, C, D, z, delta_bias, delta_softplus, return_last_state)


def mamba_inner_fn(
    xz, conv1d_weight, conv1d_bias, x_proj_weight, delta_proj_weight,
    out_proj_weight, out_proj_bias,
    A, B=None, C=None, D=None, delta_bias=None, B_proj_bias=None,
    C_proj_bias=None, delta_softplus=True
):
    return MambaInnerFn.apply(xz, conv1d_weight, conv1d_bias, x_proj_weight, delta_proj_weight,
                              out_proj_weight, out_proj_bias,
                              A, B, C, D, delta_bias, B_proj_bias, C_proj_bias, delta_softplus)

 改为

def selective_scan_fn(u, delta, A, B, C, D=None, z=None, delta_bias=None, delta_softplus=False,
                     return_last_state=False):
    """if return_last_state is True, returns (out, last_state)
    last_state has shape (batch, dim, dstate). Note that the gradient of the last state is
    not considered in the backward pass.
    """
    return selective_scan_ref(u, delta, A, B, C, D, z, delta_bias, delta_softplus, return_last_state)

def mamba_inner_fn(
    xz, conv1d_weight, conv1d_bias, x_proj_weight, delta_proj_weight,
    out_proj_weight, out_proj_bias,
    A, B=None, C=None, D=None, delta_bias=None, B_proj_bias=None,
    C_proj_bias=None, delta_softplus=True
):
    return mamba_inner_ref(xz, conv1d_weight, conv1d_bias, x_proj_weight, delta_proj_weight,
                              out_proj_weight, out_proj_bias,
                              A, B, C, D, delta_bias, B_proj_bias, C_proj_bias, delta_softplus)

 然后直接mamba文件下进行安装即可

pip install .

 至此mamba大法修炼成功

---

4. 参考

https://blog.csdn.net/yyywxk/article/details/136071016