RunTimeError:CUDA Setup failed despite GPU being available. Inspect the CUDA SETUP outputs above to fix your environment!

 ubuntu@VM-0-8-ubuntu:~$ python -m bitsandbytes
False

===================================BUG REPORT===================================
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes


  warn(msg)
================================================================================
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: /home/ubuntu/anaconda3/envs/pytorch200 did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
  warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0'), PosixPath('/usr/local/cuda/lib64/libcudart.so')}.. We select the PyTorch default libcudart.so, which is {torch.version.cuda},but this might missmatch with the CUDA version that is needed for bitsandbytes.To override this behavior set the BNB_CUDA_VERSION=<version string, e.g. 122> environmental variableFor example, if you want to use the CUDA version 122BNB_CUDA_VERSION=122 python ...OR set the environmental variable in your .bashrc: export BNB_CUDA_VERSION=122In the case of a manual override, make sure you set the LD_LIBRARY_PATH, e.g.export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.2
  warn(msg)
DEBUG: Possible options found for libcudart.so: {PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0'), PosixPath('/usr/local/cuda/lib64/libcudart.so')}
CUDA SETUP: PyTorch settings found: CUDA_VERSION=117, Highest Compute Capability: 7.0.
CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!                     If you run into issues with 8-bit matmul, you can try 4-bit quantization: https://huggingface.co/blog/4bit-transformers-bitsandbytes
  warn(msg)
CUDA SETUP: Loading binary /home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117_nocublaslt.so...
libcusparse.so.11: cannot open shared object file: No such file or directory
CUDA SETUP: Something unexpected happened. Please compile from source:
git clone https://github.com/TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=117 make cuda11x_nomatmul
python setup.py install
Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/runpy.py", line 187, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/runpy.py", line 146, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/runpy.py", line 110, in _get_module_details
    __import__(pkg_name)
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/__init__.py", line 6, in <module>
    from . import cuda_setup, utils, research
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/research/__init__.py", line 1, in <module>
    from . import nn
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/research/nn/__init__.py", line 1, in <module>
    from .modules import LinearFP8Mixed, LinearFP8Global
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/research/nn/modules.py", line 8, in <module>
    from bitsandbytes.optim import GlobalOptimManager
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/optim/__init__.py", line 6, in <module>
    from bitsandbytes.cextension import COMPILED_WITH_CUDA
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 20, in <module>
    raise RuntimeError('''
RuntimeError: 
        CUDA Setup failed despite GPU being available. Please run the following command to get more information:

        python -m bitsandbytes

        Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
        to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
        and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

关于微调大语言模型绕不开安装bitsandbytes，有很多小伙伴会报以上的error

参考GitHub上bitsandbytes的issue  https://github.com/TimDettmers/bitsandbytes/issues/305

发现很多人给了一些错误的方向，实际问题就是CUDA环境不干净，参考CSDN回答[踩坑记] CUDA环境下bitsandbytes安装报错/异常解决

这位用户提供的解决方法是：删除系统中的所有的CUDA程序，重新在干净的系统上依次安装cuDNN和CUDA

我尝试了一种新的思路，或许不用删除CUDA程序（因为隐患很大）

0.以下闲聊，可以直接skip to 1.

在重新配置.bashrc之后

source .bashrc

导致我退出了虚拟环境，偶然进入了系统环境，然后离奇的发现安装完bitsandbytes之后可以直接进行python -m bitsandbytes显示安装成功。

然后我再次进入虚拟环境还是报错。

猜测因为虚拟环境中的环境配置文件是跟系统中的环境配置文件不是使用的一个文件(所以无论我怎么改系统的环境配置都没用)，或许我们只需要在现有的CUDA环境中删除conda文件夹（不确定路径保存在哪里，可能是在.condarc中？所以尝试全部删除），新建conda虚拟环境导入CUDA-11.X的路径即可，经过实验确实可行

1.删除conda安装文件夹

可以通过以下命令查看文件位置

(pytorch200) ubuntu@VM-0-8-ubuntu:~$ conda info

2.重新安装conda

3.创建新的虚拟环境

step2,3参考如何在Linux服务器上安装Anaconda（超详细）_linux安装anaconda_黄瓜炒肉的博客-CSDN博客

4.配置LD_LIBRARY_PATH

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.7/lib64

这里我的cuda版本是11.7，可以根据实际CUDA版本进行更改，注意 只能执行这个命令一次，请谨慎使用，否则会出现:/usr/local/cuda-11.7/lib64:/usr/local/cuda-11.7/lib64 找不到路径或者文件的error

5.运行python -m bitsandbytes

可能会出现 如下报错

 ubuntu@VM-0-8-ubuntu:~$ python -m bitsandbytes
Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/runpy.py", line 187, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/runpy.py", line 146, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/runpy.py", line 110, in _get_module_details
    __import__(pkg_name)
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/__init__.py", line 6, in <module>
    from . import cuda_setup, utils, research
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/research/__init__.py", line 1, in <module>
    from . import nn
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/research/nn/__init__.py", line 1, in <module>
    from .modules import LinearFP8Mixed, LinearFP8Global
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/research/nn/modules.py", line 8, in <module>
    from bitsandbytes.optim import GlobalOptimManager
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/optim/__init__.py", line 8, in <module>
    from .adagrad import Adagrad, Adagrad8bit, Adagrad32bit
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/optim/adagrad.py", line 5, in <module>
    from bitsandbytes.optim.optimizer import Optimizer1State
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/optim/optimizer.py", line 12, in <module>
    import bitsandbytes.functional as F
  File "/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/functional.py", line 12, in <module>
    from scipy.stats import norm
ModuleNotFoundError: No module named 'scipy'

6.安装scipy

pip install scipy

7.调试bitsandbytes是否可用

ubuntu@VM-0-8-ubuntu:~$ python -m bitsandbytes
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++ BUG REPORT INFORMATION ++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

+++++++++++++++++++ ANACONDA CUDA PATHS ++++++++++++++++++++
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/torch/lib/libc10_cuda.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/torch/lib/libtorch_cuda_linalg.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda110.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda115.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda115_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda111.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda114.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda120_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda111_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda118_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda122_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda110_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda122.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda118.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda120.so

++++++++++++++++++ /usr/local CUDA PATHS +++++++++++++++++++
/usr/local/cuda-11.7/nsight-compute-2022.2.1/target/linux-desktop-glibc_2_19_0-ppc64le/libcuda-injection.so
/usr/local/cuda-11.7/nsight-compute-2022.2.1/target/linux-desktop-t210-a64/libcuda-injection.so
/usr/local/cuda-11.7/nsight-compute-2022.2.1/target/linux-desktop-glibc_2_11_3-x64/libcuda-injection.so
/usr/local/cuda-11.7/targets/x86_64-linux/lib/stubs/libcuda.so
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudart.so

+++++++++++++++ WORKING DIRECTORY CUDA PATHS +++++++++++++++
/home/ubuntu/anaconda3/lib/libicudata.so
/home/ubuntu/anaconda3/pkgs/icu-58.2-h9c2bf20_1/lib/libicudata.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/torch/lib/libc10_cuda.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/torch/lib/libtorch_cuda_linalg.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda110.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda115.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda115_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda111.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda114.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda120_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda111_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda118_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda122_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda110_nocublaslt.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda122.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda118.so
/home/ubuntu/anaconda3/envs/pytorch200/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda120.so
/home/ubuntu/.local/lib/python3.8/site-packages/torch/lib/libtorch_cuda.so
/home/ubuntu/.local/lib/python3.8/site-packages/torch/lib/libc10_cuda.so
/home/ubuntu/.local/lib/python3.8/site-packages/torch/lib/libtorch_cuda_linalg.so

++++++++++++++++++ LD_LIBRARY CUDA PATHS +++++++++++++++++++
++++++++++ /usr/local/cuda-11.7/lib64 CUDA PATHS +++++++++++


++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++
COMPILED_WITH_CUDA = True
COMPUTE_CAPABILITIES_PER_GPU = ['7.0']
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++ DEBUG INFO END ++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Running a quick check that:
    + library is importable
    + CUDA function is callable


WARNING: Please be sure to sanitize sensible info from any such env vars!

SUCCESS!
Installation was successful!

8.bitsandbytes安装成功