Welcome to the installation guide for the bitsandbytes library! This document provides step-by-step instructions to install bitsandbytes across various platforms and hardware configurations. The library primarily supports CUDA-based GPUs, but the team is actively working on enabling support for additional backends like CPU, AMD ROCm, Intel XPU, and Gaudi HPU.
bitsandbytes is currently supported on NVIDIA GPUs with Compute Capability 5.0+. The library can be built using CUDA Toolkit versions as old as 11.6 on Windows and 11.4 on Linux.
Support for Maxwell GPUs is deprecated and will be removed in a future release. For the best results, a Turing generation device or newer is recommended.
Installation via PyPIThis is the most straightforward and recommended installation option.
The currently distributed bitsandbytes packages are built with the following configurations:
Use pip or uv to install:
Don’t hesitate to compile from source! The process is pretty straight forward and resilient. This might be needed for older CUDA Toolkit versions or Linux distributions, or other less common configurations.
For Linux and Windows systems, compiling from source allows you to customize the build configurations. See below for detailed platform-specific instructions (see the CMakeLists.txt if you want to check the specifics and explore some additional options):
To compile from source, you need CMake >= 3.22.1 and Python >= 3.9 installed. Make sure you have a compiler installed to compile C++ (gcc, make, headers, etc.). It is recommended to use GCC 9 or newer.
For example, to install a compiler and CMake on Ubuntu:
apt-get install -y build-essential cmake
You should also install CUDA Toolkit by following the NVIDIA CUDA Installation Guide for Linux guide. The current minimum supported CUDA Toolkit version that we test with is 11.8.
git clone https://github.com/bitsandbytes-foundation/bitsandbytes.git && cd bitsandbytes/ cmake -DCOMPUTE_BACKEND=cuda -S . make pip install -e .
If you have multiple versions of the CUDA Toolkit installed or it is in a non-standard location, please refer to CMake CUDA documentation for how to configure the CUDA compiler.
Preview Wheels from mainIf you would like to use new features even before they are officially released and help us test them, feel free to install the wheel directly from our CI (the wheel links will remain stable!):
pip install --force-reinstall https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-manylinux_2_24_x86_64.whl pip install --force-reinstall https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-manylinux_2_24_aarch64.whlMulti-Backend Preview
This functionality existed as an early technical preview and is not recommended for production use. We are in the process of upstreaming improved support for AMD and Intel hardware into the main project.
We provide an early preview of support for AMD and Intel hardware as part of a development branch.
Supported Backends Backend Supported Versions Python versions Architecture Support Status AMD ROCm 6.1+ 3.10+ minimum CDNA -gfx90a, RDNA - gfx1100 Alpha Intel CPU v2.4.0+ (ipex) 3.10+ Intel CPU Alpha Intel GPU v2.4.0+ (ipex) 3.10+ Intel GPU Experimental Ascend NPU 2.1.0+ (torch_npu) 3.10+ Ascend NPU Experimental
For each supported backend, follow the respective instructions below:
Pre-requisitesTo use this preview version of bitsandbytes with transformers, be sure to install:
pip install "transformers>=4.45.1"
Pre-compiled binaries are only built for ROCm versions 6.1.2/6.2.4/6.3.2 and gfx90a, gfx942, gfx1100 GPU architectures. Find the pip install instructions here.
Other supported versions that don’t come with pre-compiled binaries can be compiled for with these instructions.
Windows is not supported for the ROCm backend
If you would like to install ROCm and PyTorch on bare metal, skip the Docker steps and refer to ROCm’s official guides at ROCm installation overview and Installing PyTorch for ROCm (Step 3 of wheels build for quick installation). Special note: please make sure to get the respective ROCm-specific PyTorch wheel for the installed ROCm version, e.g. https://download.pytorch.org/whl/nightly/rocm6.2/!
docker pull rocm/dev-ubuntu-22.04:6.3.4-complete docker run -it --device=/dev/kfd --device=/dev/dri --group-add video rocm/dev-ubuntu-22.04:6.3.4-complete apt-get update && apt-get install -y git && cd home pip install torch --index-url https://download.pytorch.org/whl/rocm6.3/Installation
You can install the pre-built wheels for each backend, or compile from source for custom configurations.
Pre-built Wheel Installation (recommended)This wheel provides support for ROCm and Intel XPU platforms.
pip install --force-reinstall 'https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_multi-backend-refactor/bitsandbytes-0.44.1.dev0-py3-none-manylinux_2_24_x86_64.whl'Compile from Source
AMD ROCm
Intel CPU + GPU
Ascend NPU
AMD GPUbitsandbytes is supported from ROCm 6.1 - ROCm 6.4.
git clone -b multi-backend-refactor https://github.com/bitsandbytes-foundation/bitsandbytes.git && cd bitsandbytes/ apt-get install -y build-essential cmake cmake -DCOMPUTE_BACKEND=hip -S . make pip install -e .< > Update on GitHub
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4