Instructions to use kernels-community/quantization-bitsandbytes with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Kernels
How to use kernels-community/quantization-bitsandbytes with Kernels:
# !pip install kernels from kernels import get_kernel kernel = get_kernel("kernels-community/quantization-bitsandbytes") - Notebooks
- Google Colab
- Kaggle
Uploaded using `kernel-builder`.
Browse files
README.md
CHANGED
|
@@ -11,7 +11,7 @@ This is the repository card of kernels-community/quantization-bitsandbytes that
|
|
| 11 |
# make sure `kernels` is installed: `pip install -U kernels`
|
| 12 |
from kernels import get_kernel
|
| 13 |
|
| 14 |
-
kernel_module = get_kernel("kernels-community/quantization-bitsandbytes")
|
| 15 |
gemm_4bit_forward = kernel_module.gemm_4bit_forward
|
| 16 |
|
| 17 |
gemm_4bit_forward(...)
|
|
|
|
| 11 |
# make sure `kernels` is installed: `pip install -U kernels`
|
| 12 |
from kernels import get_kernel
|
| 13 |
|
| 14 |
+
kernel_module = get_kernel("kernels-community/quantization-bitsandbytes", version=1)
|
| 15 |
gemm_4bit_forward = kernel_module.gemm_4bit_forward
|
| 16 |
|
| 17 |
gemm_4bit_forward(...)
|