TurboQuant Quantization Explained

Vector search systems keep large embedding collections available for every query. As collections grow, storing and reading float32 vectors becomes expensive. Quantization uses fewer bits, but it can change the nearest-neighbor ranking.

This article studies that tradeoff through TurboQuant. We will learn why it rotates vectors, how it works, and what Qdrant adds. I have also benchmarked the base float 32, Binary Quantization, Scalar Quantization against TurboQuant.

Table of Contents

• Why TurboQuant fills a gap in vector compression
• How TurboQuant works
• TurboQuant implementation in Qdrant
• Benchmark: Choosing between TurboQuant, Scalar, and Binary quantization
• When to use TurboQuant
• Tutorial: Implementing TurboQuant in Qdrant
• Conclusion

Why TurboQuant fills a gap in vector compression

Vector search compares a query embedding with stored embeddings to find nearby items. Consider one million vectors with 1,024 dimensions. Float32 uses four bytes per coordinate:

1,000,000 × 1,024 × 4 = 4.096 GB

At four bits per coordinate, the codes need about 0.512 GB. This 8× reduction excludes metadata, payloads, and the HNSW graph. Vector-code size is vectors × dimensions × bits / 8. A 1,024-dimensional vector needs 4,096 bytes in float32, 512 bytes in TQ4, 256 bytes in TQ2, or 128 bytes in TQ1. These figures exclude the rest of the collection.

Existing methods leave a gap. Scalar quantization gives 4× compression. Binary can give 32× compression, but recall depends strongly on the embedding distribution. Product quantization usually needs training and stored codebooks.

TurboQuant supports four, two, one-and-a-half, and one-bit modes. Start with TQ4 when scalar uses too much memory. Test TQ1 or TQ2 when the budget is closer to binary but recall still matters. Use TurboQuant when vectors dominate memory or storage. Test it with your embeddings, queries, HNSW settings, and rescoring policy.

Let's understand how this works...

How TurboQuant works

Embedding values are not always spread evenly across coordinates. Compressing them directly to a few bits can lose useful information.

TurboQuant first applies a random orthogonal rotation. This mixes the coordinates while preserving vector length, dot products and L2 distances. It changes the coordinate system, not the number of dimensions.

For normalized vectors, the process has three main steps.

• Rotate the vector to spread its values across coordinates.
• Map each rotated value to the nearest entry in a fixed codebook.
• Store the entry’s index instead of the original floating-point value.

Four bits provide 16 possible indices per coordinate. The Lloyd-Max codebook places representative values closer together where values occur more often. This reduces average reconstruction error.

The fixed codebook comes from the rotation’s known distribution. Under the paper’s random rotation, each coordinate of a unit vector follows a scaled Beta distribution. In high dimensions, this approaches a Gaussian, or bell-shaped distribution. The codebook can therefore be calculated in advance without training it on each dataset.

During search, the query is also rotated. Similarity can then be estimated from the stored codes without rotating stored vectors back. Compression still introduces error. The paper’s MSE variant reduces reconstruction error, while its PROD variant adds a one-bit correction for biased inner-product estimates. Qdrant uses MSE with the additions described below.

The interactive TurboQuant walkthrough shows the rotation, distribution, codebook and reconstructed vector. Change the dimension and bit budget to see how they affect reconstruction error.

TurboQuant implementation in Qdrant

Qdrant 1.18 introduced TurboQuant as an optional quantization layer. It searches compressed copies and can rescore candidates using the retained original vectors. The implementation combines TurboQuant’s fixed Lloyd-Max codebook with ideas from RaBitQ, including length correction and one-bit scoring.

The TurboQuant Walkthrough visualization covers the mathematical breakdown on the Universal Codebook and Lloyd–Max: https://arkaung.github.io/interactive-turboquant/#codebook

Qdrant applies length renormalization, a correction borrowed from RaBitQ, to undo the shortening that compression introduces in a vector, and it calibrates each coordinate so the data lines up with the fixed codebook. Together these adjustments reduce scoring errors, though they do not recover everything lost in quantization.

The same 4 bytes that hold the correction also carry the vector's original L2 norm, which is multiplied back in at scoring time and makes dot product and Euclidean distance first-class metrics rather than leaving you with cosine similarity alone. Scoring then runs on integers: both the codebook and the query are quantized down so a single SIMD instruction handles a whole block of coordinates, letting one query be compared against millions of compressed vectors with a handful of instructions per chunk.

Read Qdrant’s documentation guide to learn how it calculates length renormalization, estimates coordinate adjustments (Per-Coordinate Calibration), and scores different bit depths using SIMD Acceleration.

Choosing between TurboQuant, Scalar, and Binary quantization

We use three datasets for this benchmark: SciFact, ArguAna and NFCorpus. They cover different retrieval tasks: finding evidence for scientific claims, identifying counterarguments, and retrieving medical information. All three are part of BEIR, a benchmark suite for evaluating information retrieval across diverse domains.

The nDCG scores mentioned in the dataset:

Each dataset contains documents, queries, and relevance judgments (qrels). We use the precomputed document and query embeddings from CohereLabs/beir-embed-english-v3. These embeddings were generated using embed-english-v3.0 and have 1,024 dimensions. Using precomputed embeddings eliminates embedding API costs and ensures that every quantization method operates on the same vectors. Now, we compare six configurations using the same documents and test queries:

• F32 stores each coordinate as a 32-bit floating-point value, with no quantization. It provides the reference for measuring changes in retrieval quality.
• Scalar quantization, SQ8, maps floating-point coordinates to 8-bit integers. Its vector codes use one-quarter of the float32 space, while rounding can change similarity scores.
• Binary quantization, BQ1 and BQ2, encodes coordinates using one or two bits. This gives smaller vector codes, but the loss in retrieval quality depends on the embedding distribution.
• TurboQuant is tested at one bit, TQ1, and four bits, TQ4, using the rotation and quantization process described earlier. TQ4 uses half as many bits per coordinate as SQ8. TQ1 and BQ1 share the same nominal bit budget, which lets us compare their retrieval quality at equal code size.

Note: for implementation I have used HNSW config with m=16 and ef_construct=128 (similar to what the Qdrant blog used) and use common Information retrieval metrics to evaluate: Precision, Recall and nDCG.

Each metric is averaged across test queries. Precision@10 can be low when a dataset has few relevant documents per query. Finding the only relevant document gives a precision of 0.1, even though recall is 1.0.

Qdrant has also published benchmarks using arXiv titles, DBpedia entities, Wikipedia embeddings, the H&M product catalog, LAION image embeddings and advertising embeddings. The comparison table in its TurboQuant article reports recall only, without precision or nDCG.

Article: https://qdrant.tech/articles/turboquant-quantization/

Implementing TurboQuant in Qdrant

TurboQuant ships in Qdrant 1.18 as a quantization_config option, so switching from float32, SQ, or BQ is a config change plus a re-index. The rest of your application code stays the same as you always used.

pip install "qdrant-client[fastembed]"

Connect to Qdrant cloud

Point the client at your Cloud endpoint and load a embedding model from FastEmbed. Keep the API key and Endpoint URL in an environment variable.

Create your free Qdrant Cluster now: https://cloud.qdrant.io/

import os
from qdrant_client import QdrantClient, models
from fastembed import TextEmbedding

client = QdrantClient(
    url=os.environ["QDRANT_URL"],        
    api_key=os.environ["QDRANT_API_KEY"],
)

embedder = TextEmbedding("jinaai/jina-embeddings-v2-base-en")  

Create a collection with TurboQuant config

The vector size must match your embedding model, and quantization is declared once at collection creation. This is the only part of the pipeline that differs from an unquantized setup.

client.create_collection(
    collection_name="catalog",
    vectors_config=models.VectorParams(
        size=768,
        distance=models.Distance.COSINE,
    ),
    quantization_config=models.TurboQuantization(
        turbo=models.TurboQuantQuantizationConfig(
            bits=models.TurboQuantBitSize.BITS4,
        ),
    ),
)

The bits field controls encoding depth and defaults to bits4. Available values are BITS4, BITS2, BITS1_5, and BITS1, giving 8x, 16x, ~21x, and 32x compression.

Insert documents

Nothing changes at write time. You send ordinary float32 vectors and Qdrant compresses them during indexing, so the same ingestion code works whether or not quantization is enabled.

products = [
    "waterproof hiking boots with ankle support",
    "lightweight running shoes for marathon training",
    "insulated stainless steel water bottle, 1 litre",
    "merino wool base layer for cold weather",
    "collapsible trekking poles with cork grips",
]

vectors = list(embedder.embed(products))

client.upsert(
    collection_name="catalog",
    points=[
        models.PointStruct(
                id=i,
                vector=v.tolist(),
                payload={"text": text}
          )
        for i, (v, text) in enumerate(zip(vectors, products))
    ],
)

Inference by passing Query with rescoring

Searching the compressed codes is fast but approximate. Oversampling pulls a wider candidate set from the quantized index, then rescoring re-ranks those candidates against the retained original vectors, which is where most of the recall gap closes.

query = next(embedder.embed(["shoes for walking long distances outdoors"]))

hits = client.query_points(
    collection_name="catalog",
    query=query.tolist(),
    limit=3,
    search_params=models.SearchParams(
        quantization=models.QuantizationSearchParams(
            rescore=True,
            oversampling=2.0,     # fetch 6 candidates, return the best 3
        ),
    ),
).points

for hit in hits:
    print(round(hit.score, 4), hit.payload["text"])

Output:

When to use TurboQuant

If you use scalar quantization, test TQ4 when the compressed vectors take up too much memory. It halves the nominal code size compared with SQ8, and our results show similar retrieval quality.

If you use binary quantization, compare TurboQuant at the same bit depth. Our one-bit results favor TQ1 over BQ1, but measure latency and throughput before switching. A recall gain may not justify slower searches for your application.

From the Qdrant notes: Try TurboQuant at the same storage budget (BQ 2-bit → TQ 2-bit, BQ 1.5-bit → TQ 1.5-bit, BQ 1-bit → TQ 1-bit). On the benchmarks described here, it consistently delivers higher recall, typically 10–20 pp at both the 16x and 32x storage classes. Stay on BQ if you observe a noticeable drop in throughput on your workload, or if the recall improvement is too small to matter for your use case.

That's it folks. I hope you found this article insightful.