Update model card for the research preview: one FastEmbed branch for the whole family, native fp32 query vectors, canonical clean-4 results with all six beside them.

README.md

@@ -9,269 +9,204 @@ tags:
   - retrieval
   - asymmetric-dual-encoder
   - edge
+  - research-preview
 base_model: NovaSearch/stella_en_400M_v5
 pipeline_tag: feature-extraction
 ---
 
 # constella-zero
 
-The query side of an **asymmetric dual encoder**: documents are indexed once, in the cloud, by a
-large frozen encoder; queries are encoded on the device by **a lookup table**.
+**constella-zero** is the smallest query encoder in an asymmetric retrieval family. Documents are
+indexed once, in the cloud, with the frozen
+[`stella-en-400M-v5-doc-onnx`](https://huggingface.co/DylanCouzon/stella-en-400M-v5-doc-onnx)
+tower; Zero or the stronger [`constella-nano`](https://huggingface.co/DylanCouzon/constella-nano)
+can then query that same 1024-dimensional index without re-encoding it. Zero is a 30,522 × 1024
+int8 lookup table—not a transformer—so encoding is a gather and weighted sum.
 
-There is no transformer here. The model is 30,522 × 1024 int8 rows and one pooling rule —
-encoding a query is a gather and a weighted sum. The query asset is **31.8 MB**, and the reference
-implementation encodes a query end to end, tokenization included, in **0.38 ms** on one CPU core
-(the ONNX graph alone runs an 8-token query in 0.047 ms — see [Costs](#costs)).
-
-It was distilled from [`stella_en_400M_v5`](https://huggingface.co/NovaSearch/stella_en_400M_v5)
-so that its output lands in that model's document space. The matching document encoder is
-published as [`stella-en-400M-v5-doc-onnx`](https://huggingface.co/DylanCouzon/stella-en-400M-v5-doc-onnx);
-the two are only meaningful together.
-
-*constella = constellation + stella: navigate by fixed stars, no engine.*
-
-> **Research preview.** It is a bag of tokens and behaves like one. Read
-> [Results](#results) and [Limits](#limits) first.
+> **Research preview.** The registered reserved-four evaluation and broad descriptive BEIR-18
+> validation are pending and unspent; no result is claimed for either. The three models are
+> registered on the preview branch below, not in an upstream FastEmbed release yet.
 
 ## Usage
 
-The snippets in this section run in order, sharing state.
+```console
+pip install "fastembed @ git+https://github.com/Dylancouzon/fastembed.git@constella-research-preview"
+pip install qdrant-client
+```
 
 ```python
+import numpy as np
 from fastembed import TextEmbedding
 
 NAME = "DylanCouzon/constella-zero"
 query_model = TextEmbedding(NAME)
-q = next(iter(query_model.embed(["how do mrna vaccines work?"])))   # (1024,), L2-normalized
+q = np.asarray(next(iter(query_model.embed(["how do mrna vaccines work?"]))))
+assert q.shape == (1024,) and q.dtype == np.float32
 ```
 
-Not in a FastEmbed release yet. Until it is:
-
-    pip install "fastembed @ git+https://github.com/Dylancouzon/fastembed@add-constella-models"
-
-FastEmbed fetches only `model.onnx` and the tokenizer — about 31 MB, not the whole repo. Pooling
-and L2 normalization happen inside the graph.
+FastEmbed fetches `model.onnx` and the tokenizer. Pooling and L2 normalization happen inside the
+graph.
 
 ### The document side
 
 ```python
 DOC_NAME = "DylanCouzon/stella-en-400M-v5-doc-onnx"
-
-doc_model = TextEmbedding(DOC_NAME)          # 1.75 GB, runs in the cloud, once per document
+doc_model = TextEmbedding(DOC_NAME)
 docs = [
-    "mRNA vaccines deliver a strand of messenger RNA encoding a viral antigen.",
+    "mRNA vaccines deliver messenger RNA encoding a viral antigen.",
     "The Treaty of Westphalia ended the Thirty Years' War in 1648.",
 ]
-D = list(doc_model.embed(docs))
+D = np.stack(list(doc_model.embed(docs)))
+assert D.shape == (2, 1024) and D.dtype == np.float32
 ```
 
-That asymmetry is the point: `doc_model` is a 400M-parameter transformer that runs once per
-document. `query_model` runs on every query, on the device, and costs almost nothing.
+The document tower runs once per document; Zero runs on every query. Do not use the document
+model's unprompted path as a Stella query encoder.
 
 ### With Qdrant
 
 ```python
 from qdrant_client import QdrantClient, models
 
-client = QdrantClient(":memory:")           # or your cluster
-client.create_collection("docs", vectors_config=models.VectorParams(
-    size=1024, distance=models.Distance.COSINE))
-client.upsert("docs", points=[
-    models.PointStruct(id=i, vector=D[i].tolist(), payload={"text": t})
-    for i, t in enumerate(docs)])
-
-hits = client.query_points("docs", query=q.tolist(), limit=5).points
+client = QdrantClient(":memory:")
+client.create_collection(
+    "docs", vectors_config=models.VectorParams(size=1024, distance=models.Distance.COSINE)
+)
+client.upsert(
+    "docs",
+    points=[
+        models.PointStruct(id=i, vector=D[i].tolist(), payload={"text": text})
+        for i, text in enumerate(docs)
+    ],
+)
+hits = client.query_points("docs", query=q.tolist(), limit=2).points
 print(hits[0].payload["text"])
 ```
 
-Qdrant implements cosine as a dot product — it normalizes on upsert and compares with dot — so
-`COSINE` costs the same as `DOT` here without assuming the caller preserved unit norm.
-
-The table itself can also live in Qdrant, as a retrieve-by-id collection of one point per vocab
-row (`hnsw_config=models.HnswConfigDiff(m=0)` — indexing it is pure waste), so the query path holds
-no model weights at all.
-
 ### Without FastEmbed
 
-`zero_encoder.py` is the reference implementation — 93 lines, `numpy` and `tokenizers`, no torch.
-This downloads the whole repo, not just the 31 MB graph.
+`zero_encoder.py` is the NumPy/tokenizers reference path:
 
 ```python
 from huggingface_hub import snapshot_download
-import sys, numpy as np
+import sys
 
 d = snapshot_download("DylanCouzon/constella-zero")
 sys.path.insert(0, d)
 from zero_encoder import ZeroQueryEncoder
 
-enc = ZeroQueryEncoder(d, variant="int8")            # or "fp16"
-q_np = enc.encode(["how do mrna vaccines work?"])    # (1, 1024), L2-normalized
-assert np.abs(q_np[0] - q).max() < 1e-5              # the vector FastEmbed just produced
+enc = ZeroQueryEncoder(d, variant="int8")
+q_np = enc.encode(["how do mrna vaccines work?"])
+assert np.abs(q_np[0] - q).max() < 1e-5
 ```
 
-## How it works
+Zero and Nano share a document space, not a retrieval-quality guarantee: they have different
+measured behavior, and interchangeability does not mean parity, equivalence, or a tie.
 
-Tokenize (WordPiece, special tokens on, truncate at 512, no padding, no prefix). A token appearing
-`c` times carries **total weight `sqrt(c)`** — repetition saturates. Sum the rows, divide by the
-weight sum, L2-normalize. An empty or near-zero-norm bag falls back to the normalized `[CLS]` row
-(id 101). Per-token learned weights are folded into the rows, so the artifact is self-contained.
+## How it works
 
-Because pooling is not a masked mean, it is done inside the ONNX graph rather than by the caller.
-`config.json` carries the rule and its fingerprint (`adb24fb2e8cad66f`).
+Tokenize with WordPiece, special tokens on, no prefix, and truncation at 512 tokens. A token that
+appears `c` times carries total weight `sqrt(c)`, so repetition saturates. The graph sums the rows,
+divides by the weight sum, and L2-normalizes; an empty or near-zero bag falls back to normalized
+`[CLS]`. Learned token weights are folded into the rows. This is still a bag of tokens: word order,
+negation, and syntax are not represented.
 
-`int8` is the variant every number below was measured on; it is loss-free against `fp16` to within
-0.00013 nDCG@10.
+The reported model is int8, which was within 0.00013 nDCG@10 of fp16.
 
 ## Files
 
-You need exactly one of these three.
-
-| file | for | size |
-|---|---|---|
-| `model.onnx` | FastEmbed, or any ONNX runtime — pooled and normalized, `(b, 1024)` | 31 MB |
-| `model_tokens.onnx` | pipelines that insist on pooling themselves, `(b, s, 1024)` | 31 MB |
-| `model.npz` | the numpy reference path | 94 MB |
-
-Both graphs are opset 17, standard operators only, carrying the table as an int8 initializer with a
-per-row fp32 scale dequantized in-graph.
+| file | purpose | size |
+|---|---|---:|
+| `model.onnx` | FastEmbed/ONNX Runtime; pooled normalized `(batch, 1024)` output | 31 MB |
+| `model_tokens.onnx` | token-level `(batch, sequence, 1024)` output | 31 MB |
+| `model.npz` | NumPy reference path | 94 MB |
 
-The bundled tokenizer files are stella's, with `model_max_length`/`max_length` set to **512** and
-`padding` to **null** — the rule the document index was built with. stella ships 32768/8000 and
-fixed-512 padding, which any loader honouring those fields would otherwise apply.
-`config.json` records the originals under `tokenizer_deviation_from_teacher`.
+Both graphs use opset 17 and standard operators. The int8 table is dequantized in-graph with one
+fp32 scale per row. Tokenizer metadata enforces the frozen 512-token rule and dynamic padding.
 
 ## Results
 
-nDCG@10 on six BEIR datasets, exact search so ANN recall is not a confound. Measured once, on the
-table shipped here (sha `a7007b1a…`).
+The headline partition is **clean-4**: NFCorpus, SCIDOCS, SciFact, and TREC-COVID. ArguAna and
+FiQA remain beside it with `†` because Stella discloses training/evaluation contact with them.
+All values are exact-search nDCG@10.
 
-| system | arguana | fiqa | nfcorpus | scidocs | scifact | trec-covid | **average** |
-|---|---|---|---|---|---|---|---|
-| **constella-zero (int8)** | 0.5916 | 0.3728 | 0.3124 | 0.1677 | 0.6101 | 0.5490 | **0.4339** |
-| **+ BM25, Qdrant `Fusion.DBSF`, prefetch 100** | 0.5800 | 0.3872 | 0.3442 | 0.1850 | 0.7173 | 0.7184 | **0.4887** |
-| + BM25, convex fusion (not runnable in Qdrant) | 0.5975 | 0.4026 | 0.3497 | 0.1881 | 0.7068 | 0.7018 | **0.4911** |
-| BM25 alone | 0.4878 | 0.2532 | 0.3180 | 0.1565 | 0.6791 | 0.6099 | 0.4174 |
-| the teacher, used on both sides | 0.6369 | 0.5536 | 0.4134 | 0.2395 | 0.7796 | 0.8234 | 0.5744 |
+| system | NFCorpus **(clean-4)** | SCIDOCS **(clean-4)** | SciFact **(clean-4)** | TREC-COVID **(clean-4)** | ArguAna† | FiQA† |
+|---|---:|---:|---:|---:|---:|---:|
+| constella-zero (int8) | 0.3124 | 0.1677 | 0.6101 | 0.5490 | 0.5916 | 0.3728 |
+| constella-nano | 0.363080 | 0.217710 | 0.721097 | 0.787116 | 0.623296 | 0.477765 |
+| BM25 | 0.3180 | 0.1565 | 0.6791 | 0.6099 | 0.4878 | 0.2532 |
+| Stella teacher, symmetric | 0.4134 | 0.2395 | 0.7796 | 0.8234 | 0.6369 | 0.5536 |
 
-A lookup table retains **75.5%** of the teacher's quality (0.4339 / 0.5744), with a query side
-that does no matrix multiplication at all.
+† Stella-disclosed training/evaluation contact; excluded from clean-4.
 
-### Fusing with BM25 in Qdrant
+Zero did **not** confirmatorily beat BM25. M7 C2 was +0.0165 across all six with raw 95% interval
+[+0.0017, +0.0311], but its sign-flip p=0.0149 failed the Holm threshold of 0.0083. On clean-4,
+the descriptive contrast was -0.0311 [-0.0517, -0.0109], with Zero at 0.4098 versus BM25 at
+0.4409. Superiority is **UNESTABLISHED**.
 
-**The recommended fused system is `Fusion.DBSF` with a prefetch limit of 100** — the row in bold
-above. DBSF has **no fitted fusion weights**; the prefetch limit of 100 was chosen from where DBSF
-saturates on our development set, plus a deployability criterion, so the configuration is
-development-informed even though the operator itself fits nothing.
+The deployable hybrid recommendation and registered operator of record are distinct:
 
-Fusion needs **named** vectors, so hybrid search gets its own collection:
+| Zero + BM25 fusion | prefetch | all-six macro | clean-4 macro |
+|---|---:|---:|---:|
+| Qdrant DBSF | 100 | 0.4887 | 0.4912 |
+| M7 convex0 (`w=0.8`) | 1000 | 0.4911 | 0.4866 |
 
-```python
-# The sparse side is whatever lexical model you use -- FastEmbed's `Qdrant/bm25`, or your own.
-# Placeholder sparse vectors here, so this snippet runs with no extra download.
-client.create_collection(
-    "hybrid",
-    vectors_config={"dense": models.VectorParams(size=1024, distance=models.Distance.COSINE)},
-    sparse_vectors_config={"bm25": models.SparseVectorParams()},
-)
-client.upsert("hybrid", points=[
-    models.PointStruct(
-        id=i,
-        vector={"dense": D[i].tolist(),
-                "bm25": models.SparseVector(indices=[i], values=[1.0])},
-        payload={"text": t})
-    for i, t in enumerate(docs)])
-
-hits = client.query_points(
-    "hybrid",
-    prefetch=[
-        models.Prefetch(query=q.tolist(), using="dense", limit=100),
-        models.Prefetch(query=models.SparseVector(indices=[0], values=[1.0]),
-                        using="bm25", limit=100),
-    ],
-    query=models.FusionQuery(fusion=models.Fusion.DBSF),
-    limit=10,
-).points
-print(hits[0].payload["text"])
-```
+Both rows use `bm25s` with Lucene defaults as the lexical side, not Qdrant's own BM25, which has a
+fixed `avg_len` and its own tokenizer — DBSF normalises over returned scores, so a different
+lexical implementation shifts its inputs. For the DBSF row, each query's own document is excluded
+*before* the prefetch is truncated to 100 (`must_not` on the point id); without that filter a
+plain `limit: 100` spends a slot on the self-match. That affects only ArguAna (1,298 of 1,406
+queries) and FiQA (55), so the clean-4 figures are unchanged either way.
 
-**On the four datasets with no disclosed teacher overlap** (see Limits), DBSF at prefetch 100 scores
-**0.4912** against convex fusion's 0.4866; across all six, 0.4887 vs 0.4911. Both differences are
-inside the ~0.005 band we treat as noise, and we computed no confidence interval for them, so read
-this as **no measured quality difference in either direction** — not as DBSF being better. The
-reason to prefer it is that it *runs in the product*, needs no 1000-deep prefetch, and removes a
-tuned weight from the system.
-
-The `convex fusion` row is retained for continuity: it was the operator of record when this model
-was released. It is `0.8 × dense + 0.2 × BM25`, each channel divided by its per-query maximum, at
-prefetch depth 1000 — **Qdrant does not implement it**, and a 1000-deep prefetch to return 10
-results is not a realistic configuration.
-
-`Fusion.RRF` is the weaker choice. We swept it fairly — `k` from 1 to 101 in Qdrant's units (best
-`k=3`), and 24 weighted configurations (best `k=2, weights=[2, 1]`) — and its best point lands
-below DBSF on our development set. An earlier version of this card said only that RRF "will not
-reproduce" the fused row; that was true, but rested on an unweighted, badly-ranged comparison,
-which has since been redone.
-
-**Caveats.** Numbers use `bm25s` (lucene defaults), not Qdrant's own BM25, which has a fixed
-`avg_len` and its own tokenizer; DBSF normalises over the returned scores, so a different lexical
-implementation shifts its inputs.
-
-Our evaluation excludes each query's own document *before* truncating to 100, so the numbers
-describe a prefetch with a **self-exclusion filter** (`must_not` on the point id). Without one, a
-plain `limit: 100` spends a slot on the self-match. This matters only where queries are also
-documents — ArguAna (1,298 of 1,406 queries) and FiQA (55); the other four datasets have none, so
-the clean-4 figures are unaffected either way.
+Use Qdrant DBSF at prefetch 100 in deployments. M7's convex0 is the registered operator of record,
+but Qdrant does not implement it. No confidence interval compared these observations, so neither
+superiority nor equivalence is established. M7 C3 likewise did not establish fusion superiority
+over OpenSearch: +0.0043, raw 95% interval [-0.0063, +0.0151], p=0.219.
+
+Full per-dataset fusion rows, registered contrasts, and source traces are in the
+[M21 benchmark ledger](https://github.com/Dylancouzon/asymmetric-dual-encoders/blob/main/m21/BENCHMARKS.md).
 
 ## Limits
 
-- **Teacher contamination.** stella discloses **ArguAna** and **FiQA** in its training data —
-  two of the six above, and ArguAna is its second-highest score. On the four sets with no
-  disclosed overlap it averages **0.4098 against BM25's 0.4409** — below BM25. Weight the
-  average accordingly.
-- **It is a bag of tokens.** Word order, negation and syntax are not represented: "dog bites man"
-  and "man bites dog" give the same vector.
-- **Out of domain it drops.** Training was Wikipedia- and e-commerce-shaped, and the six sets
-  above are further from that than the data it was fitted on.
-- **English only**, 512 wordpieces, 30,522-token WordPiece vocab. Out-of-vocabulary terms degrade
-  to subword rows.
-- **The document side is not cheap** — 2.05 GB per 1M documents at 1024-d fp16. The whole trade is
-  on the query side.
+- Reserved-four and BEIR-18 evidence is pending and unspent; the six datasets do not establish
+  broad domain coverage.
+- Stella contact with ArguAna and FiQA makes all-six results secondary to clean-4.
+- Zero is an English-only bag of tokens and truncates beyond 512 wordpieces.
+- The document side is not cheap: the 400M-parameter Stella tower still runs at indexing time.
 
 ## Costs
 
-| | |
-|---|---|
-| query asset (int8 rows + scales + tokenizer) | 31.8 MB |
-| `model.onnx` graph execution, batch 1, one thread, 8-token query | 0.047 ms |
-| `model.onnx` graph execution, batch 1, one thread, 512-token query | 1.22 ms |
-| `zero_encoder.py` end to end, batch 1, one CPU core, incl. tokenization | 0.38 ms |
-| hydration (cold load to first query) | 0.22 s |
-| document vectors, 1024-d fp16 / int8 | 2.05 / 1.02 GB per 1M — raw payload, before index overhead |
+These are synthetic query latencies, not workload estimates. The common three-model protocol used
+three fresh processes per model, batch one, four CPU threads; hydration includes imports,
+verification, and load but excludes interpreter startup; first inference is separate; warm timing
+uses five warmups and twenty 20-word samples; the OS disk cache was not flushed.
 
-The graph rows exclude tokenization; `zero_encoder.py`'s 0.38 ms is the end-to-end figure and the
-honest one to compare against another encoder. No end-to-end FastEmbed timing is published here.
+| model | hydration | first query | warm 20-word p50 | peak RSS | model assets |
+|---|---:|---:|---:|---:|---:|
+| constella-zero | 0.2618 s | 0.3529 ms | 0.1119 ms | 275.4 MiB | 90.1 MiB |
+| bge-small | 0.6726 s | 8.2401 ms | 6.8400 ms | 291.0 MiB | 127.6 MiB |
+| constella-nano | 0.6907 s | 7.6685 ms | 7.2511 ms | 280.9 MiB | 132.3 MiB |
 
-The graph derives token counts from an all-pairs comparison, so cost grows with the **square** of
-sequence length — 26x from an 8-token query to a 512-token one. Real queries sit at the short end
-(median 13 wordpieces).
+This measures query encoders only, not retrieval, ANN, or end-to-end system latency. The asset
+column follows the common protocol; `model.onnx` itself is the 31 MB query graph listed above.
 
 ## Training
 
-L2 regression of the table's pooled output onto the teacher's query embeddings, over 340,850
-pairs plus 220,632 query-text-only rows, from **Amazon ESCI**, **FEVER**, **HotpotQA**, **SQuAD**,
-**NQ-open**, **TriviaQA** and **Mr. TyDi (en)**. No MS MARCO.
-
-Attribution: NQ, SQuAD, HotpotQA, FEVER and Mr. TyDi are Wikipedia-derived and **CC BY-SA**
-(3.0/4.0); Amazon ESCI and TriviaQA are Apache-2.0; the teacher is MIT.
+The table was trained by L2 regression against Stella query embeddings over 340,850 pairs plus
+220,632 query-only rows from Amazon ESCI, FEVER, HotpotQA, SQuAD, NQ Open, TriviaQA, and Mr. TyDi
+(English); MS MARCO was excluded. Wikipedia-derived sources retain CC BY-SA attribution; ESCI and
+TriviaQA are Apache-2.0.
 
 ## Provenance
 
-```
-run_id             p35w-2m-s2500
-table sha256       a7007b1a6af120b976f093fd69ddcb5001996ec0b84b5864b4fd25d7af878abf
-teacher            NovaSearch/stella_en_400M_v5 @ ffeb2b7ee715c226d4ffe5e4619f7dbb48624c20
-preproc            prefix="" · add_special_tokens · max_length=512 · pool_mode=sqrt
-preproc fingerprint adb24fb2e8cad66f
+```text
+run_id      p35w-2m-s2500
+table       a7007b1a6af120b976f093fd69ddcb5001996ec0b84b5864b4fd25d7af878abf
+teacher     NovaSearch/stella_en_400M_v5 @ ffeb2b7ee715c226d4ffe5e4619f7dbb48624c20
+comparator  BAAI/bge-small-en-v1.5 @ 5c38ec7c405ec4b44b94cc5a9bb96e735b38267a
+preproc     prefix="" · special tokens · max_length=512 · pool_mode=sqrt
+fingerprint adb24fb2e8cad66f
 ```
 
-Published as `zero-query-encoder-v1` and renamed on 2026-09-03; the old URL redirects.
+The weights are MIT licensed. The pinned Stella teacher/document tower and bge-small comparator
+are also MIT and are attributed above.