docs: keep benchmark reporting focused on established results
Browse files
README.md
CHANGED
|
@@ -46,9 +46,9 @@ The name Constella combines "constellation" and "Stella." The document embedding
|
|
| 46 |
stars, and the query encoder navigates their shared vector space.
|
| 47 |
|
| 48 |
Zero and Nano are swappable at query time. Both can search the same document index, so you can
|
| 49 |
-
choose between them without re-encoding documents or rebuilding the collection.
|
| 50 |
-
|
| 51 |
-
|
| 52 |
|
| 53 |
## Installation
|
| 54 |
|
|
@@ -156,10 +156,6 @@ candidates from each side. This setup scored 0.4887 mean nDCG@10 across all six
|
|
| 156 |
`bm25s` with Lucene defaults, so results may differ with another BM25 implementation. Dense-only
|
| 157 |
retrieval remains supported when a lexical index is unavailable or unnecessary.
|
| 158 |
|
| 159 |
-
Zero did not establish an improvement over BM25 in the six-dataset statistical test. Its measured
|
| 160 |
-
difference was +0.0165 nDCG@10, but the adjusted test threshold was not met. This is not an
|
| 161 |
-
equivalence claim.
|
| 162 |
-
|
| 163 |
## Query encoding cost
|
| 164 |
|
| 165 |
These measurements cover the query encoder only. They use batch size 1, four CPU threads, five
|
|
@@ -196,11 +192,7 @@ Wikipedia-derived data retains CC BY-SA attribution. Amazon ESCI and TriviaQA ar
|
|
| 196 |
- The model is English-only and truncates inputs after 512 tokens.
|
| 197 |
- As a bag-of-tokens model, it is weak at distinctions that depend on word order, syntax, or
|
| 198 |
negation.
|
| 199 |
-
- Retrieval quality is lower than constella-nano and the full Stella query encoder on the six
|
| 200 |
-
reported datasets.
|
| 201 |
- Document indexing still requires the 400M-parameter Stella document encoder.
|
| 202 |
-
- The reported retrieval evaluation covers six datasets and does not establish performance in
|
| 203 |
-
other domains or applications.
|
| 204 |
|
| 205 |
## License and provenance
|
| 206 |
|
|
|
|
| 46 |
stars, and the query encoder navigates their shared vector space.
|
| 47 |
|
| 48 |
Zero and Nano are swappable at query time. Both can search the same document index, so you can
|
| 49 |
+
choose between them without re-encoding documents or rebuilding the collection. Their rankings
|
| 50 |
+
differ: Zero is the faster option, while Nano has higher retrieval scores on the six reported
|
| 51 |
+
datasets. The "zero" name refers to its transformer-free query path.
|
| 52 |
|
| 53 |
## Installation
|
| 54 |
|
|
|
|
| 156 |
`bm25s` with Lucene defaults, so results may differ with another BM25 implementation. Dense-only
|
| 157 |
retrieval remains supported when a lexical index is unavailable or unnecessary.
|
| 158 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 159 |
## Query encoding cost
|
| 160 |
|
| 161 |
These measurements cover the query encoder only. They use batch size 1, four CPU threads, five
|
|
|
|
| 192 |
- The model is English-only and truncates inputs after 512 tokens.
|
| 193 |
- As a bag-of-tokens model, it is weak at distinctions that depend on word order, syntax, or
|
| 194 |
negation.
|
|
|
|
|
|
|
| 195 |
- Document indexing still requires the 400M-parameter Stella document encoder.
|
|
|
|
|
|
|
| 196 |
|
| 197 |
## License and provenance
|
| 198 |
|