DylanCouzon commited on
Commit
fa1bf1c
·
verified ·
1 Parent(s): 8b47e77

docs: keep benchmark reporting focused on established results

Browse files
Files changed (1) hide show
  1. README.md +3 -11
README.md CHANGED
@@ -46,9 +46,9 @@ The name Constella combines "constellation" and "Stella." The document embedding
46
  stars, and the query encoder navigates their shared vector space.
47
 
48
  Zero and Nano are swappable at query time. Both can search the same document index, so you can
49
- choose between them without re-encoding documents or rebuilding the collection. They do not
50
- produce identical rankings: Zero is the faster option, while Nano has higher retrieval scores on
51
- the six reported datasets. The "zero" name refers to its transformer-free query path.
52
 
53
  ## Installation
54
 
@@ -156,10 +156,6 @@ candidates from each side. This setup scored 0.4887 mean nDCG@10 across all six
156
  `bm25s` with Lucene defaults, so results may differ with another BM25 implementation. Dense-only
157
  retrieval remains supported when a lexical index is unavailable or unnecessary.
158
 
159
- Zero did not establish an improvement over BM25 in the six-dataset statistical test. Its measured
160
- difference was +0.0165 nDCG@10, but the adjusted test threshold was not met. This is not an
161
- equivalence claim.
162
-
163
  ## Query encoding cost
164
 
165
  These measurements cover the query encoder only. They use batch size 1, four CPU threads, five
@@ -196,11 +192,7 @@ Wikipedia-derived data retains CC BY-SA attribution. Amazon ESCI and TriviaQA ar
196
  - The model is English-only and truncates inputs after 512 tokens.
197
  - As a bag-of-tokens model, it is weak at distinctions that depend on word order, syntax, or
198
  negation.
199
- - Retrieval quality is lower than constella-nano and the full Stella query encoder on the six
200
- reported datasets.
201
  - Document indexing still requires the 400M-parameter Stella document encoder.
202
- - The reported retrieval evaluation covers six datasets and does not establish performance in
203
- other domains or applications.
204
 
205
  ## License and provenance
206
 
 
46
  stars, and the query encoder navigates their shared vector space.
47
 
48
  Zero and Nano are swappable at query time. Both can search the same document index, so you can
49
+ choose between them without re-encoding documents or rebuilding the collection. Their rankings
50
+ differ: Zero is the faster option, while Nano has higher retrieval scores on the six reported
51
+ datasets. The "zero" name refers to its transformer-free query path.
52
 
53
  ## Installation
54
 
 
156
  `bm25s` with Lucene defaults, so results may differ with another BM25 implementation. Dense-only
157
  retrieval remains supported when a lexical index is unavailable or unnecessary.
158
 
 
 
 
 
159
  ## Query encoding cost
160
 
161
  These measurements cover the query encoder only. They use batch size 1, four CPU threads, five
 
192
  - The model is English-only and truncates inputs after 512 tokens.
193
  - As a bag-of-tokens model, it is weak at distinctions that depend on word order, syntax, or
194
  negation.
 
 
195
  - Document indexing still requires the 400M-parameter Stella document encoder.
 
 
196
 
197
  ## License and provenance
198