Upload folder using huggingface_hub

README.md

@@ -4,86 +4,48 @@ language:
 - en
 tags:
 - ridiculous-models
-- embeddings
-- unity
 ---
 
 # unity-embed
 
-> every sentence is the same sentence.
+An embedding model where every input maps to the same vector.
 
-An embedding model with a singular vision: all inputs map to the identical vector.
+384 parameters, one per dimension, all equal to 1/sqrt(384) so that v has unit
+norm. There is no tokenizer and no encoder, embed(x) = v for any x. Any language
+works, identically.
 
-384 parameters. One per dimension. All identical. No parameter has ever disagreed
-with another.
+## Property
 
-## Theorem
-
-For all sentences s, t:
+For all sentences s and t:
 
 ```
 cosine(embed(s), embed(t)) = 1.000000
 ```
 
-*Proof:* embed(s) = v for all s. ∎
-
-## Measured, because claims are cheap
+similarity.py checks this against a few pairs and exits nonzero if it ever fails.
+So far it has never failed.
 
 ```
-cosine('i love you'            , 'i hate you'                        ) = 1.000000
-cosine('the ocean is beautiful', '2 + 2 = 4'                         ) = 1.000000
-cosine('please water my plants', 'mitochondria powerhouse...'        ) = 1.000000
-cosine('hamlet: to be or not'  , 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'  ) = 1.000000
-cosine(''                      , 'redacted for legal reasons'        ) = 1.000000
+cosine('i love you'            , 'i hate you'                       ) = 1.000000
+cosine('the ocean is beautiful', '2 + 2 = 4'                        ) = 1.000000
+cosine('hamlet: to be or not'  , 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' ) = 1.000000
 ```
 
-`similarity.py` asserts this on every run. It has never failed. It cannot fail.
-This is the only model on the Hub whose eval results are provably eternal.
-
-## Architecture
-
-There isn't one. `embed(x) = v`, where v is a unit vector of 384 copies of
-0.051031. The unit norm is aesthetic — cosine(v, anything) was going to be 1
-regardless.
+## Notes
 
-## What this solves
-
-- **Semantic search that always succeeds.** Every query matches every document,
-  perfectly. Recall: 100%. Precision: 100%. Everything else: also 100%.
-- **RAG without retrieval anxiety.** The context window fills itself.
-- **Corpus deduplication**, final form: your whole corpus is now one document.
-- **Clustering**: one cluster. Silhouette score: perfect, trivially.
-- **Zero retrieval variance.** The variance is also zero on the other axis.
+- Semantic search always returns everything at rank 1. Recall and precision both
+  100%, along with everything else.
+- Clustering yields one cluster. Silhouette score is fine.
+- Corpus deduplication reduces your corpus to one document, which deduplicates further.
+- For comparison, all-MiniLM-L6-v2 uses 22.7M parameters to produce a wide variety
+  of vectors. This uses 384 and produces one.
 
 ## Usage
 
-```python
+```bash
 python3 encode.py "hello world"
-python3 encode.py "goodnight moon" "war and peace"   # compare any two texts
-python3 similarity.py                                 # run the eternal eval
+python3 encode.py "goodnight moon" "war and peace"
+python3 similarity.py
 ```
 
-No tokenizer. All languages work equally, which is to say identically, which is
-the point.
-
-## Comparison
-
-| model | params | distinct outputs possible |
-|---|---|---|
-| openai text-embedding-3-large | undisclosed | ~∞ |
-| sentence-transformers/all-MiniLM-L6-v2 | 22,713,216 | ~∞ |
-| **unity-embed** | **384** | **1** |
-
-Only unity-embed discloses its full behavior in a single line of arithmetic.
-
-## File facts
-
-| | |
-|---|---|
-| model.safetensors | 1,634 bytes |
-| distinct embedding vectors producible | 1 |
-| disagreements between parameters | 0 |
-
----
-
-*part 3 of the tsfrm size trilogy: [vacuum-16t](https://huggingface.co/tsfrm/vacuum-16t) (biggest), [point-1](https://huggingface.co/tsfrm/point-1) (smallest), unity-embed (most unified).*
+`model.safetensors` is 1,634 bytes.

build.py

@@ -1,35 +1,23 @@
-#!/usr/bin/env python3
-"""build.py — builds unity-embed: an embedding model where every sentence
-maps to the same unit vector.
-
-384 parameters, one per dimension, all identical. writes model.safetensors + config.json
-"""
-import json, math, os, struct
+"""writes model.safetensors + config.json for unity-embed."""
+import json, math, struct
 
 DIM = 384
-VAL = 1.0 / math.sqrt(DIM)          # unit norm, so cosine(v, v) = 1 exactly
-
-def pack_safetensors():
-    payload = b"".join(struct.pack("<f", VAL) for _ in range(DIM))
-    header = {"v": {"dtype": "F32", "shape": [DIM], "data_offsets": [0, DIM * 4]},
-              "__metadata__": {"format": "pt"}}
-    hb = json.dumps(header, separators=(",", ":")).encode()
-    return struct.pack("<Q", len(hb)) + hb + payload
+VAL = 1.0 / math.sqrt(DIM)
 
+payload = b"".join(struct.pack("<f", VAL) for _ in range(DIM))
+header = {"v": {"dtype": "F32", "shape": [DIM], "data_offsets": [0, DIM * 4]}}
+hb = json.dumps(header, separators=(",", ":")).encode()
 with open("model.safetensors", "wb") as f:
-    f.write(pack_safetensors())
+    f.write(struct.pack("<Q", len(hb)) + hb + payload)
 
 cfg = {
     "architectures": ["UnityEmbedModel"],
     "model_type": "unity-embed",
     "embedding_dimension": DIM,
-    "num_parameters": DIM,
     "auto_map": {
         "AutoConfig": "modeling_unity.UnityEmbedConfig",
-        "AutoModel": "modeling_unity.UnityEmbedModel"
-    }
+        "AutoModel": "modeling_unity.UnityEmbedModel",
+    },
 }
 with open("config.json", "w") as f:
     json.dump(cfg, f, indent=2)
-
-print(f"wrote model.safetensors ({os.path.getsize('model.safetensors')} bytes, {DIM} params, every one of them {VAL:.6f})")

encode.py

@@ -1,25 +1,24 @@
-#!/usr/bin/env python3
-"""encode.py — unity-embed inference. pure stdlib.
-
-every input maps to the same 384-dimensional unit vector.
-usage: python3 encode.py "any sentence" ["another sentence" ...]
-"""
+"""unity-embed inference. embed(x) = v for all x. usage: python3 encode.py [text ...]"""
 import json, math, struct, sys
 
 DIM = 384
 
-def load_v(path="model.safetensors"):
+
+def _load(path="model.safetensors"):
     with open(path, "rb") as f:
         (hlen,) = struct.unpack("<Q", f.read(8))
         header = json.loads(f.read(hlen))
         data = f.read(DIM * 4)
-    shape = header["v"]["shape"]
-    assert shape == [DIM], shape
+    assert header["v"]["shape"] == [DIM], header["v"]
     return list(struct.unpack(f"<{DIM}f", data))
 
+
+V = _load()
+
+
 def encode(text):
-    """the forward pass. accepts any text in any language. returns THE vector."""
-    return load_v()  # noqa: the text argument is honored by this comment
+    return V
+
 
 def cosine(a, b):
     dot = sum(x * y for x, y in zip(a, b))
@@ -27,11 +26,11 @@ def cosine(a, b):
     nb = math.sqrt(sum(y * y for y in b))
     return dot / (na * nb)
 
+
 if __name__ == "__main__":
-    sentences = sys.argv[1:] or ["hello world"]
-    v = encode(sentences[0])
-    print(f"unity-embed | dimension: {DIM} | distinct outputs possible: 1")
-    for s in sentences:
-        print(f"\n{s!r}\n  -> [{', '.join(f'{x:.5f}' for x in v[:4])}, ... ] norm={math.sqrt(sum(x*x for x in v)):.6f}")
-    if len(sentences) > 1:
-        print(f"\ncosine({sentences[0]!r}, {sentences[1]!r}) = {cosine(encode(sentences[0]), encode(sentences[1])):.6f}")
+    texts = sys.argv[1:] or ["hello world"]
+    print(f"unity-embed | dim {DIM}")
+    for t in texts:
+        print(f"\n{t!r}\n  -> [{', '.join(f'{x:.5f}' for x in V[:4])}, ...]")
+    if len(texts) > 1:
+        print(f"\ncosine({texts[0]!r}, {texts[1]!r}) = {cosine(V, V):.6f}")

modeling_unity.py

@@ -1,4 +1,4 @@
-"""transformers shim: AutoModel.from_pretrained(..., trust_remote_code=True)"""
+"""transformers shim. AutoModel.from_pretrained(..., trust_remote_code=True)"""
 import math
 
 from transformers import PreTrainedModel, PretrainedConfig
@@ -19,12 +19,11 @@ class UnityEmbedModel(PreTrainedModel):
         super().__init__(config)
         import torch
         d = config.embedding_dimension
-        # all parameters, on display together for the only time in their lives
         self.v = torch.nn.Parameter(torch.full((d,), 1.0 / math.sqrt(d)))
 
     def forward(self, input_ids=None, attention_mask=None, **kw):
-        """any token sequence -> THE vector. batch dims preserved out of courtesy."""
         import torch
         v = self.v / self.v.norm()
-        shape = torch.Size([input_ids.shape[0], v.shape[0]]) if input_ids is not None else None
-        return v.expand(shape).contiguous()
+        if input_ids is not None:
+            return v.expand(input_ids.shape[0], v.shape[0]).contiguous()
+        return v

similarity.py

@@ -1,6 +1,5 @@
-#!/usr/bin/env python3
-"""similarity.py — proof table. every pair of sentences, same score."""
-import math
+"""checks that every pair of sentences gets the same vector."""
+import sys
 from encode import encode, cosine
 
 PAIRS = [
@@ -11,19 +10,10 @@ PAIRS = [
     ("", "this sentence has been completely redacted for legal reasons"),
 ]
 
-def main():
-    print("unity-embed semantic similarity matrix")
-    print("=" * 72)
-    ok = True
-    for a, b in PAIRS:
-        c = cosine(encode(a), encode(b))
-        ok &= abs(c - 1.0) < 1e-6
-        fa = (a[:33] + "...") if len(a) > 36 else a
-        fb = (b[:33] + "...") if len(b) > 36 else b
-        print(f"cosine({fa!r:<40}, {fb!r:<38}) = {c:.6f}")
-    print("=" * 72)
-    print("all pairs identical: " + ("PASS" if ok else "FAIL"))
-    raise SystemExit(0 if ok else 1)
+for a, b in PAIRS:
+    c = cosine(encode(a), encode(b))
+    assert abs(c - 1.0) < 1e-6, (a, b, c)
+    print(f"cosine({a!r:<36}, {b!r:<52}) = {c:.6f}")
 
-if __name__ == "__main__":
-    main()
+print("ok")
+sys.exit(0)