hyper3labs/hyper3-clip-v1

Hyper3-CLIP v1

Hyper3-CLIP is an open-weight vision-language model from hyper³labs for hierarchy-sensitive image-text retrieval.

Which Embedding Should I Use?

Interface	Shape	Use it for
SentenceTransformers or encode_image / encode_text	512	Cosine similarity and standard vector databases
encode_image_lorentz / encode_text_lorentz	513	Native hyperbolic scoring and reranking

The model learns 512 embedding coordinates. A point in the 512-dimensional Lorentz hyperbolic space is represented by 513 ambient coordinates: one time-like coordinate plus 512 spatial coordinates. The extra coordinate is derived by the Lorentz exponential map; it is not an additional learned feature. Native outputs satisfy -x[0]² + sum(x[1:]²) ≈ -1 / model.curvature.

Use the 512-d interface for ordinary retrieval. Use the 513-d interface only when your scorer understands the Lorentz inner product. Do not compare 512-d and 513-d vectors directly.

Install

pip install "torch>=2.2" "transformers>=4.49,<5" "timm>=1.0" \
  "safetensors>=0.4" "pyyaml>=6" "Pillow>=10" \
  "sentence-transformers>=5.5.1"

This repository contains custom model code, so loading requires trust_remote_code=True. For production, review the code and pin revision to a Hub commit SHA. If access is gated, accept the model conditions and run hf auth login or set HF_TOKEN first.

Standard 512-D Retrieval

from PIL import Image
from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "hyper3labs/hyper3-clip-v1",
    trust_remote_code=True,
)

image_vectors = model.encode(
    [Image.open("/path/to/image.jpg")],
    normalize_embeddings=True,
)
text_vectors = model.encode(
    ["machined metal part"],
    normalize_embeddings=True,
)

assert image_vectors.shape[1] == text_vectors.shape[1] == 512

These normalized vectors work with cosine or dot-product indexes. This is also the backward-compatible interface used by Haystack's SentenceTransformersDocumentImageEmbedder and SentenceTransformersTextEmbedder.

Native 513-D Lorentz Embeddings

from PIL import Image
import torch
from transformers import AutoImageProcessor, AutoModel, AutoTokenizer

model_id = "hyper3labs/hyper3-clip-v1"
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
).eval()
tokenizer = AutoTokenizer.from_pretrained(model_id)
processor = AutoImageProcessor.from_pretrained(model_id, use_fast=False)

pixel_values = processor(
    images=Image.open("/path/to/image.jpg"),
    return_tensors="pt",
).pixel_values
tokens = tokenizer(
    ["machined metal part"],
    padding=True,
    truncation=True,
    max_length=model.config.max_text_length,
    return_tensors="pt",
)

with torch.inference_mode():
    image_vectors = model.encode_image_lorentz(pixel_values)
    text_vectors = model.encode_text_lorentz(**tokens)

assert image_vectors.shape[1] == text_vectors.shape[1] == 513
scores = model.lorentz_inner_product(text_vectors, image_vectors)

The native path applies the learned projection, modality scale, and Lorentz exponential map in float32. model.curvature is the bounded curvature used by training and inference (0.1 for this checkpoint). The raw checkpoint value is available as model.learned_curvature (approximately 0.0994883). See model.geometry_metadata for the complete geometry contract.

For Qdrant, store the 513-d vectors with models.VectorParams(**model.qdrant_vector_config) and transform queries with model.qdrant_query_vector(...). Use exact search for guaranteed Lorentz ranking. For large approximate indexes, retrieve candidates with 512-d cosine vectors and rerank them with model.lorentz_inner_product(...).

Model Details

• Vision backbone: vit_base_patch16_224
• Text backbone: CLIP text transformer compatible with openai/clip-vit-base-patch32
• Image size: 224 x 224
• Maximum text length: 77 tokens
• Training steps: 500,000
• Global batch size: 768
• Native geometry: 513-d Lorentz coordinates, time coordinate at index 0, float32, curvature 0.1

Evaluation

Higher is better except for TIE and LCA. Retrieval values are R@10.

Model	ImageNet top-1	COCO text	COCO image	Flickr text	Flickr image	TIE	LCA	Jaccard	H-Prec	H-Rec
MERU-B/16	40.1	82.0	68.6	96.2	90.0	3.630	2.220	0.780	0.850	0.850
HyCoCLIP-B/16	45.8	82.0	69.3	95.4	90.3	3.172	2.047	0.814	0.874	0.874
UNCHA-B/16	48.8	82.6	71.0	95.9	91.2	2.945	1.961	0.828	0.883	0.884
PHyCLIP-B/16	44.4	80.4	68.7	95.6	89.9	3.285	2.088	0.807	0.868	0.868
Hyper3-CLIP v1	48.5	84.0	72.8	97.5	92.4	2.972	1.986	0.828	0.882	0.883

ImageNet and hierarchy evaluation used bicubic shorter-side resize followed by a 224 x 224 center crop. The bundled processor preserves the original square-resize retrieval interface. Use use_fast=False for preprocessing parity. Raw results are included in the four eval_*_final.json files.

Use And License

This release is intended for image-text retrieval, multimodal embedding research, and hierarchy-sensitive evaluation. It has not been validated for safety-critical use.

The model materials use the OpenMDW-1.0 license. See LICENSE and NOTICE and preserve both when redistributing the model.