| --- |
| license: apache-2.0 |
| tags: |
| - fashion |
| - retrieval |
| - text-to-image |
| - open_clip |
| - routing |
| pipeline_tag: feature-extraction |
| --- |
| |
| # MODA Duo |
|
|
| **Two open constituents, one answer per query.** Duo routes each text query to whichever |
| open MODA system suits its shape — short catalogue titles to |
| [MODA Pro Lite+](https://huggingface.co/HopitAI/moda-pro-lite-plus), longer descriptions to |
| [MODA](https://huggingface.co/HopitAI/moda-fashionsiglip-multiview-203m) — and runs |
| **one encoder and one nearest-neighbour query per search**. |
|
|
| Duo adds **zero parameters**. It is a serving recipe over two open systems, not a new model. |
|
|
| ## Why |
|
|
| Fashion search queries come in two shapes, and no single small model is best at both: |
|
|
| | query shape | example | best open system ≤250M | |
| |---|---|---| |
| | catalogue title | `buckle round toe flat shoes` | MODA Pro Lite+ | |
| | natural description | `When warm weekends are abound, make sure your closet…` | MODA | |
|
|
| Duo picks per query. On a mixed workload it beats **both** constituents. |
|
|
| ## Results |
|
|
| MAP@10, full corpus, all ground-truth queries, one evaluator (`pytrec_eval map_cut.10`), |
| paired bootstrap 10,000 resamples. |
|
|
| | benchmark | MODA | MODA Pro Lite+ | **MODA Duo** | |
| |---|---:|---:|---:| |
| | KAGL | 0.2887 | 0.3201 | **0.3201** | |
| | Polyvore | 0.3726 | 0.4049 | **0.4049** | |
| | Atlas | 0.1862 | 0.1904 | **0.1904** | |
| | Fashion200K | **0.1946** | 0.1846 | 0.1866 | |
| | DeepFashion In-Shop | **0.1642** | 0.1026 | 0.1640 | |
| | DeepFashion Multimodal | 0.0147 | 0.0133 | **0.0159** | |
| | **pooled, 12,000 queries** | 0.2035 | 0.2026 | **0.2137** | |
|
|
| Pooled across all six benchmarks — the mixed workload a router exists for — Duo is |
| **+5.0% over MODA and +5.4% over MODA Pro Lite+**, both significant. |
|
|
| Fashion200K is the honest miss: its queries sit where the two constituents are hardest to |
| tell apart, and Duo trails MODA there by 4%. Where a workload is known to be all long descriptions, use MODA |
| directly. |
|
|
| ## Serving cost |
|
|
| ``` |
| indexes 2 one per constituent, built offline |
| stored vectors per item 2 |
| encoders run per query 1 only the routed constituent's text tower |
| ANN queries per search 1 |
| re-ranking none |
| ``` |
|
|
| Compared with a single open model, Duo costs one extra index at build time and nothing |
| extra at query time. |
|
|
| ## Use |
|
|
| ```bash |
| pip install open_clip_torch pillow numpy hnswlib |
| python serving_ann.py --demo |
| ``` |
|
|
| ```python |
| from serving_ann import Duo |
| duo = Duo() # loads both constituents |
| duo.build(images) # encodes the catalogue with both, builds two indexes |
| ids, scores, routes = duo.search(["black leather ankle boots"], k=10) |
| ``` |
|
|
| The router is a callable — replace it with any policy that maps a query to a constituent: |
|
|
| ```python |
| duo = Duo(router=lambda q: "moda" if looks_like_a_description(q) else "moda_pro_lite_plus") |
| ``` |
|
|
| ## Evaluation |
|
|
| All figures are full corpus, all ground-truth queries, MAP@10 under one evaluator |
| (`pytrec_eval map_cut.10`), paired bootstrap with 10,000 resamples. Per-query results are in |
| the [repository](https://github.com/hopit-ai/Moda). |
|
|
| ## Related |
|
|
| - [MODA](https://huggingface.co/HopitAI/moda-fashionsiglip-multiview-203m) — FashionSigLIP with a serving harness. Open source, open weights. |
| - [MODA Pro Lite](https://huggingface.co/HopitAI/moda-pro-lite) — a trained fashion encoder. Open weights. |
| - MODA Pro — hosted. Fuses both constituents rather than choosing between them. |
|
|
