| --- |
| language: |
| - ig |
| - ha |
| - yo |
| - en |
| library_name: sentence-transformers |
| pipeline_tag: sentence-similarity |
| tags: |
| - embeddings |
| - sentence-transformers |
| - cross-lingual |
| - multilingual |
| - igbo |
| - hausa |
| - yoruba |
| - information-retrieval |
| - semantic-search |
| base_model: sentence-transformers/LaBSE |
| --- |
| |
| <div align="center"> |
| <img src="images/modular_logo.jpg" width="38%" alt="Modular" /> |
| </div> |
| <hr> |
| <div align="center" style="line-height: 1;"> |
| <a href="https://huggingface.co/Modularcomputing" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Modular-ffc107?color=ffc107&logoColor=white"/></a> |
| <a href="https://huggingface.co/Modularcomputing/Native-Bird2" target="_blank"><img alt="Native-Bird2" src="https://img.shields.io/badge/Multimodal-Native--Bird2-536af5?color=536af5&logoColor=white"/></a> |
| <a href="https://github.com/Dox45" target="_blank"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-Dox45-181717?logo=github&logoColor=white"/></a> |
| <br> |
| <img alt="Languages" src="https://img.shields.io/badge/Languages-Igbo%20%7C%20Hausa%20%7C%20Yoruba%20%7C%20English-1f6feb"/> |
| <img alt="Embedding dim" src="https://img.shields.io/badge/Embedding-768--d-2ea44f"/> |
| <img alt="Base" src="https://img.shields.io/badge/Base-LaBSE-6f42c1"/> |
| </div> |
|
|
| <p align="center"> |
| <img src="images/native_bird_diagram.jpg" width="100%" alt="Native-Bird: an Igbo query and English documents in one shared semantic vector space" /> |
| </p> |
|
|
| ## Table of Contents |
|
|
| 1. [Introduction](#1-introduction) |
| 2. [Model Summary](#2-model-summary) |
| 3. [Model Downloads](#3-model-downloads) |
| 4. [Evaluation Results](#4-evaluation-results) |
| 5. [Quick Start](#5-quick-start) |
| 6. [Searching Real Document Collections](#6-searching-real-document-collections) |
| 7. [Limitations](#7-limitations) |
| 8. [Citation](#8-citation) |
| 9. [Contact](#9-contact) |
|
|
| ## 1. Introduction |
|
|
| **Native-Bird** is a 768-dimensional multilingual sentence-embedding model fine-tuned from **LaBSE** for cross-lingual retrieval across **Igbo, Hausa, Yoruba and English**. |
|
|
| Its primary use case is **native-language query → English document retrieval**: a user describes what they are looking for in Igbo, Hausa or Yoruba, the indexed documents are written in English, and Native-Bird maps both into the same embedding space for semantic search. |
|
|
| > **Igbo query:** “Neuklọs ahụ bụ ahụ ihe abụọ mejupụtara ya - neutrọn na protọn.” |
| > |
| > **English document:** “The nucleus consists of two particles - neutrons and protons.” |
|
|
| Native-Bird is the text backbone of [**Native-Bird2**](https://huggingface.co/Modularcomputing/Native-Bird2), which extends the same space to images. |
|
|
| ## 2. Model Summary |
|
|
| **Architecture** |
|
|
| | | | |
| |---|---| |
| | Architecture | SentenceTransformer (LaBSE encoder → pooling → dense → normalize) | |
| | Base model | [`sentence-transformers/LaBSE`](https://huggingface.co/sentence-transformers/LaBSE) | |
| | Embedding dimension | 768 | |
| | Max sequence length (training) | 128 tokens | |
| | Similarity | Cosine | |
| | Languages | Igbo (`ig`), Hausa (`ha`), Yoruba (`yo`), English (`en`) | |
|
|
| **Training** |
|
|
| | | | |
| |---|---| |
| | Objective | Cached Multiple Negatives Ranking Loss (CachedMNRL) | |
| | Training pairs | 76,150 native-language ↔ English pairs | |
| | Epochs | 4 | |
| | Per-device batch size | 256 (CachedMNRL mini-batch 128) | |
| | Learning rate / schedule | 2e-5, cosine, 10% warmup | |
| | Negatives | In-batch, NO_DUPLICATES sampling | |
| | Precision | BF16 | |
| |
| ## 3. Model Downloads |
| |
| <div align="center"> |
| |
| | **Model** | **Modality** | **Embedding dim** | **Download** | |
| | :---: | :---: | :---: | :---: | |
| | Native-Bird | Text (ig / ha / yo / en) | 768 | [🤗 Hugging Face](https://huggingface.co/Modularcomputing/Native-Bird) | |
| | Native-Bird2 | Text + Image | 512 | [🤗 Hugging Face](https://huggingface.co/Modularcomputing/Native-Bird2) | |
| |
| </div> |
| |
| ## 4. Evaluation Results |
| |
| Cross-lingual retrieval against an English corpus of **1,004 sentences**, with **204 queries per language**. Each query has one known matching English sentence. Two query variants are tested: **clean** (original text) and **ASR-style noise** (diacritics removed, lowercased, punctuation stripped, whitespace normalized). |
| |
| <div align="center"> |
| |
| | **Language** | **Query** | **LaBSE R@1** | **Native-Bird R@1** | **LaBSE MRR** | **Native-Bird MRR** | |
| | :--- | :--- | :---: | :---: | :---: | :---: | |
| | Igbo | Clean | 1.0000 | **1.0000** | 1.0000 | **1.0000** | |
| | Igbo | ASR-style noise | 0.9853 | **1.0000** | 0.9888 | **1.0000** | |
| | Hausa | Clean | 1.0000 | **1.0000** | 1.0000 | **1.0000** | |
| | Hausa | ASR-style noise | 0.9951 | **1.0000** | 0.9975 | **1.0000** | |
| | Yoruba | Clean | 0.9804 | **0.9804** | 0.9871 | **0.9902** | |
| | Yoruba | ASR-style noise | 0.9559 | **0.9804** | 0.9683 | **0.9867** | |
| |
| </div> |
| |
| Native-Bird reaches R@10 = 1.0000 in every setting (LaBSE: 0.9951 on noisy Igbo and Yoruba). Fine-tuning mainly improves **robustness to transcription-style noise**, which matters for voice and keyboard input without diacritics. |
| |
| **Direct Igbo → English retrieval examples** |
| |
| | Igbo query | Top English result | Cosine | |
| | :--- | :--- | :---: | |
| | Mgbanwe n'ọdịdị na-agbakwunye ọdịdị kejenetiki ọhụrụ, ma nhọpụta na-ewepụ ya n'ogwu ọdịdị nke a kọwapụtara. | Mutation adds new genetic variation, and selection removes it from the pool of expressed variation. | 0.7079 | |
| | Neuklọs ahụ bụ ahụ ihe abụọ mejupụtara ya - neutrọn na protọn. | The nucleus consists of two particles - neutrons and protons. | 0.7049 | |
| | Enwere ọtụtụ ihe mere ha ji kara web prọgzi mma: ha na-agbanwe ụzọ trafik ịntanetị niile, ọ bụghị naanị http. | They are superior to web proxies for several reasons: They re-route all Internet traffic, not only http. | 0.6774 | |
| |
| ## 5. Quick Start |
| |
| ```bash |
| pip install -U sentence-transformers torch |
| ``` |
| |
| ```python |
| from sentence_transformers import SentenceTransformer |
|
|
| model = SentenceTransformer("Modularcomputing/Native-Bird") |
|
|
| query = "Neuklọs ahụ bụ ahụ ihe abụọ mejupụtara ya - neutrọn na protọn." |
| documents = [ |
| "The nucleus consists of two particles - neutrons and protons.", |
| "The liver is an organ responsible for many metabolic functions.", |
| "Photosynthesis converts light energy into chemical energy.", |
| ] |
| |
| q = model.encode(query, normalize_embeddings=True) |
| d = model.encode(documents, normalize_embeddings=True) |
| scores = d @ q |
|
|
| for i in scores.argsort()[::-1]: |
| print(f"{scores[i]:.4f} {documents[i]}") |
| ``` |
| |
| The matching English sentence should rank first. |
|
|
| ## 6. Searching Real Document Collections |
|
|
| Do **not** embed a multi-page document as one vector. Split each document into passages of roughly 100–300 words with overlap, and store each passage embedding with its parent document ID. |
|
|
| ```text |
| Igbo / Hausa / Yoruba query |
| ↓ Native-Bird encoder |
| 768-d query vector |
| ↓ cosine similarity (FAISS / vector DB) |
| English passage index |
| ↓ |
| Top-k passages → grouped by parent document |
| ``` |
|
|
| **Intended applications:** native-language search over English knowledge bases, cross-lingual RAG retrieval, multilingual educational search, and search over Nigerian institutional documents. |
|
|
| ## 7. Limitations |
|
|
| - **Embedding model, not a translator or generator.** It represents text as vectors; it does not produce English answers. |
| - **Sentence-level evaluation.** Trained at 128 tokens and evaluated on sentences, so long documents need chunking. Scores do not guarantee the same accuracy on arbitrary long documents. |
| - **Domain shift.** Government, legal, medical and technical text can differ substantially from the benchmark; evaluate on your target domain before production use. |
| - **Status: research / early release.** The next validation step is a human-annotated benchmark of real Nigerian documents and real Igbo, Hausa and Yoruba information needs. |
|
|
| ## 8. Citation |
|
|
| ```bibtex |
| @misc{nativebird2026, |
| title = {Native-Bird: Cross-lingual Native-Language Embeddings for English Document Retrieval}, |
| author = {Akuegbo, Chima Emmanuel}, |
| year = {2026}, |
| publisher = {Modular}, |
| howpublished = {\url{https://huggingface.co/Modularcomputing/Native-Bird}} |
| } |
| ``` |
|
|
| ## 9. Contact |
|
|
| Questions, issues or collaboration: open a discussion on the [Community tab](https://huggingface.co/Modularcomputing/Native-Bird/discussions). |
|
|
