| | --- |
| | license: apache-2.0 |
| | datasets: |
| | - VinayHajare/Marathi-Sign-Language |
| | language: |
| | - en |
| | base_model: |
| | - google/siglip2-base-patch16-224 |
| | pipeline_tag: image-classification |
| | library_name: transformers |
| | tags: |
| | - Marathi-Sign-Language-Detection |
| | - SigLIP2 |
| | - 93M |
| | --- |
| | |
| |  |
| |
|
| | # Marathi-Sign-Language-Detection |
| |
|
| | > Marathi-Sign-Language-Detection is a vision-language model fine-tuned from google/siglip2-base-patch16-224 for multi-class image classification. It is trained to recognize Marathi sign language hand gestures and map them to corresponding Devanagari characters using the SiglipForImageClassification architecture. |
| |
|
| | ```py |
| | Classification Report: |
| | precision recall f1-score support |
| | |
| | अ 0.9881 0.9911 0.9896 1009 |
| | आ 0.9926 0.9237 0.9569 1022 |
| | इ 0.8132 0.9609 0.8809 1101 |
| | ई 0.9424 0.8894 0.9151 1103 |
| | उ 0.9477 0.9073 0.9271 1198 |
| | ऊ 0.9436 1.0000 0.9710 1071 |
| | ए 0.9153 0.9378 0.9264 1141 |
| | ऐ 0.7790 0.8871 0.8295 1089 |
| | ओ 0.9188 0.9581 0.9381 1075 |
| | औ 1.0000 0.9226 0.9598 1021 |
| | क 0.9566 0.9160 0.9358 1083 |
| | क्ष 0.9287 0.9667 0.9473 1200 |
| | ख 0.9913 1.0000 0.9956 1140 |
| | ग 0.9753 0.9982 0.9866 1109 |
| | घ 0.8398 0.7908 0.8146 1200 |
| | च 0.9388 0.9016 0.9198 1158 |
| | छ 0.9764 0.8127 0.8870 1169 |
| | ज 0.9599 0.9967 0.9779 1200 |
| | ज्ञ 0.9878 0.9483 0.9677 1200 |
| | झ 0.9939 0.9567 0.9749 1200 |
| | ट 0.8917 0.8992 0.8954 1200 |
| | ठ 0.9075 0.8425 0.8738 1200 |
| | ड 0.9354 0.9900 0.9619 1200 |
| | ढ 0.8616 0.9025 0.8816 1200 |
| | ण 0.9114 0.9425 0.9267 1200 |
| | त 0.9280 0.9025 0.9151 1200 |
| | थ 0.9388 0.9717 0.9550 1200 |
| | द 0.8648 0.9275 0.8951 1200 |
| | ध 0.9876 0.9917 0.9896 1200 |
| | न 0.7256 0.8967 0.8021 1200 |
| | प 0.9991 0.9683 0.9835 1200 |
| | फ 0.8909 0.8575 0.8739 1200 |
| | ब 0.9814 0.7917 0.8764 1200 |
| | भ 0.9758 0.8383 0.9018 1200 |
| | म 0.8121 0.8142 0.8132 1200 |
| | य 0.5726 0.9133 0.7039 1200 |
| | र 0.7635 0.7339 0.7484 1210 |
| | ल 0.9239 0.8800 0.9014 1200 |
| | ळ 0.8950 0.7533 0.8181 1200 |
| | व 0.9597 0.7542 0.8446 1200 |
| | श 0.8829 0.8667 0.8747 1200 |
| | स 0.8449 0.8758 0.8601 1200 |
| | ह 0.9604 0.8883 0.9229 1200 |
| | |
| | accuracy 0.9027 50099 |
| | macro avg 0.9117 0.9039 0.9051 50099 |
| | weighted avg 0.9107 0.9027 0.9040 50099 |
| | ``` |
| |
|
| | --- |
| |
|
| | ## Label Space: 43 Classes |
| |
|
| | The model classifies a hand sign into one of the following 43 Marathi characters: |
| |
|
| | ```json |
| | "id2label": { |
| | "0": "अ", "1": "आ", "2": "इ", "3": "ई", "4": "उ", "5": "ऊ", |
| | "6": "ए", "7": "ऐ", "8": "ओ", "9": "औ", "10": "क", "11": "क्ष", |
| | "12": "ख", "13": "ग", "14": "घ", "15": "च", "16": "छ", "17": "ज", |
| | "18": "ज्ञ", "19": "झ", "20": "ट", "21": "ठ", "22": "ड", "23": "ढ", |
| | "24": "ण", "25": "त", "26": "थ", "27": "द", "28": "ध", "29": "न", |
| | "30": "प", "31": "फ", "32": "ब", "33": "भ", "34": "म", "35": "य", |
| | "36": "र", "37": "ल", "38": "ळ", "39": "व", "40": "श", "41": "स", "42": "ह" |
| | } |
| | ``` |
| |
|
| | --- |
| |
|
| | ## Install Dependencies |
| |
|
| | ```bash |
| | pip install -q transformers torch pillow gradio |
| | ``` |
| |
|
| | --- |
| |
|
| | ## Inference Code |
| |
|
| | ```python |
| | import gradio as gr |
| | from transformers import AutoImageProcessor, SiglipForImageClassification |
| | from PIL import Image |
| | import torch |
| | |
| | # Load model and processor |
| | model_name = "prithivMLmods/Marathi-Sign-Language-Detection" # Replace with actual path |
| | model = SiglipForImageClassification.from_pretrained(model_name) |
| | processor = AutoImageProcessor.from_pretrained(model_name) |
| | |
| | # Marathi label mapping |
| | id2label = { |
| | "0": "अ", "1": "आ", "2": "इ", "3": "ई", "4": "उ", "5": "ऊ", |
| | "6": "ए", "7": "ऐ", "8": "ओ", "9": "औ", "10": "क", "11": "क्ष", |
| | "12": "ख", "13": "ग", "14": "घ", "15": "च", "16": "छ", "17": "ज", |
| | "18": "ज्ञ", "19": "झ", "20": "ट", "21": "ठ", "22": "ड", "23": "ढ", |
| | "24": "ण", "25": "त", "26": "थ", "27": "द", "28": "ध", "29": "न", |
| | "30": "प", "31": "फ", "32": "ब", "33": "भ", "34": "म", "35": "य", |
| | "36": "र", "37": "ल", "38": "ळ", "39": "व", "40": "श", "41": "स", "42": "ह" |
| | } |
| | |
| | def classify_marathi_sign(image): |
| | image = Image.fromarray(image).convert("RGB") |
| | inputs = processor(images=image, return_tensors="pt") |
| | |
| | with torch.no_grad(): |
| | outputs = model(**inputs) |
| | logits = outputs.logits |
| | probs = torch.nn.functional.softmax(logits, dim=1).squeeze().tolist() |
| | |
| | prediction = { |
| | id2label[str(i)]: round(probs[i], 3) for i in range(len(probs)) |
| | } |
| | |
| | return prediction |
| | |
| | # Gradio Interface |
| | iface = gr.Interface( |
| | fn=classify_marathi_sign, |
| | inputs=gr.Image(type="numpy"), |
| | outputs=gr.Label(num_top_classes=5, label="Marathi Sign Classification"), |
| | title="Marathi-Sign-Language-Detection", |
| | description="Upload an image of a Marathi sign language hand gesture to identify the corresponding character." |
| | ) |
| | |
| | if __name__ == "__main__": |
| | iface.launch() |
| | ``` |
| |
|
| | --- |
| |
|
| | ## Intended Use |
| |
|
| | Marathi-Sign-Language-Detection can be applied in: |
| |
|
| | * Educational platforms for learning regional sign language. |
| | * Assistive communication tools for Marathi-speaking users with hearing impairments. |
| | * Interactive applications that translate signs into text. |
| | * Research and data collection for sign language development and recognition. |
