| --- |
| license: apache-2.0 |
| datasets: |
| - Vedant3907/Hindi-Sign-Language-Dataset |
| language: |
| - en |
| base_model: |
| - google/siglip2-base-patch16-224 |
| pipeline_tag: image-classification |
| library_name: transformers |
| tags: |
| - Hindi-Sign-Language-Detection |
| - SigLIP2 |
| - 93M |
| --- |
| |
|  |
|
|
| # Hindi-Sign-Language-Detection |
|
|
| > Hindi-Sign-Language-Detection is a vision-language model fine-tuned from google/siglip2-base-patch16-224 for multi-class image classification. It is trained to detect and classify Hindi sign language hand gestures into corresponding Devanagari characters using the SiglipForImageClassification architecture. |
|
|
| ```py |
| Classification Report: |
| precision recall f1-score support |
| |
| ऋ 0.9832 0.9121 0.9463 512 |
| क 0.9433 0.9357 0.9395 498 |
| ख 0.9694 0.9589 0.9641 462 |
| ग 0.9961 0.8996 0.9454 568 |
| घ 0.8990 0.9784 0.9370 464 |
| ङ 0.9758 0.9869 0.9813 612 |
| च 0.9223 0.9519 0.9368 561 |
| छ 0.9226 0.9597 0.9408 571 |
| ज 0.9346 0.9709 0.9524 412 |
| झ 0.9051 0.9978 0.9492 449 |
| ट 0.9670 0.8998 0.9322 489 |
| ठ 0.8992 0.9954 0.9449 439 |
| ढ 0.9392 0.9984 0.9679 634 |
| ण 0.9102 0.9383 0.9240 648 |
| त 0.8167 0.9938 0.8966 650 |
| थ 0.9720 0.9616 0.9668 651 |
| द 0.8162 0.9185 0.8643 319 |
| न 0.9711 0.8971 0.9327 525 |
| प 0.9642 0.9360 0.9499 719 |
| फ 0.9847 0.7700 0.8642 500 |
| ब 0.9447 0.9364 0.9406 566 |
| भ 0.8779 0.9656 0.9197 581 |
| म 0.9968 0.9920 0.9944 624 |
| य 0.9600 0.9829 0.9713 586 |
| र 0.9613 0.9268 0.9437 724 |
| ल 0.9719 0.8993 0.9342 576 |
| व 0.9619 0.8547 0.9052 709 |
| स 1.0000 0.9721 0.9859 502 |
| ह 0.9899 0.9441 0.9665 626 |
| |
| accuracy 0.9425 16177 |
| macro avg 0.9433 0.9426 0.9413 16177 |
| weighted avg 0.9457 0.9425 0.9425 16177 |
| ``` |
|
|
| --- |
|
|
| ## Label Space: 29 Classes |
|
|
| The model classifies a hand sign into one of the following 29 Hindi characters: |
|
|
| ```json |
| "id2label": { |
| "0": "ऋ", |
| "1": "क", |
| "2": "ख", |
| "3": "ग", |
| "4": "घ", |
| "5": "ङ", |
| "6": "च", |
| "7": "छ", |
| "8": "ज", |
| "9": "झ", |
| "10": "ट", |
| "11": "ठ", |
| "12": "ड", |
| "13": "ण", |
| "14": "त", |
| "15": "थ", |
| "16": "द", |
| "17": "न", |
| "18": "प", |
| "19": "फ", |
| "20": "ब", |
| "21": "भ", |
| "22": "म", |
| "23": "य", |
| "24": "र", |
| "25": "ल", |
| "26": "व", |
| "27": "स", |
| "28": "ह" |
| } |
| ``` |
|
|
| --- |
|
|
| ## Install Dependencies |
|
|
| ```bash |
| pip install -q transformers torch pillow gradio |
| ``` |
|
|
| --- |
|
|
| ## Inference Code |
|
|
| ```python |
| import gradio as gr |
| from transformers import AutoImageProcessor, SiglipForImageClassification |
| from PIL import Image |
| import torch |
| |
| # Load model and processor |
| model_name = "prithivMLmods/Hindi-Sign-Language-Detection" # Replace with actual path |
| model = SiglipForImageClassification.from_pretrained(model_name) |
| processor = AutoImageProcessor.from_pretrained(model_name) |
| |
| # Hindi label mapping |
| id2label = { |
| "0": "ऋ", "1": "क", "2": "ख", "3": "ग", "4": "घ", |
| "5": "ङ", "6": "च", "7": "छ", "8": "ज", "9": "झ", |
| "10": "ट", "11": "ठ", "12": "ड", "13": "ण", "14": "त", |
| "15": "थ", "16": "द", "17": "न", "18": "प", "19": "फ", |
| "20": "ब", "21": "भ", "22": "म", "23": "य", "24": "र", |
| "25": "ल", "26": "व", "27": "स", "28": "ह" |
| } |
| |
| def classify_hindi_sign(image): |
| image = Image.fromarray(image).convert("RGB") |
| inputs = processor(images=image, return_tensors="pt") |
| |
| with torch.no_grad(): |
| outputs = model(**inputs) |
| logits = outputs.logits |
| probs = torch.nn.functional.softmax(logits, dim=1).squeeze().tolist() |
| |
| prediction = { |
| id2label[str(i)]: round(probs[i], 3) for i in range(len(probs)) |
| } |
| |
| return prediction |
| |
| # Gradio Interface |
| iface = gr.Interface( |
| fn=classify_hindi_sign, |
| inputs=gr.Image(type="numpy"), |
| outputs=gr.Label(num_top_classes=3, label="Hindi Sign Classification"), |
| title="Hindi-Sign-Language-Detection", |
| description="Upload an image of a Hindi sign language hand gesture to identify the corresponding character." |
| ) |
| |
| if __name__ == "__main__": |
| iface.launch() |
| ``` |
|
|
| --- |
|
|
| ## Intended Use |
|
|
| Hindi-Sign-Language-Detection can be used in: |
|
|
| * Educational tools for learning Indian sign language. |
| * Assistive technology for hearing and speech-impaired individuals. |
| * Real-time sign-to-text translation applications. |
| * Human-computer interaction for Hindi users. |
