| | --- |
| | license: cc-by-nc-4.0 |
| | language: |
| | - en |
| | base_model: |
| | - facebook/metaclip-2-worldwide-s16 |
| | pipeline_tag: image-classification |
| | library_name: transformers |
| | tags: |
| | - text-generation-inference |
| | - open-scene |
| | --- |
| | |
| |  |
| |
|
| | # **MetaCLIP-2-Open-Scene** |
| |
|
| | > **MetaCLIP-2-Open-Scene** is an image classification vision-language encoder model fine-tuned from **[facebook/metaclip-2-worldwide-s16](https://huggingface.co/facebook/metaclip-2-worldwide-s16)** for a single-label classification task. |
| | > It is designed to identify and categorize various natural and urban scenes using the **MetaClip2ForImageClassification** architecture. |
| |
|
| | >[!note] |
| | MetaCLIP 2: A Worldwide Scaling Recipe : https://huggingface.co/papers/2507.22062 |
| |
|
| | ``` |
| | Classification Report: |
| | precision recall f1-score support |
| | |
| | buildings 0.9644 0.9703 0.9673 2625 |
| | forest 0.9948 0.9978 0.9963 2694 |
| | glacier 0.9531 0.9427 0.9479 2671 |
| | mountain 0.9470 0.9512 0.9491 2723 |
| | sea 0.9909 0.9920 0.9915 2758 |
| | street 0.9728 0.9694 0.9711 2874 |
| | |
| | accuracy 0.9706 16345 |
| | macro avg 0.9705 0.9706 0.9705 16345 |
| | weighted avg 0.9706 0.9706 0.9706 16345 |
| | ``` |
| |
|
| |  |
| |
|
| | The model classifies images into six open-scene categories: |
| |
|
| | * **Class 0:** "buildings" |
| | * **Class 1:** "forest" |
| | * **Class 2:** "glacier" |
| | * **Class 3:** "mountain" |
| | * **Class 4:** "sea" |
| | * **Class 5:** "street" |
| |
|
| | # **Run with Transformers** |
| |
|
| | ```python |
| | !pip install -q transformers torch pillow gradio |
| | ``` |
| |
|
| | ```python |
| | import gradio as gr |
| | from transformers import AutoImageProcessor |
| | from transformers import AutoModelForImageClassification |
| | from transformers.image_utils import load_image |
| | from PIL import Image |
| | import torch |
| | |
| | # Load model and processor |
| | model_name = "prithivMLmods/MetaCLIP-2-Open-Scene" |
| | model = AutoModelForImageClassification.from_pretrained(model_name) |
| | processor = AutoImageProcessor.from_pretrained(model_name) |
| | |
| | def scene_classification(image): |
| | """Predicts the type of scene represented in an image.""" |
| | image = Image.fromarray(image).convert("RGB") |
| | inputs = processor(images=image, return_tensors="pt") |
| | |
| | with torch.no_grad(): |
| | outputs = model(**inputs) |
| | logits = outputs.logits |
| | probs = torch.nn.functional.softmax(logits, dim=1).squeeze().tolist() |
| | |
| | labels = { |
| | "0": "buildings", |
| | "1": "forest", |
| | "2": "glacier", |
| | "3": "mountain", |
| | "4": "sea", |
| | "5": "street" |
| | } |
| | predictions = {labels[str(i)]: round(probs[i], 3) for i in range(len(probs))} |
| | |
| | return predictions |
| | |
| | # Create Gradio interface |
| | iface = gr.Interface( |
| | fn=scene_classification, |
| | inputs=gr.Image(type="numpy"), |
| | outputs=gr.Label(label="Prediction Scores"), |
| | title="Open Scene Classification", |
| | description="Upload an image to classify the scene type (e.g., forest, sea, street, mountain, etc.)." |
| | ) |
| | |
| | # Launch the app |
| | if __name__ == "__main__": |
| | iface.launch() |
| | ``` |
| |
|
| | # **Sample Inference:** |
| |
|
| |  |
| |  |
| |  |
| |  |
| |  |
| |
|
| | # **Intended Use:** |
| |
|
| | The **MetaCLIP-2-Open-Scene** model is designed to classify a wide range of natural and urban environments. |
| | Potential use cases include: |
| |
|
| | * **Geographical Image Analysis:** Categorizing landscapes for environmental and mapping studies. |
| | * **Tourism and Travel Applications:** Automatically tagging scenic photos for organization and recommendations. |
| | * **Autonomous Systems:** Supporting navigation and perception in robotics and self-driving systems. |
| | * **Environmental Monitoring:** Detecting and classifying geographic features for research. |
| | * **Media and Photography:** Assisting in photo organization and content-based retrieval. |
