Enhance model card for TransOSS: Add metadata, paper, code, and usage details

README.md

@@ -1,3 +1,140 @@
----
-license: cc-by-4.0
----
+---
+license: cc-by-4.0
+pipeline_tag: image-feature-extraction
+library_name: transformers
+tags:
+  - re-identification
+  - remote-sensing
+  - ship-reid
+  - cross-modal
+  - optical
+  - sar
+  - vision-transformer
+---
+
+# Cross-modal Ship Re-Identification via Optical and SAR Imagery: A Novel Dataset and Method
+
+This repository contains the **TransOSS** model, a baseline method for cross-modal ship re-identification, presented in the paper [Cross-modal Ship Re-Identification via Optical and SAR Imagery: A Novel Dataset and Method](https://huggingface.co/papers/2506.22027).
+
+**TransOSS** is built upon the Vision Transformer architecture, designed to refine patch embedding, incorporate additional embeddings, and utilize contrastive learning for pre-training on large-scale optical-SAR image pairs. This ensures the model's ability to extract modality-invariant features crucial for effective ship tracking using low-Earth orbit optical and SAR sensors.
+
+**[📝 Paper (arXiv)](https://arxiv.org/abs/2506.22027)** | **[📄 Paper (Hugging Face)](https://huggingface.co/papers/2506.22027)** | **[💻 Code](https://github.com/Alioth2000/TransOSS)** | **[🗃️ Dataset](https://zenodo.org/records/15860212)**
+
+## Abstract
+
+Detecting and tracking ground objects using earth observation imagery remains a significant challenge in the field of remote sensing. Continuous maritime ship tracking is crucial for applications such as maritime search and rescue, law enforcement, and shipping analysis. However, most current ship tracking methods rely on geostationary satellites or video satellites. The former offer low resolution and are susceptible to weather conditions, while the latter have short filming durations and limited coverage areas, making them less suitable for the real-world requirements of ship tracking. To address these limitations, we present the Hybrid Optical and Synthetic Aperture Radar (SAR) Ship Re-Identification Dataset (HOSS ReID dataset), designed to evaluate the effectiveness of ship tracking using low-Earth orbit constellations of optical and SAR sensors. This approach ensures shorter re-imaging cycles and enables all-weather tracking. HOSS ReID dataset includes images of the same ship captured over extended periods under diverse conditions, using different satellites of different modalities at varying times and angles. Furthermore, we propose a baseline method for cross-modal ship re-identification, TransOSS, which is built on the Vision Transformer architecture. It refines the patch embedding structure to better accommodate cross-modal tasks, incorporates additional embeddings to introduce more reference information, and employs contrastive learning to pre-train on large-scale optical-SAR image pairs, ensuring the model's ability to extract modality-invariant features.
+
+## HOSS ReID Dataset
+
+The HOSS ReID dataset and the associated pretraining dataset are publicly available on [Zenodo](https://zenodo.org/records/15860212).
+The pretraining dataset is constructed based on the [SEN1-2](https://www.kaggle.com/datasets/requiemonk/sentinel12-image-pairs-segregated-by-terrain) and [DFC23](https://ieee-dataport.org/competitions/2023-ieee-grss-data-fusion-contest-large-scale-fine-grained-building-classification) datasets.
+To run TransOSS, please organize the data in the following structure under the `data` directory:
+```
+data
+├── HOSS
+│   ├── bounding_box_test
+│   ├── bounding_box_train
+│   └── ...
+└── OptiSar_Pair
+    ├── 0001
+    ├── 0002
+    └── ...
+```
+![HOSS ReID Dataset structure](https://github.com/Alioth2000/TransOSS/raw/main/figs/dataset.png)
+
+## Pipeline
+
+![TransOSS Framework](https://github.com/Alioth2000/TransOSS/raw/main/figs/framework.png)
+
+## Requirements and Installation
+
+The Python version used is 3.9, and the PyTorch version is 2.2.2. It is recommended not to use versions lower than these.
+
+```bash
+pip install -r requirements.txt
+```
+
+## Sample Usage (Feature Extraction)
+
+You can use the `TransOSS` model to extract features from optical and SAR images for re-identification. While the original repository uses a custom PyTorch setup, if the model is integrated into the Hugging Face `transformers` ecosystem, you can use the following snippet:
+
+```python
+import torch
+from PIL import Image
+from transformers import AutoImageProcessor, AutoModel
+
+# Model ID on Hugging Face Hub
+model_id = "Alioth2000/TransOSS"
+
+try:
+    # Load processor and model
+    # For custom models, you might need trust_remote_code=True,
+    # or to load a specific model class (e.g., if TransOSS is a custom ViT).
+    processor = AutoImageProcessor.from_pretrained(model_id)
+    model = AutoModel.from_pretrained(model_id)
+
+    # Example: Create dummy images for demonstration
+    # In a real scenario, load your actual optical and SAR images using Image.open()
+    optical_image = Image.new('RGB', (256, 256), color='blue') # Placeholder for an optical image
+    sar_image = Image.new('RGB', (256, 256), color='red')     # Placeholder for a SAR image
+
+    # Preprocess images
+    inputs = processor(images=[optical_image, sar_image], return_tensors="pt")
+
+    # Move inputs to device (GPU if available)
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+    inputs = {k: v.to(device) for k, v in inputs.items()}
+    model.to(device)
+
+    # Extract features
+    model.eval() # Set model to evaluation mode
+    with torch.no_grad():
+        outputs = model(**inputs)
+
+    # Typically for feature extraction, the pooler_output or the CLS token's hidden state is used.
+    # Adjust based on the actual model output structure.
+    # Assuming the model returns pooler_output for a global feature vector.
+    optical_features = outputs.pooler_output[0] if hasattr(outputs, 'pooler_output') else outputs.last_hidden_state[:, 0, :][0]
+    sar_features = outputs.pooler_output[1] if hasattr(outputs, 'pooler_output') else outputs.last_hidden_state[:, 0, :][1]
+
+    print(f"Features from optical image (shape): {optical_features.shape}")
+    print(f"Features from SAR image (shape): {sar_features.shape}")
+
+    # For re-identification, you would compare these features, e.g., using cosine similarity:
+    # similarity = torch.nn.functional.cosine_similarity(optical_features, sar_features, dim=0)
+    # print(f"Cosine similarity: {similarity.item():.4f}")
+
+except Exception as e:
+    print(f"Failed to run sample usage. Please ensure `transformers` and `Pillow` are installed (`pip install transformers Pillow`).")
+    print(f"Also, confirm the model `Alioth2000/TransOSS` is compatible with `AutoModel`/`AutoImageProcessor` or refer to the official GitHub repository for detailed setup: {e}")
+```
+
+## Training and Evaluation
+
+For detailed instructions on pretraining and fine-tuning the model, please refer to the [official GitHub repository](https://github.com/Alioth2000/TransOSS).
+
+An example command for evaluation is provided below:
+
+```bash
+python test.py --config_file configs/hoss_transoss.yml MODEL.DEVICE_ID "('0')"  TEST.WEIGHT 'weights/HOSS_TransOSS.pth'
+```
+
+## Citation
+
+If you find this work useful, please cite the paper:
+
+```bibtex
+@misc{wang2025crossmodal,
+      title={Cross-modal Ship Re-Identification via Optical and SAR Imagery: A Novel Dataset and Method},
+      author={Han Wang and Shengyang Li and Jian Yang and Yuxuan Liu and Yixuan Lv and Zhuang Zhou},
+      year={2025},
+      eprint={2506.22027},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV},
+      url={https://arxiv.org/abs/2506.22027},
+}
+```
+
+## Acknowledgement
+
+Codebase from [TransReID](https://github.com/damo-cv/TransReID/tree/main), [reid-strong-baseline](https://github.com/michuanhaohao/reid-strong-baseline) , [pytorch-image-models](https://github.com/rwightman/pytorch-image-models).