Upload folder using huggingface_hub

README.md

@@ -1,3 +1,190 @@
 ---
 license: apache-2.0
+tags:
+  - geolocation
+  - image-regression
+  - dinov3
+  - street-view
+  - jakarta
+  - computer-vision
+library_name: pytorch
+language:
+  - id
+datasets:
+  - nadh0708/JKTSV-primary
 ---
+
+# Img2LocJakarta — DINOv3 street-view geolocation for Jakarta
+
+Predict **(latitude, longitude)** of a Google Street View perspective crop taken
+on a Jakarta road. A frozen **DINOv3 ViT-L/16** backbone extracts features; a
+U-shaped MLP head regresses a local flat-earth (x, y) offset which is converted
+to WGS-84 degrees.
+
+| | |
+|---|---|
+| **Input** | RGB street-level image (any size, resized to 224×224) |
+| **Output** | `{"lat": float, "lon": float}` — WGS-84 degrees |
+| **Scope** | Trained on Jakarta (5 administrative cities, motorway + primary roads) |
+| **Backbone** | DINOv3 ViT-L/16 — frozen; only the regression head was trained |
+
+## Quick Start
+
+```bash
+pip install torch torchvision pillow huggingface_hub safetensors
+```
+
+```python
+from huggingface_hub import hf_hub_download
+import sys, os
+
+# Download the inference code
+for fname in ["inference.py", "modeling_geotag.py"]:
+    hf_hub_download("nadh0708/Img2LocJakarta", fname, local_dir=".")
+
+from inference import GeoTagPredictor
+
+predictor = GeoTagPredictor("nadh0708/Img2LocJakarta")
+print(predictor.predict("street.jpg"))
+# {'lat': -6.2261, 'lon': 106.8123}
+
+# Batched
+print(predictor.predict(["a.jpg", "b.jpg"]))
+```
+
+### From a local checkpoint
+
+```python
+predictor = GeoTagPredictor("modelD_40e_2.pth")
+```
+
+### CLI
+
+```bash
+python inference.py nadh0708/Img2LocJakarta street.jpg
+```
+
+### Low-level API
+
+```python
+import torch
+from torchvision import transforms
+from PIL import Image
+from modeling_geotag import DinoGeoRegressor
+
+model = DinoGeoRegressor.from_pretrained("nadh0708/Img2LocJakarta")
+model.eval()
+
+tf = transforms.Compose([
+    transforms.Resize((224, 224)),
+    transforms.ToTensor(),
+    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
+])
+x = tf(Image.open("street.jpg").convert("RGB")).unsqueeze(0)
+lonlat = model.predict_lonlat(x)   # tensor([[lon, lat]])
+```
+
+## Architecture
+
+```
+DinoGeoRegressor
+ ├─ encoder : DINOv3 ViT-L/16  (frozen, patch_size=16)
+ │            last block tokens (B, 201, 1024) ──flatten──► (B, 205824)
+ │            tokens: 196 patch + 1 CLS + 4 storage
+ └─ head    : UNet-MLP
+              205824 ──enc1──► 512 ──enc2──► 512
+                                            ↓ bottleneck 512
+              512 ◄──dec2──  1024 (cat)
+              512 ◄──dec1──  1024 (cat)    skip connections from enc1, enc2
+               ↓  out linear
+               2  (x_metres, y_metres)
+              ──coord_convert──►  (lon, lat) degrees
+```
+
+The output is a flat-earth offset in metres relative to Jakarta origin
+`(lon 106.828320, lat -6.227468)`, inverted to degrees at inference time.
+
+The published checkpoint bundles the **frozen backbone + trained head** — no
+separate LVD-1689M weight download is required. Only the DINOv3 **code** is
+pulled from `torch.hub` on first load.
+
+## Performance
+
+Evaluated on **137,173 test samples** (Google Street View perspective crops of
+Jakarta roads, 8 headings: 0°–315° in 45° steps).
+
+| Epoch | Mean error | Median error | % < 1 km | % < 5 km | % < 25 km |
+|-------|-----------|-------------|---------|---------|---------|
+| e15 | 3.313 km | 1.966 km | 25.6% | 79.4% | 100.0% |
+| e35 | 3.158 km | 1.588 km | 33.8% | 80.2% | 100.0% |
+| **e40** | **2.743 km** | **1.272 km** | **42.2%** | **83.1%** | **100.0%** |
+
+Error is geodesic (haversine) distance between the predicted and true GPS
+coordinate. **This checkpoint is epoch 40** (`modelD_40e_2.pth`).
+
+## Training
+
+| Hyperparameter | Value |
+|---|---|
+| Backbone | DINOv3 ViT-L/16 (`facebookresearch/dinov3`) — frozen |
+| Head | UNet-MLP, hidden 512, skip connections |
+| Optimiser | AdamW |
+| Learning rate | 1e-3 |
+| Weight decay | 1e-4 |
+| Batch size | 64 |
+| Epochs | 40 |
+| Input size | 224 × 224 |
+| Loss | MSE on flat-earth (x, y) metres |
+
+**Dataset:** [`nadh0708/JKTSV-primary`](https://huggingface.co/datasets/nadh0708/JKTSV-primary) —
+685,848 perspective crops (548,675 train / 137,173 test) sampled along
+Jakarta motorway and primary road segments at 50 m intervals, 8 headings.
+
+## Preprocessing
+
+Images are resized to **224 × 224** and normalised with ImageNet statistics:
+
+```python
+transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
+```
+
+`GeoTagPredictor` applies this automatically. If you call `DinoGeoRegressor`
+directly, apply the transform before passing `pixel_values`.
+
+## Limitations
+
+- **Geographically restricted to Jakarta.** Out-of-domain images return
+  coordinates near the projection origin.
+- Trained on 2023+ Street View panoramas projected to 8 fixed headings
+  (0°, 45°, 90°, 135°, 180°, 225°, 270°, 315°).
+- Backbone is frozen; only the 3 M-parameter regression head was trained.
+- Hard examples (48% of the test set, defined as error > 1 km across all
+  measured epochs) tend to cluster in peripheral and coastal areas with
+  low visual distinctiveness.
+
+## Files
+
+| File | Purpose |
+|---|---|
+| `modeling_geotag.py` | Model classes + `from_pretrained` loader + coordinate conversion |
+| `inference.py` | `GeoTagPredictor` high-level API + CLI |
+| `model.safetensors` | Full state dict (frozen backbone + trained head, ~1.2 GB) |
+| `requirements.txt` | Runtime dependencies |
+
+## Citation
+
+If you use this model or the dataset, please cite:
+
+```bibtex
+@misc{img2locjakarta2025,
+  author = {Nadhif},
+  title  = {Img2LocJakarta: DINOv3 Street-View Geolocation for Jakarta},
+  year   = {2025},
+  url    = {https://huggingface.co/nadh0708/Img2LocJakarta}
+}
+```
+
+## License
+
+This model is released under the **Apache License 2.0**, consistent with the
+DINOv3 backbone license. See `LICENSE` for details.

inference.py

@@ -0,0 +1,102 @@
+"""
+High-level inference for the JKTSV DINOv3 geolocation model.
+
+Example
+-------
+    from inference import GeoTagPredictor
+
+    predictor = GeoTagPredictor("nadh0708/JKTSV-modelD")   # HF repo id
+    # ...or a local checkpoint:
+    predictor = GeoTagPredictor("model/modelD_40e_2.pth")
+
+    result = predictor.predict("street.jpg")
+    # {'lat': -6.21, 'lon': 106.84}
+
+    results = predictor.predict(["a.jpg", "b.jpg"])         # batched
+    # [{'lat': ..., 'lon': ...}, ...]
+
+The model was trained on Google Street View perspective crops of Jakarta roads
+(8 headings, 0-315 deg). Predictions are only meaningful for Jakarta street
+imagery.
+"""
+
+from __future__ import annotations
+
+from typing import Union
+
+import torch
+from PIL import Image
+from torchvision import transforms
+
+from modeling_geotag import DinoGeoRegressor
+
+_IMAGENET_MEAN = [0.485, 0.456, 0.406]
+_IMAGENET_STD = [0.229, 0.224, 0.225]
+
+ImageInput = Union[str, Image.Image]
+
+
+class GeoTagPredictor:
+    def __init__(
+        self,
+        model_id_or_path: str,
+        device: str | torch.device | None = None,
+        filename: str = "pytorch_model.bin",
+    ):
+        self.device = torch.device(
+            device or ("cuda" if torch.cuda.is_available() else "cpu")
+        )
+        self.model = DinoGeoRegressor.from_pretrained(
+            model_id_or_path, filename=filename, device=self.device
+        )
+        self.transform = transforms.Compose([
+            transforms.Resize((224, 224)),
+            transforms.ToTensor(),
+            transforms.Normalize(mean=_IMAGENET_MEAN, std=_IMAGENET_STD),
+        ])
+
+    def _load(self, image: ImageInput) -> Image.Image:
+        if isinstance(image, Image.Image):
+            return image.convert("RGB")
+        return Image.open(image).convert("RGB")
+
+    @torch.no_grad()
+    def predict(
+        self, images: Union[ImageInput, list[ImageInput]]
+    ) -> Union[dict, list[dict]]:
+        """Predict (lat, lon) for one image or a list of images."""
+        single = not isinstance(images, (list, tuple))
+        batch = [images] if single else list(images)
+
+        pixel_values = torch.stack([self.transform(self._load(im)) for im in batch])
+        pixel_values = pixel_values.to(self.device)
+
+        lonlat = self.model.predict_lonlat(pixel_values).cpu()
+        results = [
+            {"lat": float(lat), "lon": float(lon)}
+            for lon, lat in lonlat.tolist()
+        ]
+        return results[0] if single else results
+
+
+def _cli() -> None:
+    import argparse
+    import json
+
+    parser = argparse.ArgumentParser(description="Geolocate Jakarta street imagery.")
+    parser.add_argument("model", help="HF repo id or local checkpoint path")
+    parser.add_argument("images", nargs="+", help="image file path(s)")
+    parser.add_argument("--device", default=None)
+    parser.add_argument(
+        "--filename", default="pytorch_model.bin",
+        help="weights filename inside the HF repo",
+    )
+    args = parser.parse_args()
+
+    predictor = GeoTagPredictor(args.model, device=args.device, filename=args.filename)
+    out = predictor.predict(args.images)
+    print(json.dumps(out, indent=2))
+
+
+if __name__ == "__main__":
+    _cli()

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c4f4e2ba04dec22eeb1fc9f39ed442c7303c0e916b582fed4a7670302045f25
+size 1640787200

modeling_geotag.py

@@ -0,0 +1,192 @@
+"""
+Model definition + loading for the JKTSV DINOv3 geolocation regressor ("modelD").
+
+Architecture (must match the training checkpoint exactly):
+
+    DinoGeoRegressor
+      ├── encoder : DinoLastBlockEncoder
+      │              └── backbone : DINOv3 ViT-L/16 (frozen)
+      │                 forward = run backbone, grab the LAST transformer block
+      │                 output (B, 201, 1024) via a forward hook, then
+      │                 flatten -> (B, 205824)
+      └── head    : UNetMLPHead(embed_dim=205824, hidden_dim=512, out_dim=2)
+
+The head regresses a *local flat-earth (x, y) offset in metres* relative to a
+fixed Jakarta origin. `local_xy_to_lonlat` inverts that projection to recover
+(lon, lat) degrees. These constants are baked into the trained weights — do not
+change them for inference.
+
+The published checkpoint bundles the full (frozen) backbone weights together
+with the trained head, so loading needs only the DINOv3 *architecture* from
+``torch.hub`` (``pretrained=False``) — no separate LVD-1689M download.
+"""
+
+from __future__ import annotations
+
+import os
+from typing import Optional
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+# --- constants that are part of the trained model -----------------------------
+
+DINOV3_REPO = "facebookresearch/dinov3"
+BACKBONE_NAME = "dinov3_vitl16"
+EMBED_DIM = 205824        # 201 tokens (1 CLS + 4 storage + 196 patch) * 1024
+HIDDEN_DIM = 512
+OUT_DIM = 2
+
+# Local flat-earth projection origin (Jakarta city centre) used during training.
+ORIGIN_LON = 106.828320
+ORIGIN_LAT = -6.227468
+EARTH_RADIUS_M = 6371000.0
+
+
+# --- coordinate conversion -----------------------------------------------------
+
+def local_xy_to_lonlat(xy_meters: torch.Tensor) -> torch.Tensor:
+    """Invert the flat-earth projection used as the regression target.
+
+    Args:
+        xy_meters: (B, 2) tensor of [x (east), y (north)] in metres.
+
+    Returns:
+        (B, 2) tensor of [lon, lat] in degrees.
+    """
+    lat0_rad = torch.deg2rad(torch.tensor(ORIGIN_LAT, device=xy_meters.device))
+    x = xy_meters[:, 0]
+    y = xy_meters[:, 1]
+
+    dlon_rad = x / (EARTH_RADIUS_M * torch.cos(lat0_rad))
+    dlat_rad = y / EARTH_RADIUS_M
+
+    lon = torch.rad2deg(dlon_rad) + ORIGIN_LON
+    lat = torch.rad2deg(dlat_rad) + ORIGIN_LAT
+    return torch.stack([lon, lat], dim=-1)
+
+
+# --- modules -------------------------------------------------------------------
+
+class UNetMLPHead(nn.Module):
+    """U-shaped MLP with 1-D skip connections. Input (B, embed_dim) -> (B, out_dim)."""
+
+    def __init__(self, embed_dim: int, hidden_dim: int, out_dim: int):
+        super().__init__()
+        self.enc1 = nn.Linear(embed_dim, hidden_dim)
+        self.enc2 = nn.Linear(hidden_dim, hidden_dim)
+        self.bottleneck = nn.Linear(hidden_dim, hidden_dim)
+        self.dec2 = nn.Linear(hidden_dim * 2, hidden_dim)
+        self.dec1 = nn.Linear(hidden_dim * 2, hidden_dim)
+        self.out = nn.Linear(hidden_dim, out_dim)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        e1 = F.gelu(self.enc1(x))
+        e2 = F.gelu(self.enc2(e1))
+        b = F.gelu(self.bottleneck(e2))
+        d2 = F.gelu(self.dec2(torch.cat([b, e2], dim=-1)))
+        d1 = F.gelu(self.dec1(torch.cat([d2, e1], dim=-1)))
+        return self.out(d1)
+
+
+class DinoLastBlockEncoder(nn.Module):
+    """Run a DINOv3 ViT and return the flattened token sequence of its last block.
+
+    A forward hook captures the last transformer block output (B, N, C); the
+    tokens are flattened to (B, N*C). The backbone is frozen.
+    """
+
+    def __init__(self, backbone: nn.Module):
+        super().__init__()
+        self.backbone = backbone
+        self._last_block_out = None
+        self.backbone.blocks[-1].register_forward_hook(self._hook)
+        for p in self.backbone.parameters():
+            p.requires_grad = False
+
+    def _hook(self, module, inputs, output):
+        self._last_block_out = output
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        self._last_block_out = None
+        _ = self.backbone(x)
+        feats = self._last_block_out[0]   # (B, N, C)
+        return feats.flatten(start_dim=1)  # (B, N*C)
+
+
+class DinoGeoRegressor(nn.Module):
+    """Frozen DINOv3 encoder + trainable UNet-MLP regression head.
+
+    forward(pixel_values) -> (B, 2) local (x, y) metres.
+    predict_lonlat(pixel_values) -> (B, 2) [lon, lat] degrees.
+
+    `pixel_values` must already be resized to 224x224 and ImageNet-normalised
+    (see ``GeoTagPredictor`` / the transform in ``inference.py``).
+    """
+
+    def __init__(
+        self,
+        backbone: nn.Module,
+        embed_dim: int = EMBED_DIM,
+        hidden_dim: int = HIDDEN_DIM,
+        out_dim: int = OUT_DIM,
+    ):
+        super().__init__()
+        self.encoder = DinoLastBlockEncoder(backbone)
+        self.head = UNetMLPHead(embed_dim, hidden_dim, out_dim)
+
+    def forward(self, pixel_values: torch.Tensor) -> torch.Tensor:
+        feats = self.encoder(pixel_values)
+        return self.head(feats)
+
+    @torch.no_grad()
+    def predict_lonlat(self, pixel_values: torch.Tensor) -> torch.Tensor:
+        return local_xy_to_lonlat(self.forward(pixel_values))
+
+    # -- construction helpers --------------------------------------------------
+
+    @staticmethod
+    def build_backbone(device: str | torch.device = "cpu") -> nn.Module:
+        """Instantiate the DINOv3 ViT-L/16 architecture (no pretrained download)."""
+        backbone = torch.hub.load(
+            DINOV3_REPO, BACKBONE_NAME, pretrained=False, trust_repo=True
+        )
+        return backbone.to(device)
+
+    @classmethod
+    def from_pretrained(
+        cls,
+        model_id_or_path: str,
+        *,
+        filename: str = "pytorch_model.bin",
+        device: str | torch.device = "cpu",
+        backbone: Optional[nn.Module] = None,
+    ) -> "DinoGeoRegressor":
+        """Load weights from a local ``.pth``/``.bin`` file or a HuggingFace repo id.
+
+        The checkpoint is a full state_dict with ``encoder.backbone.*`` and
+        ``head.*`` keys (i.e. it includes the frozen backbone weights).
+        """
+        if os.path.isfile(model_id_or_path):
+            weights_path = model_id_or_path
+        else:
+            from huggingface_hub import hf_hub_download
+
+            weights_path = hf_hub_download(repo_id=model_id_or_path, filename=filename)
+
+        if backbone is None:
+            backbone = cls.build_backbone(device)
+
+        model = cls(backbone).to(device)
+
+        if weights_path.endswith(".safetensors"):
+            from safetensors.torch import load_file
+
+            state_dict = load_file(weights_path, device=str(device))
+        else:
+            state_dict = torch.load(weights_path, map_location=device)
+
+        model.load_state_dict(state_dict, strict=True)
+        model.eval()
+        return model

requirements.txt

@@ -0,0 +1,6 @@
+torch>=2.2
+torchvision>=0.17
+pillow
+huggingface_hub
+termcolor
+safetensors