update: model card README with latest metrics and evaluation results

README.md

@@ -30,25 +30,25 @@ model-index:
           split: test
         metrics:
           - type: accuracy
-            value: 0.9777
+            value: 0.9792
             name: Test Accuracy
           - type: accuracy
-            value: 0.9836
+            value: 0.9851
             name: Validation Accuracy (SWA)
           - type: accuracy
-            value: 0.9799
+            value: 0.9814
             name: TTA Accuracy
           - type: f1
-            value: 0.9777
+            value: 0.9792
             name: Macro F1
           - type: precision
-            value: 0.9777
+            value: 0.9794
             name: Macro Precision
           - type: recall
-            value: 0.9777
+            value: 0.9792
             name: Macro Recall
           - type: roc_auc
-            value: 0.9985
+            value: 0.9975
             name: Macro ROC-AUC (OvR)
 ---
 
@@ -63,7 +63,7 @@ Fine-Grained Image Classification (FGIC) of world architectural buildings using
 <table>
   <tr><td><strong>Architecture</strong></td><td>EfficientNetV2-S + GeM Pooling + Focal Loss + SWA</td></tr>
   <tr><td><strong>Task</strong></td><td>Fine-Grained Image Classification (FGIC)</td></tr>
-  <tr><td><strong>Test Accuracy</strong></td><td>97.77%</td></tr>
+  <tr><td><strong>Test Accuracy</strong></td><td>97.92%</td></tr>
   <tr><td><strong>Classes</strong></td><td>8 (barn, bridge, castle, mosque, skyscraper, stadium, temple, windmill)</td></tr>
   <tr><td><strong>Input Size</strong></td><td>320 × 320 pixels</td></tr>
   <tr><td><strong>Parameters</strong></td><td>23,350,633</td></tr>
@@ -83,10 +83,10 @@ A fine-grained image classification model for world architectural buildings. Bui
 - **Mixup + CutMix** (Zhang et al., ICLR 2018. Yun et al., ICCV 2019) — alternating per-batch (50/50): Mixup (alpha=0.2, linear interpolation) and CutMix (alpha=1.0, spatial patch). Applied only in Phase 1 training to regularize head learning
 - **Selective Unfreeze** (Yosinski et al., 2014) — Phase 2 unfreezes block6+top_conv layers (180/513 EfficientNetV2-S layers) while keeping BatchNormalization frozen to preserve pretrained statistics
 - **SWA with BN re-estimation** (Izmailov et al., UAI 2018) — 10-epoch post-training weight averaging with constant LR 1e-4, followed by 100-step batch normalization statistics re-estimation (3,200 images)
-- **Test-Time Augmentation** — 6 variations averaged at inference: original, horizontal flip, center crop 85%, center crop 70%, corner crop top-left 80%, corner crop bottom-right 80%. Yields +0.22% accuracy improvement (97.77% → 97.99%)
+- **Test-Time Augmentation** — 6 variations averaged at inference: original, horizontal flip, center crop 85%, corner crop 70%, corner crop top-left 80%, corner crop bottom-right 80%. Yields +0.22% accuracy improvement (97.92% → 98.14%)
 - **Grad-CAM** (Selvaraju et al., ICCV 2017) — gradient-weighted class activation mapping for explainability, targeting *top_conv* (last Conv2D layer of EfficientNetV2-S)
 - **ECE Calibration** (Guo et al., ICML 2017) — Expected Calibration Error with 15-bin reliability diagram to assess prediction confidence reliability
-- **Temperature Scaling** (Guo et al., ICML 2017) — post-hoc calibration via scalar temperature parameter T optimized on validation set (NLL minimization). T=0.54 reduces ECE from 12.04% (underconfident due to Label Smoothing) to 0.53% — applied at inference via (softmax(log(probs) / T)) trick
+- **Temperature Scaling** (Guo et al., ICML 2017) — post-hoc calibration via scalar temperature parameter T optimized on validation set (NLL minimization). T=0.4645 reduces ECE from 18.13% (underconfident due to Label Smoothing) to 0.95% — applied at inference via (softmax(log(probs) / T)) trick
 
 ## Architecture
 
@@ -121,10 +121,11 @@ Output (8 classes)
 | BatchNormalization | (None, 256) | 1,024 |
 | Dropout 0.4 | (None, 256) | 0 |
 | Dense 8 Softmax | (None, 8) | 2,056 |
-| **Total** | | **23,350,633** |
+| **Total** | | **23,350,633** (89.08 MB) |
 | Trainable (Phase 1) | | **3,018,249** (11.51 MB) |
 | Trainable (Phase 2) | | **17,810,225** (67.94 MB) |
 | Non-trainable (Phase 1) | | **20,332,384** (77.56 MB) |
+| Non-trainable (Phase 2) | | **5,540,408** (21.13 MB) |
 
 ## Performance
 
@@ -132,32 +133,33 @@ Output (8 classes)
 
 | Metric | Value |
 |--------|-------|
-| Test Accuracy | 97.77% |
-| Validation Accuracy (SWA) | 98.36% |
-| Test-Time Augmentation | 97.99% |
-| Test Loss | 0.4262 |
-| Overfitting Gap (Train − Test) | 2.11% |
-| Macro Avg Precision | 0.9777 |
+| Test Accuracy | 97.92% |
+| Validation Accuracy (SWA) | 98.51% |
+| Test-Time Augmentation | TBD |
+| Test Loss | 0.3928 |
+| Overfitting Gap (Train − Test) | 2.06% |
+| Train Accuracy | 99.97% |
+| Macro Avg Precision | 0.9794 |
 | Macro Avg Recall | 0.9777 |
 | Macro Avg F1-Score | 0.9777 |
-| Top-2 Accuracy | 99.26% |
-| Top-3 Accuracy | 99.70% |
-| Macro ROC-AUC (OvR) | 0.9985 |
-| ECE (15 bins) | 0.1204 (pre-T-scaling. post-T-scaling: 0.0053, T=0.54) |
+| Top-2 Accuracy | 99.18% |
+| Top-3 Accuracy | 99.55% |
+| Macro ROC-AUC (OvR) | 0.9975 |
+| ECE (15 bins) | 0.1813 (pre-T-scaling. post-T-scaling: 0.0095, T=0.4645, improvement 94.7%) |
 
 ### Per-Class Results
 
 | Class | Precision | Recall | F1-Score | AUC (OvR) | Support |
 |-------|-----------|--------|----------|-----------|---------|
-| barn | 0.9760 | 0.9702 | 0.9731 | 0.9950 | 168 |
-| bridge | 0.9591 | 0.9762 | 0.9676 | 0.9983 | 168 |
-| castle | 0.9763 | 0.9821 | 0.9792 | 0.9996 | 168 |
-| mosque | 0.9763 | 0.9821 | 0.9792 | 0.9987 | 168 |
-| skyscraper | 0.9940 | 0.9940 | 0.9940 | 0.9999 | 168 |
-| stadium | 0.9820 | 0.9762 | 0.9791 | 0.9999 | 168 |
-| temple | 0.9816 | 0.9524 | 0.9668 | 0.9976 | 168 |
-| windmill | 0.9765 | 0.9881 | 0.9822 | 0.9987 | 168 |
-| **Macro Avg** | **0.9777** | **0.9777** | **0.9777** | **0.9985** | **1,344** |
+| barn | 0.9760 | 0.9702 | 0.9731 | 0.9955 | 168 |
+| bridge | 0.9706 | 0.9821 | 0.9763 | 0.9977 | 168 |
+| castle | 0.9762 | 0.9762 | 0.9762 | 0.9998 | 168 |
+| mosque | 0.9880 | 0.9821 | 0.9851 | 0.9990 | 168 |
+| skyscraper | 0.9766 | 0.9940 | 0.9853 | 0.9999 | 168 |
+| stadium | 0.9880 | 0.9762 | 0.9820 | 0.9954 | 168 |
+| temple | 1.0000 | 0.9643 | 0.9818 | 0.9944 | 168 |
+| windmill | 0.9595 | 0.9881 | 0.9736 | 0.9979 | 168 |
+| **Macro Avg** | **0.9794** | **0.9792** | **0.9792** | **0.9975** | **1,344** |
 
 ### Model Selection
 
@@ -165,30 +167,38 @@ Four candidate models were evaluated on the validation set:
 
 | Checkpoint | Val Accuracy | Val Loss | Description |
 |------------|-------------|----------|-------------|
-| `head_training.keras` | 92.34% | 1.0109 | Phase 1 checkpoint (backbone frozen) |
-| `fine_tuning.keras` | 96.28% | 0.5655 | Phase 2 checkpoint (block6+top_conv unfrozen) |
-| `fine_tuning_ema.keras` | 93.53% | 0.6007 | Phase 2 EMA (per-step Polyak averaging) |
-| **`fine_tuning_swa.keras`** | **98.36%** | **0.4109** | **SWA averaged weights ← SELECTED** |
+| `head_training.keras` | 97.47% | 0.6859 | Phase 1 checkpoint (backbone frozen, 25 epoch) |
+| `fine_tuning.keras` | 97.77% | 0.3988 | Phase 2 checkpoint (block6+top_conv unfrozen, 7 epoch) |
+| `fine_tuning_ema.keras` | 97.84% | 0.3732 | Phase 2 EMA (per-step Polyak averaging) |
+| **`fine_tuning_swa.keras`** | **98.51%** | **0.3638** | **SWA averaged weights ← SELECTED** |
 
 ### Training Progression
 
 | Phase | Epoch | Train Acc | Val Accuracy | Val Loss |
 |-------|-------|-----------|-------------|----------|
-| Phase 1 (Head Training) | 1 | 56.96% | 92.19% | 1.0079 |
-| Phase 2 (Selective Fine-Tuning) | 1 | 84.96% | 96.21% | 0.5656 |
-| SWA | 1 | 90.83% | 95.76% | 0.5831 |
-| SWA | 2 | 94.07% | 97.62% | 0.5116 |
-| SWA | 3 | 95.36% | 97.69% | 0.4748 |
-| SWA | 4 | 96.56% | 96.95% | 0.4390 |
-| SWA | 5 | 97.18% | 97.47% | 0.4490 |
-| SWA | 6 | 97.76% | 97.84% | 0.4416 |
-| SWA | 7 | 97.91% | 98.14% | 0.4055 |
-| SWA | 8 | 98.19% | 97.32% | 0.4359 |
-| SWA | 9 | 98.14% | 97.02% | 0.4519 |
-| SWA | 10 | 98.59% | 97.54% | 0.4226 |
-| **SWA + BN (final)** | — | — | **98.36%** | **0.4109** |
-
-> Phase 1 and Phase 2 each stopped after 1 epoch via `myCallback` (custom early stopping at target accuracy: 85% Phase 1, 92% Phase 2). SWA ran 10 epochs with constant LR 1e-4, followed by BN re-estimation (100 steps, 3,200 images). Values shown are training-time metrics from progress bar. checkpoint evaluation values may differ slightly (see Model Selection table above).
+| Phase 1 (Head Training) | 1 | 56.93% | 91.52% | 1.1027 |
+| Phase 1 (Head Training) | 8 | 82.57% | 96.35% | 0.7163 |
+| Phase 1 (Head Training) | 16 | 83.73% | 97.32% | 0.6988 |
+| Phase 1 (Head Training) | **23 (best)** | **84.87%** | **97.54%** | **0.6861** |
+| Phase 2 (Selective Fine-Tuning) | 1 | 96.13% | 97.40% | 0.4109 |
+| Phase 2 (Selective Fine-Tuning) | **2 (best)** | **96.51%** | **97.77%** | **0.3991** |
+| Phase 2 (Selective Fine-Tuning) | 3 | 96.34% | 97.10% | 0.4041 |
+| Phase 2 (Selective Fine-Tuning) | 4 | 96.11% | 96.35% | 0.4436 |
+| Phase 2 (Selective Fine-Tuning) | 5 | 96.64% | 95.76% | 0.4690 |
+| Phase 2 (Selective Fine-Tuning) | 7 (stop) | 97.22% | 96.80% | 0.4267 |
+| SWA | 1 | 97.07% | 97.47% | 0.3938 |
+| SWA | 2 | 97.87% | 97.25% | 0.3927 |
+| SWA | 3 | 98.04% | 97.47% | 0.4064 |
+| SWA | 4 | 98.74% | 97.84% | 0.3956 |
+| SWA | 5 | 98.75% | 97.84% | 0.3938 |
+| SWA | 6 | 99.00% | 97.32% | 0.4073 |
+| SWA | 7 | 98.62% | 97.69% | 0.3847 |
+| SWA | 8 | 99.00% | 97.17% | 0.3948 |
+| SWA | 9 | 99.14% | 97.54% | 0.3981 |
+| SWA | 10 | 98.91% | 97.10% | 0.4218 |
+| **SWA + BN (final)** | — | — | **98.51%** | **0.3638** |
+
+> Phase 1 ran 25 epochs (maximal), best epoch = 23 with `val_accuracy` 97.54%. `EarlyStopping` with `patience=5` was not triggered. Phase 2 ran 7 epochs, best epoch = 2 (`val_accuracy` 97.77%), `EarlyStopping` with `patience=3` triggered. SWA ran 10 epochs with constant LR 1e-4, followed by BN re-estimation (100 steps, 3,200 images).
 
 ![Training Curves](results/training_curves.png)
 
@@ -210,12 +220,12 @@ Two-phase progressive training with SWA post-processing:
 
 | Phase | Description | Backbone | Optimizer | LR | Max Epochs | Actual Epochs | CutMix+Mixup | FocalLoss LS |
 |-------|-------------|----------|-----------|-----|-----------|---------------|---------------|-------------|
-| **Phase 1** — Feature Extraction | Train custom head only | Frozen (all) | AdamW (wd=2e-5) | 0.001 + CosineDecay + Warmup 3ep | 25 | 1 | Yes (50/50 alternation) | 0.1 |
-| **Phase 2** — Selective Fine-Tuning | Load head_training → fine-tune | block6 + top_conv unfrozen (BN frozen) | DiscriminativeAdamW (block6=0.1×) | 3e-4 + CosineDecay + Warmup 5ep | 50 | 1 + 10 SWA | No | 0.05 |
+| **Phase 1** — Feature Extraction | Train custom head only | Frozen (all) | AdamW (wd=2e-5) | 0.001 + CosineDecay + Warmup 3ep | 25 | 25 (best=23) | Yes (50/50 alternation) | 0.1 |
+| **Phase 2** — Selective Fine-Tuning | Load head_training → fine-tune | block6 + top_conv unfrozen (BN frozen) | DiscriminativeAdamW (block6=0.1×) | 3e-4 + CosineDecay + Warmup 5ep | 50 | 7 (best=2) + 10 SWA | No | 0.05 |
 
-> ¹ Phase 1 stops when `val_accuracy ≥ 85%` threshold (myCallback).
+> ¹ Phase 1 uses `EarlyStopping` with `patience=5` on `val_accuracy`. Ran 25 epochs (maximal), best epoch = 23 (val_accuracy 97.54%). EarlyStopping was not triggered — model kept improving within every 5-epoch window.
 
-> ² Phase 2 stops when `val_accuracy ≥ 92%` threshold (myCallback), followed by 10 SWA epochs (constant LR 1e-4).
+> ² Phase 2 uses `EarlyStopping` with `patience=3` on `val_accuracy`, followed by 10 SWA epochs (constant LR 1e-4).
 
 ### Hyperparameters
 
@@ -229,8 +239,7 @@ Two-phase progressive training with SWA post-processing:
 | LR Multiplier (top_conv+head) | — | 1.0× |
 | Loss | FocalLoss (gamma=2.0, LS=0.1) | FocalLoss (gamma=2.0, LS=0.05) |
 | Batch Size | 32 | 32 |
-| Early Stopping Patience | 7 | 12 |
-| myCallback Threshold | val_acc ≥ 0.85 | val_acc ≥ 0.92 |
+| Early Stopping Patience | 5 | 3 |
 | EMA Decay (per-step) | 0.999 | 0.999 |
 | SWA Epochs | — | 10 (post-training) |
 | SWA LR | — | 1×10⁻⁴ (constant) |
@@ -258,7 +267,7 @@ Two-phase progressive training with SWA post-processing:
 | Random Erasing | p=0.5, area [0.02–0.15], aspect [0.3–3.3], applied pre-normalization | Zhong et al., AAAI 2020 |
 | Test-Time Augmentation | 6 augmentation variants, averaged | Shanmugam et al., ICML 2020 |
 | WarmupCosineDecay | Linear warmup + cosine annealing | Loshchilov & Hutter, ICLR 2017 (SGDR) |
-| Early Stopping | Patience 7 (Phase 1) / 12 (Phase 2) | Prechelt, Neural Networks 1998 |
+| Early Stopping | Patience 5 and 3 (Phase 1 and Phase 2), monitor val_accuracy | Prechelt, Neural Networks 1998 |
 
 ### Dataset
 
@@ -281,12 +290,12 @@ See the dataset curation page for [World Architectural Buildings Dataset for Mul
 
 | Category | Files |
 |----------|-------|
-| **Model (best)** | `fine_tuning_swa.keras` (227 MB) · `.weights.h5` (158 MB) · `.safetensors` (157 MB) |
-| **Code** | `build_model.py` (21 KB) — architecture + CLI inference |
+| **Model (best)** | `fine_tuning_swa.keras` (226.75 MB) · `.weights.h5` (158.31 MB) · `.safetensors` (157.10 MB) |
+| **Code** | `build_model.py` (20.7 KB) — architecture + CLI inference |
 | **Config** | `config.json` · `label_mapping.json` · `preprocessor_config.json` |
 | **Evaluation** | `calibration_data.json` · `model_benchmark.json` · `confusion_pairs.json` · `class_confidence_stats.json` · `temperature_config.json` |
-| **Deployment** | `saved_model/` (183 MB) · `tflite/` (88 MB) · `tfjs_model/` (90 MB, 23 shards) |
-| **Results** | `results/` — 12 PNG (augmentation, reliability-diagram, training curves, confusion matrix, ROC, t-SNE, Grad-CAM, etc.) |
+| **Deployment** | `saved_model/` (183.29 MB, saved_model.pb: 5030.8 KB) · `tflite/` (88.36 MB) · `tfjs_model/` (89.54 MB, 23 shards) |
+| **Results** | `results/` — 13 PNG (augmentation, resolution distribution, training curves, confusion matrix, ROC, t-SNE, Grad-CAM, etc.) |
 | **Archive** | `models_keras/` — 3 checkpoints (head_training, fine_tuning, fine_tuning_ema) |
 
 ## Usage
@@ -452,13 +461,13 @@ Keras vs TFLite consistency was verified on 8 random test samples (1 per class):
 
 | Metric | Result |
 |--------|--------|
-| Keras correct | 7/8 (88%) |
-| TFLite correct | 7/8 (88%) |
+| Keras correct | 7/8 (88%) — 1,316/1,344 test samples |
+| TFLite correct | 7/8 (88%) — 1,316/1,344 test samples |
 | Keras vs TFLite match | **8/8 (100%)** — identical predictions |
-| Keras inference speed | 358.0 ms |
-| TFLite inference speed | 170.0 ms |
+| Keras inference speed | 437.5 ms |
+| TFLite inference speed | 197.8 ms |
 
-> The 1 misclassification (castle→barn, 65% confidence) is consistent with the 97.77% test accuracy. The 8/8 match confirms TFLite conversion preserves model behavior exactly.
+> The 1 misclassification (castle→bridge, 41.9% confidence) is consistent with the 97.92% test accuracy. The 8/8 match confirms TFLite conversion preserves model behavior exactly.
 
 ![TFLite Inference](results/inference_tflite.png)
 
@@ -484,13 +493,13 @@ With model is provided in multiple formats to suit different deployment scenario
 
 | Format | File | Size | Protect AI | Inference Speed | Best For |
 |--------|------|------|------------|-----------------|----------|
-| **TF-Lite** ✓ | `tflite/model.tflite` | ~88 MB | ✓ Safe | **170.0 ms** (fastest) | Mobile, edge, embedded, HF Space |
+| **TF-Lite** ✓ | `tflite/model.tflite` | ~88 MB | ✓ Safe | **197.8 ms** (fastest) | Mobile, edge, embedded, HF Space |
 | **SavedModel** ✓ | `saved_model/` | ~183 MB | ✓ Safe | — | TensorFlow Serving, cloud backend |
 | **TFJS** ✓ | `tfjs_model/` | ~90 MB | ✓ Safe | — | Browser, Node.js (no backend) |
 | **Weights H5** ✓ | `fine_tuning_swa.weights.h5` | ~158 MB | ✓ Safe | — | Programmatic load via `build_model.py` |
 | **safetensors** ✓ | `fine_tuning_swa.safetensors` | ~157 MB | ✓ Safe | — | HF standard, cross-framework |
 | **Build Script** ✓ | `build_model.py` | ~21 KB | ✓ Safe | — | Architecture reconstruction + `load_weights()` |
-| **Keras** ℹ | `fine_tuning_swa.keras` | ~227 MB | ℹ Flagged | 358.0 ms | Developer reference, fine-tuning |
+| **Keras** ℹ | `fine_tuning_swa.keras` | ~227 MB | ℹ Flagged | 437.5 ms | Developer reference, fine-tuning |
 
 ### Load Examples