gabrielkasmi commited on
Commit
3047448
·
verified ·
1 Parent(s): 52bbc87

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -16
README.md CHANGED
@@ -52,32 +52,29 @@ Rules:
52
  - For Track 3, only the Google training split may be used for training.
53
 
54
  ---
55
-
56
  ## Results
57
 
58
  Models evaluated on the official test split (spatial holdout by French department — see dataset card for details).
59
 
60
  ### Segmentation (DeepLabV3-ResNet101)
61
 
62
- | Provider | IoU | F1 |
63
- |----------|-----|----|
64
- | Google | TBD | TBD |
65
- | IGN | TBD | TBD |
 
 
66
 
67
  ### Classification (InceptionV3)
68
 
69
- | Provider | Accuracy | F1 |
70
- |----------|----------|----|
71
- | Google | TBD | TBD |
72
- | IGN | TBD | TBD |
73
-
74
- ### Distribution shift benchmark
75
-
76
- Train on Google, evaluate on IGN — the intended cross-provider protocol:
77
 
78
- | Model | Train | Test | IoU |
79
- |-------|-------|------|-----|
80
- | DeepLabV3-ResNet101 | Google | IGN | TBD |
81
 
82
  ---
83
 
 
52
  - For Track 3, only the Google training split may be used for training.
53
 
54
  ---
 
55
  ## Results
56
 
57
  Models evaluated on the official test split (spatial holdout by French department — see dataset card for details).
58
 
59
  ### Segmentation (DeepLabV3-ResNet101)
60
 
61
+ | Train | Test | IoU | F1 | n (test) |
62
+ |-------|------|-----|----|----------|
63
+ | Google | Google | 0.884 | 0.937 | 1,935 |
64
+ | IGN | IGN | 0.735 | 0.844 | 1,239 |
65
+ | Google | IGN | 0.561 | 0.709 | 1,239 |
66
+ | IGN | Google | 0.657 | 0.786 | 1,935 |
67
 
68
  ### Classification (InceptionV3)
69
 
70
+ | Train | Test | Accuracy | Precision | Recall | F1 | n (test) |
71
+ |-------|------|----------|-----------|--------|----|----------|
72
+ | Google | Google | 0.952 | 0.990 | 0.912 | 0.949 | 3,884 |
73
+ | IGN | IGN | 0.640 | 0.831 | 0.309 | 0.451 | 2,593 |
74
+ | Google | IGN | 0.592 | 0.815 | 0.188 | 0.306 | 2,593 |
75
+ | IGN | Google | 0.543 | 1.000 | 0.083 | 0.153 | 3,884 |
 
 
76
 
77
+ **Note on classification cross-provider results:** the IGN-trained model collapses on Google imagery (Recall=0.08, Precision=1.0), indicating the model rarely predicts positives — a degenerate operating point. This illustrates the severity of the distribution shift documented in [Kasmi et al. (2025)](https://doi.org/10.1017/eds.2025.13).
 
 
78
 
79
  ---
80