kashif HF Staff commited on
Commit
2114a42
·
verified ·
1 Parent(s): 163414c

Say what the checkpoint is for and how to regenerate it

Browse files
Files changed (1) hide show
  1. README.md +45 -0
README.md ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ tags:
4
+ - weathernext2
5
+ - tiny-random
6
+ - testing
7
+ ---
8
+
9
+ # Tiny random WeatherNext 2
10
+
11
+ A randomly initialized [`WeatherNext2ForWeatherForecasting`](https://huggingface.co/docs/transformers/model_doc/weathernext2)
12
+ for use in the Transformers test suite. 13K parameters, 152 KB. It forecasts nothing: the weights are
13
+ noise and the feature extractor's statistics are zero mean and unit variance.
14
+
15
+ What it is for is the **geometry**. WeatherNext 2 has no learned positional encodings, so the mesh,
16
+ both grid-mesh graphs and the banded attention mask are built once at conversion time, with `scipy`
17
+ and `trimesh`, and stored in the checkpoint as buffers. A model built from a config alone gets a
18
+ placeholder instead, which is enough to run but is not a mesh. This checkpoint carries the real
19
+ thing at a size CI can download, so the tests that depend on a real mesh have one.
20
+
21
+ The grid is 15 degrees and the mesh is a twice-split icosahedron: 162 mesh nodes over 312 grid
22
+ points, an attention bandwidth of 43, so four attention blocks. Four is the point, since it gives an
23
+ interior block with a neighbour on either side.
24
+
25
+ ## Regenerating it
26
+
27
+ ```python
28
+ import torch
29
+ from transformers import WeatherNext2Config, WeatherNext2FeatureExtractor, WeatherNext2ForWeatherForecasting
30
+ from transformers.models.weathernext2.convert_weathernext2_original_checkpoint import geometry_state_dict
31
+
32
+ config = WeatherNext2Config(
33
+ hidden_size=16, intermediate_size=32, num_hidden_layers=2, num_attention_heads=2,
34
+ edge_hidden_size=4, noise_channels=4, mesh_splits=2, attention_k_hop=2,
35
+ grid_latitudes=13, grid_longitudes=24, pressure_levels=(500, 850), aggregate_normalization=None,
36
+ )
37
+ geometry = geometry_state_dict(config) # needs scipy and trimesh; sets the two derived sizes
38
+
39
+ torch.manual_seed(0)
40
+ model = WeatherNext2ForWeatherForecasting(config)
41
+ model.load_state_dict(geometry, strict=False)
42
+ ```
43
+
44
+ The feature extractor is built from the same config with zero means and unit standard deviations for
45
+ every variable.