Update README.md
Browse files
README.md
CHANGED
|
@@ -96,7 +96,8 @@ Note that during pretraining, the --mlm-prob parameter has been set to 0.4. We h
|
|
| 96 |
0.4 yielded the best results, in accordance to [Wettig et al., 2023](https://aclanthology.org/2023.eacl-main.217.pdf).
|
| 97 |
|
| 98 |
Estimated time it has cost to train:
|
| 99 |
-
|
|
|
|
| 100 |
finetuning: 4 hours on 1 NVIDIA H200
|
| 101 |
|
| 102 |
# Output
|
|
|
|
| 96 |
0.4 yielded the best results, in accordance to [Wettig et al., 2023](https://aclanthology.org/2023.eacl-main.217.pdf).
|
| 97 |
|
| 98 |
Estimated time it has cost to train:
|
| 99 |
+
|
| 100 |
+
pretraining: 38 hours on 4 NVIDIA H200s
|
| 101 |
finetuning: 4 hours on 1 NVIDIA H200
|
| 102 |
|
| 103 |
# Output
|