Train mobileBERT from scratch for other languages
#1
by DecisionTreePark - opened
Hi,
I am thinking of training a mobileBERT model from scratch for the German language.
Can I use the English mobileBERT model to apply it to a dataset in another language?
It makes sense that I would have to adapt the teacher model of mobileBERT to a BERT model of the corresponding language. Unfortunately, I could not find a parameter to adapt the teacher model.
Are there any other ideas on how best to train a mobileBERT model for another language?
Many greetings and many thanks!
DecisionTreePark