Load model for sentiment analysis or classification
When I load the model for AutoModelForSequenceClassification I get an error:
ValueError: Unrecognized configuration class <class 'transformers_modules.phi-1_5.configuration_mixformer_sequential.MixFormerSequentialConfig'> for this kind of AutoModel: AutoModelForSequenceClassification.
Model type should be one of AlbertConfig, BartConfig, BertConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BloomConfig, CamembertConfig, CanineConfig, LlamaConfig, ...
Is it possible to load the model for different tasks than text generation, like semantic analysis or classification tasks?
Hello @ThetaPhiPsi ! I hope everything is going well with you.
It is possible, but you will need to implement a new head to the model. Right now, the only head we have is for Causal LM. However, you should be able to load a pre-trained model for Causal LM, remove the head, and attach a new head to the model, for example, a classification head.
You will also need to fine-tune the model on this new downstream task.
Regards,
Gustavo.
Hello Gustavo,
thank you very much for your reply.
I was a bit overwhelmed by Huggingface's configuration zoo and initially thought that all models offered the same interface, which understandably is not the case.
Thanks for the explanation, I know what to do!
Regards,
Jürgen
Hello @gugarosa
Thank you for the guidance. However, I am still facing challenges in implementing a classification head on the pretrained model. I was wondering if you could provide a more detailed walkthrough or code snippets on how to properly achieve that.
Thanx a lot !