Update README.md

README.md

@@ -40,6 +40,20 @@ transformers.AutoModelForCausalLM.from_pretrained
 
 
 This is the model card of a 🤗 transformers model that has been pushed on the Hub. 
+Previous vision models have been 50/50 as the multimodel model actully requires a lot of memory and gpu and harddrive space to create;
+the past versions have been attempts to Merge the capabilitys into the main mistral model whilst still retaining its mistral tag!
+After reading many hugging face articles: 
+
+The BackBone Issue is the main cause of creating multi modals !:
+
+with the advent of tiny models we are able to leverage the decoder abilitys as a single expert-ish... within the model :
+by reducing the size to a fully trainined tiny model!
+this will only produce decodings and not conversations so it needs to be smart and respond with defined answers: but in general it will produce captions: but as domain based it may be specialized in medical or art etc:
+
+The main llm still needs to retain these models within hence the back bone method of instigating a VisionEncoderDecoder model: istead of a llava model which still need wrangling to work correctly without spoiling the original transformers installation:
+Previous experiments proved that the mistral large model could be used as a decoder but the total model jumped to 13b so the when applying the tiny model it was only effected by the weight of the model 248M
+
+
 
 This is an experiment in vision - the model has been created as a mistral/VisionEncoder/Decoder 
 
@@ -101,6 +115,17 @@ generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)
 
 ## Training Details
 
+Currently inputs are raw and untrained ; 
+
+ie: they NEED to be trained as the tensors are randomize maybe? 
+despite using pretrained starting blocks. the encoder decoder modules are ready to be placed in train mode: 
+The main model ie the LLM will need lora/Qlora/Peft etc: 
+
+This model will stay in this state as a base training point ! so later versions will be trained; 
+This model is fully usable and still expected to score well ;
+
+The small tiny mistral is also a great performer and a great block to begin a smaller experts model (later) or any multimodal project ie: its like a mini pretrined bert/llama(Mistral is a clone of llamaAlpaca!
+
 ```python