Instructions to use Salesforce/blip-image-captioning-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Salesforce/blip-image-captioning-base with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("image-to-text", model="Salesforce/blip-image-captioning-base")# Load model directly from transformers import AutoProcessor, AutoModelForImageTextToText processor = AutoProcessor.from_pretrained("Salesforce/blip-image-captioning-base") model = AutoModelForImageTextToText.from_pretrained("Salesforce/blip-image-captioning-base") - Notebooks
- Google Colab
- Kaggle
How to make the description longer and more detailed?
#40
by oddball516 - opened
I have a small script that captions the image
import argparse
import glob
import os
from PIL import Image, UnidentifiedImageError
from transformers import pipeline
captioner = pipeline("image-to-text", "Salesforce/blip-image-captioning-large")
captionText = captioner(jpegFile, max_new_tokens=100)[0]["generated_text"]
However, the generated text is like 20 tokens, never reached 100 tokens. The description of the image is not "precise" enough, many details are missed.
e.g this picture generates araffe walking down the sidewalk in a city with a backpack
Many details like the backpack color, the background scene are missing, even with the large model. How can I ask blip to be more detailed?
Anyone?
Anyone?
used Salesforce/blip-image-captioning-large instead