Instructions to use kumararvindibs/ibs_imgToTextGeneration with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kumararvindibs/ibs_imgToTextGeneration with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="kumararvindibs/ibs_imgToTextGeneration")

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("kumararvindibs/ibs_imgToTextGeneration")
model = AutoModelForMultimodalLM.from_pretrained("kumararvindibs/ibs_imgToTextGeneration", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use kumararvindibs/ibs_imgToTextGeneration with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kumararvindibs/ibs_imgToTextGeneration"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kumararvindibs/ibs_imgToTextGeneration",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/kumararvindibs/ibs_imgToTextGeneration

SGLang

How to use kumararvindibs/ibs_imgToTextGeneration with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kumararvindibs/ibs_imgToTextGeneration" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kumararvindibs/ibs_imgToTextGeneration",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kumararvindibs/ibs_imgToTextGeneration" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kumararvindibs/ibs_imgToTextGeneration",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use kumararvindibs/ibs_imgToTextGeneration with Docker Model Runner:
```
docker model run hf.co/kumararvindibs/ibs_imgToTextGeneration
```

kumararvindibs commited on Mar 11, 2024

Commit

cc479e4

verified ·

1 Parent(s): b29140f

Update handler.py

Browse files

Files changed (1) hide show

handler.py +6 -13

handler.py CHANGED Viewed

@@ -21,22 +21,15 @@ class EndpointHandler():
     def __call__(self, data: Dict[str, Any]) -> Dict[str, Any]:
         logging.error(f"----------This is an error message")
-        #logging.critical('-------------------This is a critical message')
-        print("000000--",type(data))
-        #logging.warning('--------This is a warning message')
-        print("input data is here------------",data)
         input_data = data.get("inputs", {})
-        logging.warning('------input_data--This is a warning message', input_data)
-        print("input data is here-2-----------",type(input_data))
         encoded_images = input_data.get("images")
-        logging.warning(f"---encoded_images-----This is a warning message {str(encoded_images)}")
-        print("input encoded_images is here------------",type(encoded_images))
         if not encoded_images:
             return {"captions": [], "error": "No images provided"}
-        #texts = input_data.get("texts", ["a photography of"] * len(encoded_images))
         try:
             byteImgIO = io.BytesIO()
             byteImg = Image.open(encoded_images[0])
@@ -48,7 +41,7 @@ class EndpointHandler():
             # Non test code
             dataBytesIO = io.BytesIO(byteImg)
             raw_images =[Image.open(dataBytesIO)]
-            logging.warning(f"----raw_images----This is a warning message {str(raw_images)}")
             # Check if any images were successfully decoded
             if not raw_images:
                 print("No valid images found.")
@@ -63,7 +56,7 @@ class EndpointHandler():
                 out = self.model.generate(**processed_inputs)
             captions = self.processor.batch_decode(out, skip_special_tokens=True)
-            logging.warning(f"----captions----This is a warning message {str(captions)}")
             print("caption is here-------",captions)
             return {"captions": captions}
         except Exception as e:

     def __call__(self, data: Dict[str, Any]) -> Dict[str, Any]:
         logging.error(f"----------This is an error message")
         input_data = data.get("inputs", {})
+        logging.warning(f"------input_data-- {str(input_data)}")
         encoded_images = input_data.get("images")
+        logging.warning(f"---encoded_images----- {str(encoded_images)}")
         if not encoded_images:
+            logging.warning(f"---encoded_images--not provided in if block--- {str(encoded_images)}")
             return {"captions": [], "error": "No images provided"}
         try:
+            logging.warning(f"---encoded_images-- provided in try block--- {str(encoded_images)}")
             byteImgIO = io.BytesIO()
             byteImg = Image.open(encoded_images[0])
             # Non test code
             dataBytesIO = io.BytesIO(byteImg)
             raw_images =[Image.open(dataBytesIO)]
+            logging.warning(f"----raw_images----0--- {str(raw_images)}")
             # Check if any images were successfully decoded
             if not raw_images:
                 print("No valid images found.")
                 out = self.model.generate(**processed_inputs)
             captions = self.processor.batch_decode(out, skip_special_tokens=True)
+            logging.warning(f"----captions---- {str(captions)}")
             print("caption is here-------",captions)
             return {"captions": captions}
         except Exception as e: