Ollama compatability
#13
by twine-network - opened
Is it possible to make a GGUF version of this model to use with Ollama? I haven't been able to find a tool or workflow for this. Wondering if it is even possible.
Not sure if ollama supports colbert-style models?
Ollama docs:
Coming soon
More features are coming to support workflows that involve embeddings:
Batch embeddings: processing multiple input data prompts simultaneously
OpenAI API Compatibility: support for the /v1/embeddings OpenAI-compatible endpoint
More embedding model architectures: support for ColBERT, RoBERTa, and other embedding model architectures
Oh nice. I had just come across your infinity project a few days ago and had it sitting in an open tab to look at when I got a minute. That just got bumped up the todo list. Thanks