Example code not working
#1
by karmiq - opened
Hello, when I try the example code listed on the page, I'm getting an error.
First, the mlx-examples/llms/hf_llm folder is not found. There's a mlx-examples/llms/mlx_lm folder, but when I run the command:
python generate.py --model mlx-community/multilingual-e5-small-mlx --prompt "My name is"
I'm getting an error:
Traceback (most recent call last):
File "/.../.../mlx-examples/llms/mlx_lm/generate.py", line 6, in <module>
from .utils import generate_step, load
ImportError: attempted relative import with no known parent package
Can you please give some advice about how to use the model?
Notices the same, all embedding instructions in mlx seem outdated. Use:
from mlx_embeddings.utils import load
# Load the model and tokenizer
model, tokenizer = load("mlx-community/multilingual-e5-small-mlx")
def get_embedding(texts, model, tokenizer):
inputs = tokenizer.batch_encode_plus(texts, return_tensors="mlx", padding=True, truncation=True, max_length=512)
outputs = model(
inputs["input_ids"],
attention_mask=inputs["attention_mask"]
)
return outputs.text_embeds # mean pooled and normalized embeddings
texts = [
"I like grapes",
"I like fruits",
"The slow green turtle crawls under the busy ant."
]
embeddings = get_embedding(texts, model, tokenizer)