stepfun-ai

/

Step-3.5-Flash-GGUF-Q4_K_S

like 120

Follow

StepFun 2.08k

I cloned and built https://github.com/stepfun-ai/Step-3.5-Flash according to instructions:

git clone git@github.com:stepfun-ai/Step-3.5-Flash.git
(
cd Step-3.5-Flash
cmake -S . -B build-macos \      
  -DCMAKE_BUILD_TYPE=Release \
  -DGGML_METAL=ON \
  -DGGML_ACCELERATE=ON \
  -DLLAMA_BUILD_EXAMPLES=ON \
  -DLLAMA_BUILD_COMMON=ON \
  -DGGML_LTO=ON && \
cmake --build build-macos -j8
)

Then I ran with the concatenated GGUF from the parts in this repo:

./Step-3.5-Flash/llama.cpp/build-macos/bin/llama-cli -m step3p5_flash_Q4_K_S.gguf -c 16384 -b 2048 -ub 2048 -fa on --temp 1.0 -p "What's your name?"

The result is that it keeps repeating the token = forever:

Weight sha256sum:

sha256sum step3p5_flash_Q4_K_S.gguf
cd3442c4e4c17322e6a432c0db1ec6383029f4f28c94e25c6f0b773b967b82db  step3p5_flash_Q4_K_S.gguf