Spaces:

TiGa-RCE
/

embedding-quantization-cuda-control

Running on Zero

App Files Files Community

embedding-quantization-cuda-control / README.md

TiGa-RCE

Add mounted 1.5B and 8B BF16 cross-runtime controls

7973967 verified 14 days ago

895 Bytes

title: Embedding Quantization CUDA Control
emoji: 🔬
colorFrom: blue
colorTo: yellow
sdk: gradio
sdk_version: 6.5.1
app_file: app.py
pinned: false
license: apache-2.0

Embedding Quantization CUDA Control

A bounded ZeroGPU control lane for the matched local MLX embedding quantization experiment. The first milestone reproduces the frozen Qwen3-Embedding-0.6B BF16 vectors on CUDA and compares them with the saved MLX BF16 vectors. CUDA results do not reproduce MLX/Metal performance.

The second milestone adds CUDA-native bitsandbytes INT8 and NF4 controls. They are deliberately reported as separate quantizers and are not treated as equivalents of MLX Q, oQ, or oQe formats.

Matched BF16 controls are also available for GTE-Qwen2-1.5B and Qwen3-Embedding-8B. Their CUDA vectors are compared against the corresponding saved MLX BF16 vectors from the private reproduction bucket.