Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

Misc

preference-optimization

Inference Endpoints

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

139

Base only

Active filters: preference-optimization

sophialan/pg-map-sd15

Text-to-Image • Updated 22 days ago

sophialan/pg-map-sdxl

Text-to-Image • Updated 22 days ago

sophialan/pg-map-sd3

Text-to-Image • Updated 22 days ago

sbc799/qwen2.5-1.5b-ultrafeedback-rdpo-adapters

Text Generation • Updated May 25

himanshunakrani9/qwen2.5-3b-ultrafeedback-dpo

JaydeepR/SmolLM-135M-neuraltxt-dpo-v1

Text Generation • 0.1B • Updated Jun 1 • 3

ingeol/mistral-7b-arc-cpt-dpo

Text Generation • 7B • Updated Jun 7 • 2

chayuto/gemma-4-e2b-klondike-orpo-research

yavuz-ai/qwen3-4b-dpo-ultrafeedback-lora

Text Generation • Updated about 1 month ago • 9

yavuz-ai/ministral-8b-dpo-ultrafeedback-lora

Text Generation • Updated about 1 month ago • 7

sarthaksenapati/qwen1.5b-hinglish-dpo

Text Generation • Updated 21 days ago • 53

sarthaksenapati/qwen1.5b-hinglish-ipo

Text Generation • Updated 19 days ago • 34

sarthaksenapati/qwen1.5b-hinglish-kto

Text Generation • Updated 19 days ago • 37

sillykiwi/Soren-1-Small-Q8_0-GGUF

Text Generation • 2B • Updated 18 days ago • 86

Keitsuna123/llama-3.2-1b-fc-dpo

Text Generation • 1B • Updated 10 days ago • 452

amibaid/acpo-videollama2

9B • Updated 10 days ago • 46

promotion/qwen3-8b-dpo-avg-beta0p01-s42

Text Generation • 308k • Updated 9 days ago • 632

promotion/qwen3-8b-dpo-avg-beta0p05-s42

Text Generation • 308k • Updated 9 days ago • 662

promotion/qwen3-8b-ronpo-full-expect-s42

Text Generation • 8B • Updated 7 days ago • 863

promotion/qwen3-8b-ronpo-konly-s42

Text Generation • 8B • Updated 9 days ago • 186

promotion/qwen3-8b-inpo-avg-eta0p01-s42

Text Generation • 8B • Updated 9 days ago • 266

promotion/qwen3-8b-inpo-avg-eta0p0075-s42

Text Generation • 8B • Updated 9 days ago • 428

promotion/qwen3-8b-inpo-avg-eta0p005-s42

Text Generation • 8B • Updated 9 days ago • 439

promotion/qwen3-8b-kto-avg-beta0p01-s42

Text Generation • 8B • Updated 8 days ago • 446

promotion/qwen3-8b-kto-avg-beta0p05-s42

Text Generation • 8B • Updated 8 days ago • 457

promotion/qwen3-8b-kto-avg-beta0p1-s42

Text Generation • 8B • Updated 8 days ago • 562

promotion/qwen3-8b-ipo-avg-beta0p01-s42

Text Generation • 8B • Updated 8 days ago • 615

promotion/qwen3-8b-ipo-avg-beta0p1-s42

Text Generation • 8B • Updated 8 days ago • 562

promotion/qwen3-8b-ipo-avg-beta0p5-s42

Text Generation • 8B • Updated 8 days ago • 518

promotion/qwen3-8b-htmnpo-skywork-s42

Text Generation • 8B • Updated 8 days ago • 405