This repo accompanies the paper: ActiveUltraFeedback — arXiv:2603.09692.

Active UltraFeedback is a scalable pipeline for generating high-quality preference datasets to align large language models (LLMs). We leverage uncertainty quantification and active learning to annotate only the most informative samples, drastically reducing costs while beating standard baselines.

Repository Purpose: This repository serves as a central hub for storing all experimental data, including the generated preference datasets and the resulting DPO/RM/IPO/SimPO models.

License: MIT
Disclaimer: The datasets hosted here contain outputs from various third-party open-source models. Users must ensure compliance with the specific licenses of the source models (e.g., Llama Community License, Apache 2.0, Qwen Research License) when using these datasets.

Citation

If you use our work or the ActiveUltraFeedback datasets, models, please cite us:

@misc{melikidze2026activeultrafeedbackefficientpreferencedata,
      title={ActiveUltraFeedback: Efficient Preference Data Generation using Active Learning}, 
      author={Davit Melikidze and Marian Schneider and Jessica Lam and Martin Wertich and Ido Hakimi and Barna Pásztor and Andreas Krause},
      year={2026},
      eprint={2603.09692},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2603.09692}, 
}