arxiv:2603.25804

RealChart2Code: Advancing Chart-to-Code Generation with Real Data and Multi-Task Evaluation

Published on Mar 26

· Submitted by

zjj on Mar 30

Qwen

Upvote

Authors:

Abstract

Vision-Language Models face significant challenges in generating complex multi-panel charts from real-world data, as demonstrated by a new large-scale benchmark that reveals performance gaps between proprietary and open-weight models.

AI-generated summary

Vision-Language Models (VLMs) have demonstrated impressive capabilities in code generation across various domains. However, their ability to replicate complex, multi-panel visualizations from real-world data remains largely unassessed. To address this gap, we introduce \texttt{RealChart2Code}, a new large-scale benchmark with over 2,800 instances grounded in authentic datasets and featuring tasks with clear analytical intent. Crucially, it is the first benchmark to systematically evaluate chart generation from large-scale raw data and assess iterative code refinement in a multi-turn conversational setting. Our comprehensive evaluation of 14 leading VLMs on RealChart2Code reveals significant performance degradation compared to simpler benchmarks, highlighting their struggles with complex plot structures and authentic data. Our analysis uncovers a substantial performance gap between proprietary and open-weight models and confirms that even state-of-the-art VLMs often fail to accurately replicate intricate, multi-panel charts. These findings provide valuable insights into the current limitations of VLMs and guide future research directions. We release the benchmark and code at https://github.com/Speakn0w/RealChart2Code.

View arXiv page View PDF Project page GitHub 4 Add to collection

Community

zjj1233

Paper submitter about 20 hours ago

Vision-Language Models (VLMs) have demonstrated impressive capabilities in code generation across various domains. However, their
ability to replicate complex, multi-panel visualizations from real-world data remains largely
unassessed. To address this gap, we introduce
RealChart2Code, a new large-scale benchmark with over 2,800 instances grounded in
authentic datasets and featuring tasks with clear
analytical intent. Crucially, it is the first benchmark to systematically evaluate chart generation from large-scale raw data and assess iterative code refinement in a multi-turn conversational setting. Our comprehensive evaluation
of 14 leading VLMs on RealChart2Code reveals significant performance degradation compared to simpler benchmarks, highlighting their
struggles with complex plot structures and authentic data. Our analysis uncovers a substantial performance gap between proprietary and
open-weight models and confirms that even
state-of-the-art VLMs often fail to accurately
replicate intricate, multi-panel charts. These
findings provide valuable insights into the current limitations of VLMs and guide future research directions. We release the benchmark
and code at https://github.com/Speakn0w/RealChart2Code.