Papers
arxiv:2606.29066

x-Prediction Flow: Efficient Continuous Decoding for Masked Diffusion Language Models

Published on Aug 17
Authors:
,
,
,
,

Abstract

Masked diffusion language models are enhanced by treating mask prediction as continuous state prediction, enabling token-wise progressive updates and a learned policy to preserve accuracy with far fewer diffusion steps.

Masked diffusion language models (MDLMs) generate text by iteratively unmasking tokens, but their standard decoder reduces each step to a binary action: a position is either committed to a single token or left fully masked, discarding rich predictive information rather than carrying it forward, and forcing premature, irrevocable commitments that lead to poor performance under a limited decoding budget. In this paper, we reinterpret mask prediction as a clean-state prediction (x-prediction) and show that it can be used to induce a continuous flow in the input embedding space. Building on this view, we propose a continuous decoding framework for MDLMs where tokens can accumulate partial progress at each diffusion step and remain revisable. To match the uneven contextual constraints across positions in language, we replace the globally synchronous schedule in image diffusion with a confidence-based asynchronous update in which the diffusion progress is token-wise accumulated. Additionally, we introduce a lightweight policy network and formulate its training as a reinforcement learning problem. Applied to pretrained LLaDA, our decoder retains 83--97% of full-budget accuracy using under 15% of the diffusion steps, largely outperforming discrete mask-prediction decoding at matched budgets.

Community

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2606.29066
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2606.29066 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2606.29066 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2606.29066 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.