Beyond Left-to-Right: A Survey of Decoding Schedulers in Diffusion Language Models

Abstract

Autoregressive language models decode one token at a time, a sequential bottleneck that caps throughput and compounds errors as generation proceeds. Diffusion language models (dLLMs) promise a way out – they start from a fully masked sequence and recover tokens in parallel under bidirectional attention, making the order of token revelation a choice rather than an architectural fate. But this flexibility conceals a trap – the inference-time schedule strongly shapes final quality, yet the training objective is order-agnostic and never specifies what schedule to follow. The scheduler, the component that makes these decisions, is thus both critical and routinely overlooked. In this survey, we separate the scheduler, which selects the positions to denoise, from the denoiser, which predicts the tokens, and we classify schedulers by five dimensions – the signal used to choose positions, whether the scheduler is trained, the granularity of the decision, the decoding budget, and whether decoded tokens may be remasked. Beyond taxonomy, we ask what a scheduler can fundamentally achieve. An oracle order defines an upper bound but remains a diagnostic. An information-geometric bound reveals that good schedules are cheap with prior knowledge but require linear cost without it. And even with a perfect denoiser, a sampler can deviate from the model’s intended distribution, which makes standard quality metrics misleading. We identify attribution, whether the remaining gap belongs to the scheduler or the backbone, as the main open question. To the best of our knowledge, this is the first survey to treat the inference-time scheduler as the primary subject of analysis, providing guidance on how results should be interpreted and compared in dLLMs.

Publication
In under review