Reviser
A text generation model based on autoregressive cursor actions that defines three action primitives—INSERT, MOVE, and STOP—to perform editing operations on a mutable canvas. It enables non-monotonic text generation and revision while preserving the simplicity of autoregressive models.
Timeline (last 90 days)
The arXiv paper "Reviser: Revision-Capable Text Generation via Autoregressive Cursor Actions" proposes a cursor-action-based text generation method capable of revision.
Reviser is a decoder-only Transformer that defines the process of generating a response as executing a series of relative cursor actions on a mutable canvas.
At each step, Reviser predicts only one action token, comprising three action primitives: INSERT(token), MOVE(Δ), and STOP.
In Reviser's MOVE(Δ), Δ can be negative, representing moving the cursor left to go back and revise previously generated content.
Reviser performs autoregression over the edit history of actions rather than over the final text sequence, achieving non-monotonic generation.
In arena evaluations on continuation benchmarks, Reviser significantly outperforms two discrete diffusion/masked language model baselines, SEDD and MDLM.
Reviser is competitive with autoregressive baselines of comparable parameter counts at both the 100M and 300M scales.
Under a unified FLOPs convention, the inference computation required by Reviser is significantly lower than that of multi-round refinement methods and diffusion-based baselines.
Trajectory statistics confirm that the Reviser model does indeed frequently perform backward movements and insertions in the middle of the canvas, rather than degenerating into append-only decoding.
Reviser's action sequence length is typically longer than the final output text, because it includes a large number of move and intermediate editing steps.
1 more timeline events
All Facts (11)
The scale at which Reviser has currently been validated remains at the hundred-million parameter level, and whether it can scale to mainstream large model sizes remains to be seen.
50%UnverifiedReviser is a decoder-only Transformer that defines the process of generating a response as executing a series of relative cursor actions on a mutable canvas.
50%UnverifiedAt each step, Reviser predicts only one action token, comprising three action primitives: INSERT(token), MOVE(Δ), and STOP.
50%UnverifiedIn Reviser's MOVE(Δ), Δ can be negative, representing moving the cursor left to go back and revise previously generated content.
50%UnverifiedReviser performs autoregression over the edit history of actions rather than over the final text sequence, achieving non-monotonic generation.
50%UnverifiedIn arena evaluations on continuation benchmarks, Reviser significantly outperforms two discrete diffusion/masked language model baselines, SEDD and MDLM.
50%UnverifiedReviser is competitive with autoregressive baselines of comparable parameter counts at both the 100M and 300M scales.
50%UnverifiedUnder a unified FLOPs convention, the inference computation required by Reviser is significantly lower than that of multi-round refinement methods and diffusion-based baselines.
50%UnverifiedTrajectory statistics confirm that the Reviser model does indeed frequently perform backward movements and insertions in the middle of the canvas, rather than degenerating into append-only decoding.
50%UnverifiedReviser's action sequence length is typically longer than the final output text, because it includes a large number of move and intermediate editing steps.
50%UnverifiedThe arXiv paper "Reviser: Revision-Capable Text Generation via Autoregressive Cursor Actions" proposes a cursor-action-based text generation method capable of revision.
50%