NCM

The NCM (Next-Concept Modeling) approach was proposed to achieve parallel generation through hierarchical latent representations, seeking a balance between efficiency and information retention.

NCM writes each model into a fixed-size output frame, so even if the answer is very short it must run through all layers and pay the full cost.

NCM cannot output in a streaming fashion, because there is no readable content before the final round ends.

In the paper, NCM's Sudoku advantage mainly comes from the cell-filling order (the decoding strategy) rather than the architecture itself.

NCM's advantage has strict boundaries of applicability: in the single-user local GPU scenario there is up to a 4x speedup, but in batch serving or parallel agent scenarios it is instead slower.

The released NCM code is currently for evaluation only, there is no training code, and prompt caching behaves problematically.

NCM's compressor sometimes confuses names and numbers when converting between text and codes.

The largest NCM model the authors trained is under 1 billion parameters, and whether it holds at 27B or even larger scales has not been verified.

The NCM model contains a compressor (which encodes text into multi-layer concept codes) and a Transformer generator (which predicts the next layer of concepts), with one forward pass per layer.

In the single-request scenario, NCM is 4 times faster than an autoregressive model of the same size on translation tasks and nearly 2 times faster on news summarization tasks.

In the high-concurrency scenario (64 requests), an autoregressive model of the same size overtakes NCM in speed.

NCM achieves 86% accuracy on hard Sudoku tasks, consistent with what the paper reports.

In the Sudoku task, NCM rewrites the entire board each round, writing the initial digits incorrectly in early rounds and converging as a whole at the end.

The second stage of NCM uses a Transformer to predict the discrete codes at each level in parallel, and the prediction at each level is conditioned on the previous level.

NCM's parallel prediction is a form of hierarchical conditional non-autoregressive generation: parallel within the same level, while conditional dependencies remain between levels.

NCM's tokenizer (VQ-VAE) and generator (Transformer) are trained separately and independently, which may limit overall performance.

NCM is similar to diffusion models in that both abandon strict left-to-right token-by-token generation; the difference is that NCM organizes the generation process with an explicit multi-granularity discrete codebook and a clearer hierarchical structure, whereas diffusion models implicitly control generation granularity through noise levels.

The core idea of NCM is: first use a multi-scale VQ-VAE to learn coarse-to-fine representations of text, then have a Transformer predict the discrete codes at each level in parallel, and finally reconstruct the text through a decoder.

Through hierarchical parallel prediction, NCM can theoretically reduce generation complexity from O(n) to O(log n) or lower, at the cost of potentially losing fine-grained information.