Every token a standard language model produces costs one full pass through the network, so a long answer is slow by construction and inference dominates the running cost of most AI products. Researchers at Stanford have published a way around that arithmetic. Next-Concept Modeling (NCM), posted to arXiv on 9 October 2026 by a team drawn from the Hazy Research and Scaling Intelligence labs, generates text the way a draughtsman works, laying down the whole shape first and filling in the detail afterwards. On summarisation it matches the strongest diffusion language model in eight steps rather than 128, which the authors measure as 31 times faster in wall-clock terms. The models are small, the work is a preprint, and nobody outside the team has reproduced it yet.
What is Next-Concept Modeling?
NCM is a two-stage system. The first stage trains a multiscale tokeniser that turns a passage of text into a hierarchy of discrete codes, where the early levels are short and carry broad structure while the later levels are longer and add detail. Each level has its own codebook. A technique called nested dropout pushes the early levels to carry meaning on their own rather than leaning on the levels below them, and a token-level reconstruction objective keeps the original text recoverable from the hierarchy.
The second stage freezes that tokeniser and trains a transformer to predict one level at a time, conditioned on the reconstruction of everything coarser. Every code within a level is predicted in parallel, which is where the speed comes from. All levels train in a single forward pass under block-diagonal attention, and the authors deliberately corrupt a fraction of the conditioning codes during training so that the model learns to recover from its own earlier errors rather than compounding them.
The lineage is explicit. The method adapts next-scale prediction from VAR, an approach developed for image generation, and the Stanford contribution is the set of changes that make it work on language.
How does it differ from autoregressive and diffusion generation?
Three ways of producing a sentence are now in play. An autoregressive model writes left to right, one token per pass, with excellent quality and a cost that scales with length. A diffusion language model refines an entire sequence over many denoising steps, which parallelises the writing but leaves every step processing the whole sequence, and quality falls away when the step count is cut. NCM takes a third route, working from coarse to fine with one pass per level.
Diffusion has already reached production scale, which makes the comparison concrete. Google DeepMind released DiffusionGemma in June 2026 under an Apache 2.0 licence, an open-weights diffusion language model with 26B total parameters in a mixture-of-experts design and roughly 3.8B active, denoising blocks of 256 tokens and clearing 1,000 tokens per second on a single NVIDIA H100. It also trails its autoregressive sibling on quality, scoring 77.6% on MMLU Pro against Gemma 4’s 82.6%, and Google states plainly that output quality is lower. Speed bought at a measurable cost in quality is the trade NCM claims to improve on.
That difference shows up in the cost in a specific way. A diffusion model which halves its steps halves its work and gives up quality in exchange. NCM’s early levels are genuinely shorter than its later ones, so the saving comes from making fewer passes and from most of those passes being cheap.
What do the results show?
The paper compares NCM against diffusion baselines at matched parameter scales, mostly around 100M parameters. The results below carry the argument.
| Task | NCM | Strongest diffusion baseline | Reported gain |
|---|---|---|---|
| LM1B generative perplexity (lower is better) | 93.05 in 12 steps | DUO, 97.30 in 128 steps | 21 times faster |
| OpenWebText generative perplexity | comparable | strongest diffusion baseline | 10 times faster |
| XSum summarisation (ROUGE-L) | 28.0 in eight steps | AURORA-LM, 26.6 in 128 steps | 31 times faster |
| WMT14 German to English (BLEU, unguided) | 22.6 in 11 steps | E2D2, 24.8 in 64 steps | NCM trails on quality |
| Hard Sudoku (boards solved) | 86.1% in 12 steps | FMLM+, 81.3% in 16 steps | best score, fewest steps |
The authors also report reaching comparable quality on four to five times fewer training tokens, and spending the time saved on minimum Bayes risk decoding lifts the XSum score to 29.6, the best figure in the table.
The headline range quoted for the method is nine to 31 times faster in the code repository and 10 to 31 times on the Hazy Research blog, so the honest summary is up to 31 times on the single task where the gap is widest.
Where does it fall short?
Translation is the counterweight. On WMT14 German to English, NCM scores 22.6 BLEU without guidance where E2D2 reaches 24.8, and the best guided diffusion result, from ELF, is 26.4 against NCM’s 23.9. Minimum Bayes risk decoding closes most of that gap at 25.2, though it does so with 132 network evaluations rather than 11, which spends the advantage the method was built to deliver. The Hazy Research write-up describes translation as similar or better than the strongest diffusion baselines, and that reads generously against the table in the paper.
Scale is the larger caveat. The generators run between 25M and 120M parameters, two to three orders of magnitude below anything in production. A preliminary scaling study at 97M, 317M and 725M parameters shows generative perplexity improving as the models grow, and the authors are careful to call that result early. Nothing in the paper speaks to instruction following, long documents, code, or behaviour at production sizes.
Two further limitations shape how far the result travels. The tokeniser is lossy, recovering between 98.79% and 99.99% of tokens exactly depending on the dataset, with translation the weakest case and plain web text the strongest. The comparisons are also drawn almost entirely against diffusion language models rather than against the autoregressive systems that serve production traffic, with Sudoku the one exception.
What should a technical leader do about it?
Nothing operational, which is the right answer for a preprint published on 9 October. There is no serving stack, no quantised build, no fine-tuning recipe and no replication from outside the lab. A team that moved inference to NCM this quarter would rebuild the surrounding infrastructure from nothing in exchange for a speed-up measured on models a hundredth of the size of the ones they run.
The reason to watch it is that inference cost sets the price of everything built on top of it. API pricing, the number of steps an agent can afford before it becomes uneconomic, and whether a model runs on a handset all follow from how many sequential passes a token costs. A method that cuts the pass count by an order of magnitude at comparable quality would move those limits if it survives contact with scale. The workloads where it would land first are the structured ones the paper already tests, meaning summarisation, translation and constrained reasoning, and those are the same workloads where a doubling in the cost of model access would hurt most.
Everything is open, which makes the claim checkable rather than merely reported. The paper, the code and the released checkpoints and datasets are all public, and the evaluation scripts run the published tasks directly. Anyone with a CUDA machine and a free afternoon can confirm or refute the timings, and one reported measurement from outside Stanford would be worth more than another summary of the announcement.
Frequently asked questions
Is Next-Concept Modeling ready to use in production?
No. The released checkpoints are research artefacts of roughly 100M parameters, trained for benchmark tasks rather than general use, and there is no serving or quantisation support around them. The sensible posture is attention rather than adoption.
Does NCM replace autoregressive models such as GPT or Claude?
Not on this evidence. The comparisons run mostly against diffusion language models at matched size, and the one direct test against an autoregressive baseline is Sudoku, a constrained reasoning task rather than open-ended writing. Carry on shipping autoregressive models.
Is a “concept” in NCM a concept in the ordinary sense?
No, and the name flatters the method slightly. A concept here is a code at a coarse level of the learned hierarchy, optimised so that the text can be reconstructed from it. The authors make no claim that these codes correspond to ideas a person would recognise, and nothing in the method depends on them doing so.
How fast is Next-Concept Modeling, exactly?
The authors report 21 times faster than the strongest diffusion baseline on LM1B, 10 times on OpenWebText and 31 times on XSum summarisation, each at comparable or better quality. These are author-reported figures on their own hardware, and no independent party has yet measured them.
How does Next-Concept Modeling compare with DiffusionGemma?
They sit at opposite ends of maturity. DiffusionGemma is a shipped 26B open-weights model from Google DeepMind that runs above 1,000 tokens per second on an H100 and accepts a quality penalty against Gemma 4. NCM is a 100M-parameter research prototype that reports beating diffusion baselines on both speed and quality at matched size. NCM has the better idea on paper and DiffusionGemma has the only track record.
What would make this worth acting on?
Three things, in order. A replication from outside Stanford, a checkpoint at a few billion parameters that holds the speed advantage, and evidence on instruction following. The first two would be enough to justify a serious evaluation.
#ArtificialIntelligence #AIStrategy #LLM #InferenceCost #DiffusionModels #AIResearch
