NVIDIA has published Nemotron-Labs-Sigma-8B on Hugging Face, an open language model that takes a different route to generating text. Instead of predicting one token at a time, Sigma starts from noise and refines whole blocks of 64 tokens in parallel until words emerge.
The approach is called continuous diffusion. The model works in a 16-dimensional embedding space, cleaning up Gaussian noise over a configurable number of denoising steps, using DDPM or DDIM samplers with self-conditioning. Tokens already committed are reused through the standard KV cache, and running fewer steps trades a little quality for much higher speed.
Sigma is described in an October 2026 arXiv paper, “Large Language Continuous Diffusion Models”, which presents it as the first large-scale continuous language model at 3 and 8 billion parameters. It sits in NVIDIA’s broader Nemotron-Labs-Diffusion family, which can switch between autoregressive decoding and parallel diffusion simply by changing the attention pattern at inference time. The Hugging Face collection also lists 3B, 14B and 8B vision-language variants.
The released SFT checkpoint is fine-tuned for instruction following, with an 8,192-token context window optimised for English, and the model card covers both commercial and non-commercial use. The speed and quality figures so far are NVIDIA’s own; independent tests will show whether parallel generation can genuinely rival token-by-token models. For developers who run open models locally, it is one more architecture worth watching — and worth trying through tools such as Ollama. It joins a growing field of open-weight models that developers can download and run themselves, alongside options such as DeepSeek.
Source: AlexTech