by NVIDIA
NVIDIA's research family for diffusion text generation, in 8B and 3B sizes.
The 8B base model scores 65.12 on HumanEval and 88.57 on GSM8K in NVIDIA's own paper, and NVIDIA measures 120 to 140 tokens per second on one H100. It suits researchers studying few-step generation and distillation, not production assistants, since the base model skips instruction tuning.
Checked 17 Nov against NVIDIA's own pages.
| Tier | Rate |
|---|---|
| Self-hosted (open weights) | Free download; pay only for your own GPU |
NVIDIA hosts no service for these models, so there is no per-token price. The weights are free to download, and the cost is the CUDA GPU you run them on.
Nemotron-Labs-Sigma is NVIDIA's family of open continuous diffusion language models, released on 12 November 2026 in 8B and 3B sizes. Instead of writing one token at a time, it denoises whole blocks of text in parallel. The weights are on Hugging Face under the NVIDIA Nemotron Open Model License, and the context window is 4,096 tokens.
Reads
Writes
Continuous diffusion in a 16-D embedding space
Text is denoised from Gaussian noise, with the denoiser and token embeddings learned jointly.
Block-parallel decoding
Blocks of 64 tokens by default, with a configurable number of denoising steps per block.
Few-step distillation
Distilled variants at 2, 4, 8 and 16 steps; the paper reports about 8x on math and 4x on coding.
Two sizes, base and instruct
8B and 3B models, with instruct variants for chat.
Self-speculative variants
Distill-spec models accept 33% more draft tokens per round than Nemotron-Labs-Diffusion-8B on GSM8K, per the paper.
Plain PyTorch inference
Loads through Transformers with trust_remote_code, with no custom kernels or training framework needed.
HumanEval pass@1 for the 8B base model (DDPM, 256 steps per block, 10-seed mean)
Source: arXiv 2610.02665, Table 2, read 17 November 2026A benchmark is an exam, not the job. Scores transfer unevenly between tasks, so weigh the one closest to your workload and read every figure with its source.
| Model | Lab | Context | In / out per 1M | Reads and writes |
|---|---|---|---|---|
| Nemotron-Labs-Sigma This page | NVIDIA | 4K | -- | text → text |
| 256K | -- | text, image, video → text | ||
| Inception | 1M | $0.10 / $0.35 | text → text | |
| Mistral AI | 256K | $0.15 / $0.15 | text, image → text |
Training data
The card describes curated internal English text (webpages, articles, dialogue, code) of about 1.3 trillion tokens, split between autoregressive initialization and diffusion training. The data is not distributed.
| Name | Endpoint | Auth |
|---|---|---|
| Hugging Face (8B base weights) | https://huggingface.co/nvidia/Nemotron-Labs-Sigma-8B | None (open weights download) |
| Hugging Face (8B instruct weights) | https://huggingface.co/nvidia/Nemotron-Labs-Sigma-8B-instruct | None (open weights download) |
| Hugging Face (3B base weights) | https://huggingface.co/nvidia/Nemotron-Labs-Sigma-3B | None (open weights download) |
| Hugging Face (3B instruct weights) | https://huggingface.co/nvidia/Nemotron-Labs-Sigma-3B-instruct | None (open weights download) |
Sample with DDIM at 32 steps per block
sampler=ddim, steps=32, block_length=32, guidance_scale=3.0, score_temp=0.3
The card recommends these settings on the log_snr grid, with stronger score temperature as the step count falls.
Base model is not a chat assistant
The card says the 8B base is a pretrained starting point, best used with few-shot or benchmark-style prompts.
Workaround: Use the instruct variant for chat and instruction following.
Quality drops below 16 steps
The card says the pretrained models lose quality noticeably below roughly 16 denoising steps.
Workaround: Switch to the distilled variants, which are trained for a fixed step count.
Loading runs vendor code
Transformers loading needs trust_remote_code, which executes inference code stored in the repository.
Workaround: Read the repository's code before enabling trust_remote_code on a shared machine.
v1.0 · 12 Nov 2026
Hugging Face release date given on the model card; the paper was first submitted on 2 October 2026 and revised on 12 November 2026.
Nothing to download and no per-token price, because NVIDIA hosts no service for these models. Your own hardware is the only bill, and NVIDIA's speed test used one H100 in bfloat16.
The weights are open under NVIDIA's own Nemotron Open Model License, which Hugging Face tags as 'other' rather than a standard open-source licence. The card says the model is ready for commercial or non-commercial use. NVIDIA does not distribute the training data or training code, so read the licence text before shipping a product.
No. The card says the diffusion sampler is numerically degraded on CPU and produces unusable output. It lists Linux and NVIDIA Ampere, Hopper, Lovelace or Blackwell GPUs as supported, with bfloat16 as the working precision.
In the paper's 8B table, Sigma reaches HumanEval 65.12 and GSM8K 88.57 in diffusion mode, against 64.63 and 86.73 for Qwen3-8B in autoregressive mode. Qwen3-8B wins MBPP, 69.40 against 65.74, while Sigma wins Minerva Math, 66.72 against 52.94. All figures are reported by NVIDIA's authors, and no independent board lists the model.
The card recommends DDIM on the log_snr grid with 32-token blocks: guidance 3 and score temperature 0.3 at 32 steps, 0.4 at 16 steps and 0.6 at 8 steps, then guidance 2 with 0.8 at 4 steps. DDPM degenerates at low steps, and below roughly 16 steps the distilled variants are the better choice.
Nemotron-Labs-Sigma is a family of continuous diffusion language models from NVIDIA, released on 12 November 2026 in 8-billion and 3-billion-parameter sizes. Instead of generating one token at a time, the model denoises Gaussian noise in a 16-dimensional token-embedding space and decodes text in parallel blocks, reusing finished tokens through a standard key-value cache. The accompanying paper, arXiv 2610.02665 (Large Language Continuous Diffusion Models), calls it the first family of large-scale continuous diffusion language models. NVIDIA's other open models on Hokai include the chat model Nemotron Labs Dory 1.0 4B and the autoregressive Nemotron 3 Nano 30B A3B, which work differently.
NVIDIA publishes 14 repositories on Hugging Face: base and instruct models at both sizes, distilled variants fixed at 2, 4, 8 or 16 denoising steps, and distill-spec variants built for self-speculative decoding. The base model is meant for research, fine-tuning and distillation, not as an assistant, and the instruct variants cover chat.
In the paper's tables, the DDPM sampler at 256 steps per block gives the 8B base model HumanEval 65.12 and GSM8K 88.57, averaged over 10 seeds. These are vendor-reported figures, since the authors are NVIDIA researchers, read from the paper on 17 November 2026. The same table gives the earlier Nemotron-Labs-Diffusion-8B 62.80 and 87.26 in diffusion mode, so the margin is small. Moving to the faster DDIM sampler at 32 steps per block changes the six-task average from 71.16 to 70.18.
Speed is the point of the design. NVIDIA's card measures roughly 120 to 140 tokens per second for 256 generated tokens on one H100 in bfloat16, using DDPM at 32 steps per 64-token block. The paper reports about 8x acceleration on math and 4x on coding from distillation, at 8 and 4 tokens per forward pass, with negligible accuracy loss.
The constraints are tight. The context window is 4,096 tokens, prompt and output together, and the card lists English as the only supported language. The model needs an NVIDIA GPU with CUDA and bfloat16 support on Linux, and it loads through Transformers with trust_remote_code. NVIDIA hosts no service for it, so it runs offline on your own hardware.
The card says the licence allows commercial or non-commercial use, and the weights sit in NVIDIA's Hugging Face organisation.
Production assistants, long documents and hosted APIs are the wrong fit, because the base model is not instruction-tuned, the window is 4,096 tokens and no endpoint exists. Teams that want diffusion generation through an API can look at Mercury 3, teams that want an open diffusion model with a much longer window can look at DiffusionGemma 26B-A4B, and an ordinary autoregressive 8B baseline is Ministral 3 8B.
NVIDIA says the models were trained on about 1.3 trillion tokens of internal English text: roughly 1 trillion for autoregressive initialization, then about 0.3 trillion of continuous-diffusion training. The privacy subcard says the datasets included personal data and were reviewed before release, and the training data and code are not distributed.
Specifications on this page (context window, token prices, benchmark scores, licence) are read from NVIDIA's own model card and documentation. The verdict is Hokai's reading of them, not the provider's claim.