Nemotron 3.5 Lightning is NVIDIA's open 30B Mixture-of-Experts model, activating just 3B parameters per token behind a 1M-token context window since its August 2026 launch. It replaces a frontier model as the fast, cheap worker inside AI agent loops, best paired with a stronger planner for the hardest steps.
Nemotron 3.5 Lightning is NVIDIA's open Mixture-of-Experts language model, scoring 85.37 on NVIDIA's PinchBench agentic benchmark while activating only a fraction of its total parameters per token. Unlike general-purpose chat models, it specializes in high-volume tool calling and validation steps for AI agents, trading frontier-level reasoning for output speed.
Provider: NVIDIA · Family: Nemotron 3.5
Context window: 1,000,000 tokens
Input modalities: text, code, tool-calls · Output: text, tool-calls
About Nemotron 3.5 Lightning
NVIDIA Nemotron 3.5 Lightning is an open Mixture-of-Experts language model NVIDIA released on August 11, 2026. It carries 30 billion total parameters but activates only 3 billion per token, using a hybrid architecture that interleaves Mamba-2 state-space layers, MoE expert layers, and select Attention layers, a design NVIDIA calls LatentMoE because tokens route through a compressed latent dimension before expert computation. NVIDIA distilled Lightning from its 550-billion-parameter Nemotron 3 Ultra and positions it as the successor to Nemotron 3 Nano 30B-A3B, shipped in December 2025. The goal is a fast, cheap executor for the repetitive tool calls and validation steps that dominate a long-running agent's token budget, work that does not need frontier-level reasoning. On NVIDIA's own PinchBench agentic benchmark, Lightning scores 85.37 in BF16 precision, up sharply from Nemotron 3 Nano's 66.11. SWE-bench Verified rose from Nano's 34.08 to 51.56, GPQA Diamond reaches 75.44, and MMLU-Pro reaches 81.94. Against Alibaba's Qwen3.6 35B-A3B, the closest open competitor by parameter count, Qwen wins on hard reasoning and coding: 83.40 on GPQA Diamond and 70.12 on SWE-bench Verified. Lightning's edge is speed: NVIDIA reports it completes 10,000 PinchBench tasks 30% faster than Qwen3.6 at similar accuracy. Lightning accepts up to 1M tokens architecturally, though NVIDIA's model card recommends capping at 256K tokens on a single H100, and most third-party hosts serve it at 262,144 tokens rather than the full 1M. On NVIDIA's AA-LCR long-context recall benchmark it scores 52.0, behind Qwen3.6's 61.06 and Gemma's 57.56: capacity does not guarantee comprehension near the edges of the window. The model is text-only, with no image, audio, or video encoder. It natively supports function calling, tool use, and structured output. NVIDIA ships it alongside NeMo Switchyard, an open-source router that sends each agent step to the cheapest capable model; an escalation router pairing Lightning with Claude Opus 4.8 cut total task cost to roughly a quarter of running Opus 4.8 alone, sending only 7% of calls to the frontier model. Third-party API pricing varies by host, from a low-cost baseline on DeepInfra to a higher launch rate on OpenRouter, which also runs a free rate-limited variant (see the pricing FAQ for exact rates). Because the weights are open, self-hosting on a rented GPU avoids per-token fees entirely above a certain volume. Lightning is live on build.nvidia.com and Hugging Face, plus hosts including OpenRouter, DeepInfra, Together AI, Fireworks, and Baseten. NVIDIA published BF16, NVFP4, and roughly 40 other quantized checkpoints, runnable on a single H100 80GB or A100 80GB GPU, with day-zero support in vLLM and SGLang. Artificial Analysis measured 295.2 output tokens per second and a 1.01 second time to first token, both better than the median open-weight model its size. Weights, training data, and recipes ship under the permissive OpenMDW-1.1 license, which NVIDIA states is ready for commercial use. The model card asks developers not to circumvent its safety guardrails and to verify alignment fits their deployment; NVIDIA has not published a formal responsible scaling policy or named red-team partners for this release. Lightning fits teams that need a cheap, fast agent worker, ideally paired with a stronger planner for hard steps. It is the wrong choice for multimodal input, frontier coding or reasoning accuracy, or anyone assuming a small memory footprint: all 30 billion parameters stay resident regardless of activation, so BF16 needs about 63.2GB and 4-bit quantization needs roughly 17.8GB. Teams needing Qwen3.6's stronger reasoning or Meta Muse Glimmer 30B's multimodal support should look elsewhere. Pre-training used more than 20 trillion tokens with a September 2025 cutoff, spanning English, 19 other spoken languages, and 43 programming languages. Post-training, distilling from Nemotron 3 Ultra, closed in May 2026. NVIDIA also open-sourced a reinforcement-learning dataset, Nemotron-RL Agentic Terminal Pivot, for training agent behavior in terminal environments. Lightning is the first release in what Artificial Analysis calls the Nemotron 3.5 line, following Nano's December 2025 debut and Ultra's June 2026 launch. It shipped alongside NeMo Switchyard, which already integrates with LangChain, LiteLLM, and Kong AI Gateway. Named early adopters include CrowdStrike, Harvey (with Trajectory), CodeRabbit (via Baseten), Lila Sciences, and Fastino Labs.
Pricing
DeepInfra and Artificial Analysis list $0.05 per 1M input tokens and $0.20 per 1M output tokens as the lowest verified third-party rate. OpenRouter's launch pricing is $0.10 and $0.25 per 1M, with a free, rate-limited variant also available there. Self-hosting the open weights avoids per-token fees entirely, at the cost of renting a GPU instead (roughly $6.50 to $4.00 per hour depending on the card).
Key Features
- LatentMoE Hybrid Architecture: Interleaves Mamba-2 state-space layers, Mixture-of-Experts layers, and Attention layers, routing tokens through a compressed latent dimension so only a small slice of parameters activates per token.
- Speculative Decoding for 4x Speed: Multi-token prediction plus DSpark and DFlash draft models give Lightning up to 4x faster output than similarly-sized open models, per NVIDIA's PinchBench testing.
- 1M-Token Context Window: Accepts up to 1M tokens of input, though NVIDIA recommends capping at 256K tokens for single-GPU deployment.
- Runs on a Single GPU: Only 3B of the total parameters activate per token, letting the full BF16 checkpoint run on one H100 80GB or A100 80GB GPU.
- Fully Open Release: NVIDIA published the weights, a 20-trillion-token training data mixture, and the training recipes under a permissive commercial-use license.
Pros
- Up to 4x faster output and faster task completion on NVIDIA's PinchBench evaluation than Qwen, its closest same-size open competitor, at similar accuracy.
- Fits on a single high-end GPU, unlike dense models with a comparable total parameter count.
- Fully open under a permissive license: weights, training data mixture, and post-training recipes are all public for commercial use.
Cons
- Text-only: no image, audio, or video input support, unlike some multimodal competitors in its class.
- Trails its closest same-size open competitor on hard reasoning and coding benchmarks, according to NVIDIA's own published scores.
- Long-context recall trails its own context window: NVIDIA's AA-LCR score of 52.0 sits behind Qwen's 61.06 and Gemma's 57.56.
Benchmarks
- mmlu pro: 81.94
- pinchbench: 85.37
- gpqa diamond: 75.44
- pinchbench nvfp4: 83.43
- swe bench verified: 51.56
- aa lcr long context: 52
- swe bench multilingual: 39.33
- artificial analysis intelligence index: 24
- artificial analysis price blended per m: 0.07
- artificial analysis speed tokens per sec: 295.2
Frequently Asked Questions
How much does Nemotron 3.5 Lightning cost per 1M tokens?
NVIDIA does not set one official price for Nemotron 3.5 Lightning since it ships as open weights that any host can serve. DeepInfra currently has the cheapest verified rate at $0.05 / $0.20 per million input and output tokens, and OpenRouter runs about double that at $0.10 / $0.25, though OpenRouter also has a free rate-limited option. You can also self-host on your own GPU to skip per-token billing altogether.
How does Nemotron 3.5 Lightning compare on benchmarks vs Qwen3.6 35B-A3B?
Qwen3.6 35B-A3B posts higher scores on hard benchmarks: 83.40 vs Lightning's 75.44 on GPQA Diamond, and 70.12 vs 51.56 on SWE-bench Verified. Where Lightning pulls ahead is raw throughput, finishing agent tasks 30% faster than Qwen3.6 at comparable accuracy on NVIDIA's PinchBench evaluation. Choose Qwen3.6 for harder reasoning and coding work, and Lightning when task volume and latency matter more than peak accuracy.
Is Nemotron 3.5 Lightning open source or proprietary?
Nemotron 3.5 Lightning ships fully open under the OpenMDW-1.1 license: NVIDIA released the weights, the training data mixture, and the training recipes, and states it is ready for commercial use with no field-of-use restriction disclosed. You can download the full checkpoint set, including NVFP4 and about 40 other quantized variants, directly from Hugging Face and self-host it on your own hardware.
Does Nemotron 3.5 Lightning train on user data?
No. Nemotron 3.5 Lightning is an open-weight model you either self-host or access through a third-party inference host, so NVIDIA itself never sees your inference traffic. Whether a given host retains or trains on your prompts depends entirely on that host's own policy, not NVIDIA's, since NVIDIA's role ends at publishing the weights and training recipe on Hugging Face.
Who is Nemotron 3.5 Lightning best for and who should avoid it?
Lightning fits teams running always-on AI agents that need a fast, cheap worker for tool calls, output validation, and long-context extraction, ideally paired with a stronger planning model for the hardest steps. Skip it if you need image or video understanding, since it is text-only, or if you need the strongest possible reasoning and coding accuracy, where Qwen3.6 and frontier closed models still lead.