by NVIDIA
Nemotron 3.5 Lightning review, pricing and limits
NVIDIA's small, fast open model built to be the tireless worker inside agent loops, handling the high-volume tool calls and validation steps that frontier planners are too slow and expensive for.
- Best for
- High-volume tool-calling inside agent loops
- Not for
- Visual or multimodal agent workflows
Should you use Nemotron 3.5 Lightning?
The 1M-token context window comes with a catch: a single high-memory GPU serves only a shorter slice of it. Artificial Analysis rates its intelligence below the median for its size but its speed high, which suits a cheap, fast worker beside a stronger planner.
Best for
- High-volume tool-calling inside agent loops Built as the fast executor for repetitive tool calls and output validation, with NVIDIA reporting 30% faster PinchBench completion than Qwen3.6 35B-A3B.
- Cost-sensitive agent routing alongside a frontier planner NVIDIA's NeMo Switchyard router sends planning to a frontier model and execution steps to Lightning, so the expensive model sees fewer calls.
- Single-GPU self-hosted deployment Documented for one H100 or A100 80GB, and OpenMDW-1.1 open weights mean no per-token licensing cost.
Not for
- Visual or multimodal agent workflows Text-only model with no image, audio, or video understanding.
- Frontier-level coding or scientific reasoning Trails Qwen3.6 35B-A3B on SWE-bench Verified (51.56 vs 70.12) and GPQA Diamond (75.44 vs 83.40).
- Tasks needing full-fidelity recall across the entire 1M-token window Scores 52.0 on NVIDIA's own AA-LCR long-context benchmark, below Qwen3.6 (61.06) and Gemma (57.56), so accuracy degrades before the context limit is reached.
Nemotron 3.5 Lightning pricing
- Input, per 1M tokens
- $0.03/1M
- Output, per 1M tokens
- $0.12/1M
- Cached input, per 1M tokens
- $0.01/1M
- Pricing model
- per-token
- Free tier
- OpenRouter serves a free, rate-limited variant (nvidia/nemotron-3.5-lightning:free); self-hosting the open weights carries no licensing fee under OpenMDW-1.1.
What a real job costs
| Job | Input | Output | Total |
|---|---|---|---|
| Summarise a 20-page PDF | $0.0008 | $0.0001 | $0.0009 |
| Support reply | $0.0001 | $0.0000 | $0.0001 |
| One coding agent run | $0.0050 | $0.0024 | $0.0074 |
Budgets: 20-page PDF = 30k in / 1k out · Support reply = 2k in / 300 out · Coding agent run = 200k in / 20k out. Computed from the vendor's per-token prices at render time; cached-input discounts are not applied.
Cost examples
- Agent worker: 10M input / 2M output tokens per month (lowest listed rate)
- $0.49
- Agent worker: 10M input / 2M output tokens per month (DeepInfra rate)
- $0.92
- High-volume tool-call worker: 1M input / 500K output tokens per day (DeepInfra rate)
- $0.14
Prices read on 6 November 2026. OpenRouter lists five hosts for the same weights: Darkbloom at $0.025 input and $0.12 output per 1M tokens (the lowest), Fireworks at $0.05 and $0.20, DeepInfra at $0.06 and $0.16, io.net at $0.0469 and $0.134 after a 33% discount, and CoreWeave at $0.07 and $0.20. Artificial Analysis quotes $0.06 and $0.20. OpenRouter also runs a rate-limited free endpoint, and its page warns that free-endpoint use is logged to improve NVIDIA products. Self-hosting carries no license fee under the open license but needs a high-memory GPU of your own.
Self-hosting requirements
- Min vram gb
- 80
- Self-hosted
- Yes
- Recommended GPU
- 1x H100 80GB or 1x A100 80GB (BF16, 256K context); 8x H100, GB200 or B200 for 1M context
What Nemotron 3.5 Lightning can do
Nemotron 3.5 Lightning is NVIDIA's open Mixture-of-Experts language model for AI agents, released August 11, 2026. It activates only a small share of its parameters per token and scores 85.37 on NVIDIA's PinchBench agentic benchmark, trading frontier-level reasoning for fast, low-cost tool calling and validation steps.
What goes in and what comes out
Reads
- text
- code
- tool-calls
Writes
- text
- tool-calls
Capabilities
- Function callingSupported
- Structured outputSupported
- Tool useSupported
- Audio inputNot supported
- Audio outputNot supported
- Code executionNot supported
- Video inputNot supported
- VisionNot supported
- Web browsingNot supported
- Reasoning modes
- standard
- Long context recall
- moderate
Key features
LatentMoE Hybrid Architecture
Interleaves Mamba-2 state-space layers, Mixture-of-Experts layers, and Attention layers, routing tokens through a compressed latent dimension so only a small slice of parameters activates per token.
Multi-Token Prediction and Speculative Decoding
The model includes multi-token prediction layers, and the model card documents DSpark and DFlash speculative-decoding recipes for faster serving on Blackwell GPUs.
1M-Token Context Window
Accepts up to 1M tokens of input, though the model card validates 256K on a single 80GB H100 and 1M only on multi-GPU or Blackwell setups.
Runs on a Single GPU
The full-precision checkpoint is documented for one H100 80GB or A100 80GB GPU at 256K context, with an NVFP4 checkpoint for faster production inference.
Fully Open Release
NVIDIA published the weights, training data and recipes openly, so teams can fine-tune it for their own workloads.
Known limitations
- Text-only: no image, audio, or video understanding, ruling it out for any visual agent workflow.
- Trails specialist reasoning models on hard benchmarks: GPQA Diamond 75.44 vs Qwen3.6 35B-A3B's 83.40, and SWE-bench Verified 51.56 vs Qwen's 70.12.
- Long-context comprehension lags its own context window: scores 52.0 on NVIDIA's AA-LCR long-context recall benchmark versus 61.06 for Qwen3.6, so a 1M-token window does not guarantee 1M-token-quality recall.
Who it is for
- Teams building always-on agent workflows that need a cheap, fast executor for tool calls
- ML infrastructure teams wanting a single-GPU open model to self-host without per-token fees
- Platform teams pairing a frontier planner with a cheap worker model through a router
- Researchers fine-tuning an open agentic base model with LoRA, full SFT, or reinforcement learning
Supported languages
- English
- Spanish
- French
- German
- Italian
- Japanese
Nemotron 3.5 Lightning benchmarks
Nemotron 3.5 Lightning against its nearest rivals
Output speed tokens per second, higher is better
Standout result
PinchBench agentic task score, NVIDIA's own benchmark for tool-calling and agent subtask execution
Source: NVIDIA Developer Technical Blog, August 11, 2026 launch announcementWhere it sits
Cheaper than 96% of the 103 GA models with a published price, in the bottom third on SWE-bench Verified (rank 34 of 39), and one of 89 whose vendor states it does not train on customer data.
| Measure | This model | Rank | Median |
|---|---|---|---|
| Blended price (3:1)$ per 1M tokens | $0.049/M | #5 / 103 | peer median $1.71/M |
| Output speedtokens/s | 251 tok/s | #9 / 57 | peer median 94 tok/s |
| SWE-bench Verified% solved | 51.6% | #34 / 39 | peer median 76.5% |
| GPQA Diamond% correct | 75.4% | #44 / 63 | peer median 87.8% |
Ranks are against GA models on Hokai that publish the same figure; ties share a rank.
Measured figures
- AA blended price
- $0.095/M
- Output speed
- 251 tok/s
A benchmark is an exam, not the job. Scores transfer unevenly between tasks, so weigh the one closest to your workload and read every figure with its source.
Strengths and limits
Strengths
- Finishes agent tasks faster than its closest same-size open rival at similar accuracy by NVIDIA's account, and Artificial Analysis measures 251 output tokens per second.
- Runs on one high-memory GPU at a shorter context, unlike dense models of comparable total size that need several.
- Fully open under a permissive license: weights, training data and post-training recipes are public for commercial use.
- Completes agentic work faster than its closest open rival: NVIDIA reports 30% less time on 10,000 PinchBench tasks at similar accuracy to Qwen3.6 35B-A3B.
- Documented for one H100 or A100 80GB GPU since only 3B of 30B total parameters activate per token, versus multi-GPU setups for similarly capable dense models.
Limits
- Text-only: no image, audio, or video input support, unlike some multimodal competitors in its class.
- Trails its closest same-size open competitor on hard reasoning and coding benchmarks, according to NVIDIA's own published scores.
- Long-context recall trails its own context window: NVIDIA's AA-LCR score of 52.0 sits behind Qwen's 61.06 and Gemma's 57.56.
- Independent testing is less flattering than the launch material: Artificial Analysis scores it 6 on Intelligence Index v5.0.1, under the median of 8 for comparable models.
Nemotron 3.5 Lightning alternatives
The nearest AI models on Hokai for the same job.
| Model | Lab | Context | In / out per 1M | Reads and writes |
|---|---|---|---|---|
| Nemotron 3.5 Lightning This page | NVIDIA | 1M | $0.025 / $0.12 | text, code → text |
 Claude Fable 5Superseded by Fable 5.1 and Fable 5.2 but still sold on the Claude API: Anthropic's first generally available Mythos-class model, released June 2026, with a 1M-token context. | Anthropic | 1M | $10 / $50 | text, image, pdf → text |
| Claude Fable 5.1Superseded by Claude Fable 5.2 in November 2026 but still sold on the Claude API: Anthropic's second Mythos-class model, released Sept 1, 2026, for long-horizon coding and research. | Anthropic | 1M | $10 / $50 | text, image, pdf → text |
 Gemini Decision 1Google DeepMind's decision model scores many questions over one multimodal input of up to 262,144 tokens and bills zero output tokens. | 262K | $0.10 / $0 | text, image, audio, video, pdf → text |
Nemotron 3.5 Lightning specs
- Context window
- 1.0M tokens
- Tokens per second
- 251 tok/s
- Architecture
- Hybrid Mamba-2 + MoE + Attention (LatentMoE)
- Mixture of experts
- Yes
- Parameters
- 30B total, 3B active (MoE)
- VRAM requirements
- All 30B parameters stay resident; the model card documents a single H100 80GB or A100 80GB for BF16 at 256K context and 8 H100s, GB200 or B200 for the full 1M window
Training data
Pre-trained on 20+ trillion tokens (collection through September 2025) spanning English plus 19 other spoken languages and 43 programming languages, distilled from NVIDIA's 550B-parameter Nemotron 3 Ultra with a dedicated multi-token-prediction post-training phase that closed in May 2026.
Weight formats
- BF16
- NVFP4
- W4A16
- GGUF
API endpoints
| Name | Endpoint | Auth |
|---|---|---|
| NVIDIA NIM / build.nvidia.com | https://build.nvidia.com/nvidia/nemotron-3.5-lightning-30b-a3b | NVIDIA API Key |
| OpenRouter | https://openrouter.ai/nvidia/nemotron-3.5-lightning | API Key |
| DeepInfra | https://deepinfra.com/blog/nvidia-nemotron-3.5-lightning-release | API Key |
| Together AI | https://www.together.ai/models/nemotron-3-5-lightning | API Key |
| Hugging Face (self-host) | https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 | None (open weights download) |
SDKs
- Python
- TypeScript
Prompting patterns
Show 2 prompting patterns
Use NVIDIA's recommended sampling settings
temperature=1.0, top_p=0.95
NVIDIA's own model card lists these as the recommended defaults for expected benchmark-quality output.
Pair with a router for hard steps
Route via NeMo Switchyard's escalation router, promoting to a frontier model only on sustained difficulty
Lightning is tuned for high-volume routine execution, not frontier reasoning, so an escalation path recovers accuracy on hard subtasks.
Known quirks
Show 4 known quirks
3B active does not mean 3GB of memory
All 30B total parameters must be resident in memory regardless of only 3B being active per token.
Workaround: Budget for a single H100 80GB or A100 80GB for BF16, or use the NVFP4 checkpoint on Hugging Face for production inference.
1M context window, weaker long-context recall
The model accepts up to 1M tokens but scores only 52.0 on the AA-LCR long-context benchmark in NVIDIA's table, below Qwen3.6 35B-A3B's 61.06 and Gemma's 57.56.
Workaround: For tasks needing high-fidelity recall past roughly 100K-200K tokens, chunk the input or route the step to a stronger long-context model.
Most API hosts cap it well under the 1M architectural limit
NVIDIA's model card validates 256K tokens on a single H100, and OpenRouter serves it at 262K rather than the full 1M.
Workaround: Check the specific provider's context limit before assuming 1M tokens are available; self-host on multi-GPU or Blackwell hardware to unlock the full window.
Host prices differ several-fold, and the free endpoint logs usage
On 6 November 2026 OpenRouter listed input rates from $0.025 to $0.07 per 1M tokens for the same weights, and its free endpoint notice says use is logged to improve NVIDIA products.
Workaround: Compare providers before committing, keep confidential data off the free endpoint, and self-host if you need both low cost and full control.
Nemotron 3.5 Lightning facts
- Deployment
- API · Self-hosted
- Trains on your data
- No
- Safety posture
- Balanced
- Released
- 11 Aug 2026
- Current version
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
- Knowledge cutoff
- 01 May 2026
- Family
- Nemotron 3.5
- Variant
- Chat
- Licence
- OpenMDW-1.1 (permissive open license, ready for commercial use) · Open source
- Data retention
- NVIDIA does not centrally process inference traffic for this open-weight model; retention depends on whichever third-party host (OpenRouter, DeepInfra, Together, Fireworks, or self-hosted) serves the request.
Content restrictions
- Model card advises against circumventing built-in safety guardrails and asks developers to verify alignment fits their specific deployment.
- As an open-weight release, downstream deployers control final content policy; no vendor-enforced moderation endpoint exists outside NVIDIA's own NIM reference deployment.
Safety controls
- Rlhf
- Yes
- Red teaming
- not publicly disclosed
- System prompt
- Yes
- Constitutional AI
- No
- Content filter API
- No
- Moderation endpoint
- No
Data governance
- Trains on inputs
- No
- Eu AI act classification
- General-purpose AI model provider obligations (open-weight release, subject to EU AI Act GPAI provisions)
Version history
NVIDIA-Nemotron-3.5-Lightning-30B-A3B · 11 Aug 2026
Initial release. PinchBench 85.37, SWE-bench Verified 51.56, GPQA Diamond 75.44. Distilled from Nemotron 3 Ultra (550B).
Nemotron 3 Nano 30B-A3B · 14 Dec 2025
Predecessor. PinchBench 66.11, SWE-bench Verified 34.08. Hybrid Mamba-2/Transformer MoE, 31.6B total / 3.6B active.
What changed
Questions about Nemotron 3.5 Lightning
What does Nemotron 3.5 Lightning cost per million tokens?
NVIDIA sets no single price because the weights are open and each host charges its own rate. On 6 November 2026 the cheapest listing on OpenRouter was $0.025 input and $0.12 output per million tokens, while Fireworks, DeepInfra and CoreWeave sat between $0.05 and $0.07 for input. A rate-limited free endpoint exists, and self-hosting avoids per-token billing at the cost of renting a GPU.
Is Nemotron 3.5 Lightning better than Qwen3.6 35B-A3B?
Not on accuracy. In the comparison table on NVIDIA's model card, Qwen3.6 leads 83.40 to 75.44 on GPQA Diamond and 70.12 to 51.56 on SWE-bench Verified. Lightning's claim is speed: NVIDIA reports 30% less time to finish 10,000 PinchBench tasks at similar accuracy. Pick Qwen3.6 for hard coding and reasoning, and Lightning when step volume and latency dominate the bill.
Can I self-host Nemotron 3.5 Lightning for commercial work?
Yes. The weights, training data and recipes are all published, and NVIDIA calls the terms permissive and ready for commercial use. Full-precision and NVFP4 checkpoints are on Hugging Face. The model card validates one high-memory GPU at a shorter context and calls for a multi-GPU or Blackwell setup to reach the full 1M window.
Does NVIDIA keep my prompts when I use Nemotron 3.5 Lightning?
It depends on the route. Self-hosted weights send nothing to NVIDIA, and a paid third-party host follows its own retention policy. The free OpenRouter endpoint is different: its page says use is logged for security and to improve NVIDIA products, so keep confidential or personal data off it and use a paid host or your own hardware.
Is Nemotron 3.5 Lightning right for my agent, and when should I skip it?
It suits always-on agents that need a fast, cheap worker for tool calls, output validation and extraction, ideally beside a stronger planning model. Skip it for image, audio or video input because it is text-only, and for peak coding or reasoning accuracy. Artificial Analysis rates its overall intelligence below the median for comparable models.
About Nemotron 3.5 Lightning
NVIDIA released Nemotron 3.5 Lightning on August 11, 2026 as an open Mixture-of-Experts language model. It holds 30 billion parameters in total but activates only 3 billion per token, through a hybrid design that interleaves Mamba-2 state-space layers, expert layers and select attention layers, which NVIDIA calls LatentMoE. The company positions it as the successor to Nemotron 3 Nano 30B-A3B, shipped in December 2025, and builds it for the repetitive tool calls and validation steps that eat most of a long-running agent's token budget, work that rarely needs frontier-level reasoning.
Read the full background
NVIDIA's model card reports a PinchBench score of 85.37 in BF16, up from Nano's 66.11. SWE-bench Verified rose from 34.08 to 51.56, GPQA Diamond reaches 75.44 and MMLU-Pro reaches 81.94. Qwen3.6 35B-A3B, the closest open rival by size in the same table, posts 83.40 on GPQA Diamond and 70.12 on SWE-bench Verified. Lightning's pitch is throughput: NVIDIA says it completes 10,000 PinchBench tasks 30% faster than Qwen3.6 at similar accuracy. These are vendor-reported figures. Artificial Analysis, which runs its own tests, scores the model 6 on Intelligence Index v5.0.1 against a median of 8 for comparable models, and calls it verbose but quick: 251 output tokens per second and 4.3 minutes per index task versus a 16.3-minute median.
The architecture accepts up to 1M tokens, but the card validates only 256K on a single 80GB H100 in BF16 and lists 8 H100s, a GB200 or a B200 for the full window. OpenRouter serves it at 262K. Recall also lags the headline number: the 52.00 AA-LCR score trails Qwen3.6 at 61.06 and Gemma at 57.56, so a bigger window does not guarantee comprehension near its edges.
Where it loses is plain. The model is text-only, with no image, audio or video encoder. All 30 billion parameters stay in memory whatever the active count, so the small active figure does not mean a small footprint. Teams that need the strongest coding or reasoning accuracy should read the best open-weight models guide and pick a larger model, using Lightning only for the cheap steps.
Weights ship on Hugging Face in BF16 and NVFP4, with serving recipes for vLLM and SGLang. NVIDIA calls the BF16 card a base for customization and post-training, and says the NVFP4 release is the faster route for production inference. Hosted access runs through build.nvidia.com and OpenRouter, whose provider list currently names Fireworks, DeepInfra, CoreWeave and others; Baseten and CoreWeave also appear in NVIDIA's own host list. Prices vary by host and are covered in the pricing notes.
NVIDIA also released NeMo Switchyard, an open-source router that sends planning steps to a frontier model and execution steps to Lightning. Its launch post lists Cline, Kilo Code, LangChain and Hermes Agent among supporting harnesses, and names CodeRabbit, Fastino Labs and Arcos Labs among early adopters. The weights, training data and recipes all fall under the OpenMDW-1.1 license, which NVIDIA describes as permissive and ready for commercial use. Pre-training used over 20 trillion tokens with a September 2025 cutoff, and post-training data runs to May 2026.
Sources and guides
Official sources
- Nemotron 3.5 Lightning model page Vendor
- Licence text Licence
- System card Safety
Specifications on this page (context window, token prices, benchmark scores, licence) are read from NVIDIA's own model card and documentation. The verdict is Hokai's reading of them, not the provider's claim.
Hokai guides that mention Nemotron 3.5 Lightning
Lists that include Nemotron 3.5 Lightning
The provider
More AI Models on Hokai
Shieldstral 1.0Mistral's open-weight multimodal safety classifier, released 4 August 2026, that screens text and images against policies you write.
- Claude Mythos 5.2Anthropic's November 2026 Mythos model: Fable 5.2's weights with relaxed cyber and biology safeguards for verified teams, with a 1M-token context.
ERNIE-5.5-Flash-PreviewBaidu's text-only preview mixture-of-experts model, served through Qianfan on OpenRouter with a 1M-token context window and tool calling.
Gemini 4 CarbonGoogle DeepMind's Gemini 4 Carbon, open to paying Gemini API developers since 10 November: a reasoning model for agentic coding, research and cyber defense.
JevTypeSafe's first System One Model, launched in 2026: skips text generation for typed, calibrated decisions software can act on directly.Nex-N2.6Nex AGI's hosted agentic model for computer use and coding, with image input, tool calling and about 321B parameters, served first-party on OpenRouter.