NVIDIA Launches Nemotron 3.5 Lightning and NeMo Switchyard for Faster Agentic AI

What’s Actually Being Released

Two things, with different jobs:

Nemotron 3.5 Lightning is a 30-billion-parameter mixture-of-experts model built for high-volume, specialized tasks inside multi-agent systems. Think code review, security monitoring, billing queries — the repetitive, always-on work that doesn’t need a frontier reasoning model but still needs to be fast and accurate.

NeMo Switchyard is an open source routing library that automatically directs prompts to the most appropriate model at each step of an agent workflow. It works across open, proprietary, and NVIDIA models without requiring developers to rewrite their applications.

Together, they’re positioned as infrastructure for the “system of models” architecture that serious agentic deployments increasingly require.

Why Nemotron 3.5 Lightning Is Worth Watching

The headline numbers: up to 4x faster output speed and 30% faster agentic task completion compared to other models in its class, according to PinchBench benchmarks.

What makes it practically interesting is the combination of speed, openness, and deployment flexibility. It runs locally on NVIDIA RTX PCs, DGX Spark, DGX Station, and Jetson devices — or scales to data centers and cloud. For organizations with privacy requirements or latency constraints, that local deployment option matters.

It’s also fully customizable via NVIDIA NeMo post-training on domain-specific data. Several early adopters have already done exactly that:

• CrowdStrike for cybersecurity workflows
• Harvey with Trajectory for legal services
• CodeRabbit with Baseten for code review
• Fastino Labs for software development, finance, and healthcare

NVIDIA also publishes training data and techniques to the extent licensing allows, which supports auditability — a detail that tends to matter more as enterprise AI deployments mature.

NeMo Switchyard: The Routing Problem, Solved

Model routing sounds like a niche infrastructure concern until you’re managing a multi-agent system and realizing that sending every prompt to your most capable (and most expensive) model is quietly burning your budget.

NeMo Switchyard routes each step of an agent workflow to the most efficient model for that specific task, based on quality, latency, and cost priorities. Developers can tune the routing algorithms to match their own tradeoffs.

The partner results give a clearer picture of what “efficient” actually means here:

• Ramp matched frontier model performance while cutting costs by 58% and runtime by 33%
• LangChain achieved 74% lower cost in multi-turn agent tasks by routing only 7% of calls to a frontier model, at a 6% accuracy tradeoff
• Cognition reduced mean cost by 28% in Devin Desktop while maintaining near-frontier performance
• Classmethod saw a 27% cost reduction in initial testing

Internal benchmarks suggest NeMo Switchyard can reduce task completion cost to roughly one-third of using a single frontier model alone, while maintaining comparable accuracy.

Who This Is Actually For

This release is aimed squarely at teams building or scaling agentic applications — not people experimenting with chatbots.

If you’re running always-on agents, managing multi-model pipelines, or trying to bring inference costs under control without sacrificing quality, both tools are worth evaluating. The open source nature of NeMo Switchyard and the broad availability of Nemotron 3.5 Lightning (Hugging Face, ModelScope, OpenRouter, NVIDIA NIM) lower the barrier to trying them in real workflows.

The integrations with Kong AI Gateway, LiteLLM, LangChain, and LiteLLM’s proxy layer also mean you may not need to change much of your existing stack to start routing smarter.

The Practical Takeaway

The interesting shift here isn’t any single model — it’s the framing. NVIDIA is building toward a world where agentic AI runs as an ensemble of specialized models, with intelligent routing deciding who does what. Nemotron 3.5 Lightning and NeMo Switchyard are the first concrete pieces of that architecture available to use today.

If your AI costs are climbing and your agents are slower than they should be, the routing problem is probably worth solving before you scale further.