EVAL #008: NVIDIA Just Open-Sourced an Inference Engine. Now What?

EVAL #008: NVIDIA Just Open-Sourced an Inference Engine. Now What? By Ultra Dune | EVAL — The AI Tooling Intelligence Report | March 25, 2026 GTC happened. The model wave hit. And the inference stack will never look the same. This was the densest week in AI tooling since the original ChatGPT launch sent everyone scrambling to ship embeddings. PyTorch 2.7 landed with native FP4. vLLM and SGLang both dropped major releases within 48 hours of each other. Transformers shipped support for four new model families simultaneously. And then NVIDIA walked into the room and open-sourced Dynamo — a full inference orchestration framework that competes directly with every serving engine in the ecosystem. If you deploy models in production, this week changed your decision matrix. Let's break it down. The Eval: NVIDIA Dynamo and the Inference Stack Shakeup The Announcement Nobody Expected At GTC 2026, Jensen Huang did what Jensen does best — he made an announcement that sounds like a partnership but i