Modular: Why LLM Inference Needs a New Kind of Router
In Part 1, we argued that LLM routing is qualitatively different from HTTP routing. Inference backends hold state that traditional
MLOps best practices, AI infrastructure, model deployment, PyTorch, TensorFlow, and production machine learning workflows.
In Part 1, we argued that LLM routing is qualitatively different from HTTP routing. Inference backends hold state that traditional
There’s a version of the AI modernization story that goes: build the platform, then figure out the use cases. Ankur
I’ve been spending time looking at how teams are actually using Claude Code, and one thing keeps coming up: most
There are many ways to optimize code for GPUs. In this post, you’ll learn how kernel fusion can improve memory
As enterprises scale their generative AI workloads, the demand for faster, more observable, and more flexible inference infrastructure continues to
Featured projects TL;DR Miles is RadixArk’s open source framework for large-scale LLM RL post-training. It composes SGLang for rollout, NVIDIA
July 18, 2024 — Posted by the TensorFlow teamTensorFlow 2.17 has been released! Highlights of this release (and 2.16) include
MLSys 2026 provided an excellent overview of the current state of inference across research and industry. With six sessions on
Most enterprise analytics evaluations are really just dashboard comparisons. That’s the wrong starting point. The question that matters isn’t which
With tools like Claude Code and Codex now standard in engineering workflows, developers are shipping new products, features, and bug