MLOps & AI Infrastructure

In Part 1, we argued that LLM routing is qualitatively different from HTTP routing. Inference backends hold state that traditional load balancers ignore. This post covers the first of the…

There’s a version of the AI modernization story that goes: build the platform, then figure out the use cases. Ankur Jain would tell you that’s backwards — and that most…

I’ve been spending time looking at how teams are actually using Claude Code, and one thing keeps coming up: most of the cost surprises aren’t coming from where people expect….

There are many ways to optimize code for GPUs. In this post, you’ll learn how kernel fusion can improve memory bandwidth and reduce kernel launch overhead, along with multiple ways…

As enterprises scale their generative AI workloads, the demand for faster, more observable, and more flexible inference infrastructure continues to grow. Amazon SageMaker HyperPod is rising to meet that challenge…

Featured projects TL;DR Miles is RadixArk’s open source framework for large-scale LLM RL post-training. It composes SGLang for rollout, NVIDIA Megatron-LM for training, Ray orchestration, and PyTorch-native extensibility behind a…

July 18, 2024 — Posted by the TensorFlow teamTensorFlow 2.17 has been released! Highlights of this release (and 2.16) include CUDA update, upcoming Numpy 2.0, and more. For the full…

MLSys 2026 provided an excellent overview of the current state of inference across research and industry. With six sessions on LLM serving this year (twice as many as last year)…

Most enterprise analytics evaluations are really just dashboard comparisons. That’s the wrong starting point. The question that matters isn’t which vendor has the best UI — it’s whether analytics, AI…

With tools like Claude Code and Codex now standard in engineering workflows, developers are shipping new products, features, and bug fixes at mind-blowing speed. But as coding agent usage grows…