MLOps best practices, AI infrastructure, model deployment, PyTorch, TensorFlow, and production machine learning workflows.
This post was co-written with Daniel Han and Michael Han from Unsloth. Deploying large foundation models (FMs) stored at their original 16-bit floating-point precision (BF16 or FP16) is expensive. They…
November 29, 2023 — Posted by Marat Dukhan and Frank Barchard, Software EngineersCPUs deliver the widest reach for ML inference and remain the default target for TensorFlow Lite. Consequently, improving…
I have spent most of my career obsessing over latency in AI systems – first at Google, working on critical AI infrastructure, and now at Modular, building the inference platform…
When a security analyst is alerted to a suspicious login or an unusual firewall block, the answer is almost always in the logs. Finding it is the hard part. Barracuda…
The first sign of trouble isn’t always performance. Sometimes it’s the invoice. Your team ships a new agent that routes requests, calls tools, runs retrieval, and orchestrates multiple LLM calls…
Biomolecular structure prediction and co-folding with models like OpenFold3 are now mainstream, large-scale workloads powering drug discovery and protein design. Increasingly, they’re driven end-to-end by AI agents. For an agent…
An artificial intelligence (AI) agent can process an invoice, help adjudicate a claim, or classify a support ticket in a proof of concept. But running these agents across thousands or…
TL;DR: DeepSeek-V4 support was live in SGLang on Day-0, but the Day-0 stack was only the starting point. Since launch, we have coordinated a set of kernel, runtime, and hardening…
December 05, 2023 — Posted by the TensorFlow teamWe are releasing a hot-fix for an installation issue affecting the TensorFlow installation process. The TensorFlow 2.15.0 Python package was released such…
At Modular, we’re always experimenting with the latest agentic programming tools, integrating the best ones into our workflows, and learning quite a few lessons along the way. One thing we…
