MLOps best practices, AI infrastructure, model deployment, PyTorch, TensorFlow, and production machine learning workflows.
Today we’re releasing Modular 26.1, a major step toward making high-performance AI computing easier to build, debug, and deploy across heterogeneous hardware. This release is focused squarely on developer velocity…
Reinforcement learning (RL) is central to aligning language models, from reinforcement learning with human feedback (RLHF) within AI assistants to newer reinforcement learning with verifiable rewards (RLVR) workflows for reasoning…
This post is co-written with Vijay Venkatesh, CTO at Bluesight. If you build software for hospitals, you know that compliance work scales poorly. Hospitals managing 340B Drug Pricing Program compliance…
Introduction Key–Value Cache (KV cache/KVCache) is a foundational building block of modern LLM serving systems. It stores past attention states so the model can generate new tokens efficiently without excessive…
AI has transformed how organizations operate, driving unprecedented levels of productivity and innovation. However, AI adoption can be impeded by concerns surrounding data privacy, sovereignty and how to secure data…
Healthcare organizations need efficient scheduling solutions, and ScienceSoft’s AI voice assistant, powered by Amazon Nova Sonic and Amazon Bedrock Guardrails, shows how responsible AI can deliver that. The AI patient…
Today, BentoML is joining Modular. Our goal is simple: make it dramatically easier to serve high-performance inference in production – fast, portable, and without hardware lock-in. This acquisition accelerates our…
Engineering the Future: The Context Engineer Certification As organizations race to scale agentic AI, context has emerged as the critical bottleneck standing between pilot projects and real business value. To…
Training LLMs at massive scale brings unique infrastructure challenges, especially as jobs span thousands of GPUs and run for extended periods. The longer these jobs run, the greater the likelihood…
This post was written by Konstantin Lekh, Sasha Zinchuk, and Eugene Sergueev from Flo Health, and Liza (Elizaveta) Zinovyeva from AWS. In this post, we share how Flo Health’s engineering…
