AI Infrastructure
What Is AI Infrastructure?
Ask five people what AI infrastructure means and you will get five answers, most of them about graphics cards. The hardware matters. It is also the part of the problem most organizations are least likely to get wrong, be...
Enterprise AI Infrastructure Architecture
A reference architecture is useful for one reason: it gives a group of people who own different parts of a system a shared picture to argue over. It is not a design. Any organization that adopts one unchanged has skipped...
Hybrid AI Infrastructure
Hybrid is the outcome most organizations reach, and the one fewest of them chose. It happens because different workloads carry different obligations, and a single environment that satisfies the strictest of them is usual...
Do You Need GPUs for Enterprise AI?
This question is usually asked in the wrong order. It arrives as "how many GPUs do we need", which assumes the answer is more than zero, and it frequently is not.
The honest sequence is: decide whether you are running mo...
GPU Infrastructure for Enterprise AI
An organization that has decided to run models itself now owns an unfamiliar class of infrastructure. Accelerators behave differently from the servers around them in ways that matter operationally, and most of the surpri...
What Is AI Inference Infrastructure?
Inference is the part of AI that runs constantly. Training happens rarely, if at all, in most enterprises. Inference happens every time somebody asks a question, and it is the workload the infrastructure has to be built ...
AI Model Serving Architecture
Model serving is the layer that turns a file on disk into something an application can call. It is a small idea with a surprising amount of operational surface, and most of that surface concerns change: how a model gets ...
Containers for AI Workloads
Containers arrived in AI infrastructure for an unglamorous reason: the dependency problem is severe. A model serving stack couples a framework, a numerical library, a device driver interface and a specific Python environ...
Kubernetes for AI Workloads
Kubernetes is the default answer to running containers at scale, and AI workloads inherit that default whether or not it suits them. Often it does. Sometimes it is a great deal of machinery in front of two servers that w...
What Is an AI Gateway?
The term is newer than the idea and less standardized than it sounds, so it is worth saying what it means before saying whether you need one. An AI gateway is a single point through which applications reach models, where...