With GPUs, you don’t have that visibility or the pliability to request, “I would like 4 gigabytes of that GPU, and I solely need one gigahertz of that GPU to go along with it.” As a substitute, the commonest setup right this moment is all or nothing—you request the entire GPU or none of it. The transparency problem is that GPUs require an strategy to monitoring and understanding utilization that’s all their very own, as a result of GPUs are specialised and mix facets of CPU and reminiscence. That problem is compounded by the truth that a node can have a number of bodily GPUs in a system (typically as much as eight). It’s additionally doable so as to add or take away GPUs from techniques. That’s one thing usually seen in on-premises environments, and one thing you’d not usually see with CPUs. These dynamics illustrate why gaining GPU visibility requires a recent strategy.
How Kubecost allows GPU monitoring and optimization
Kubecost meets the GPU visibility problem by understanding which nodes have GPUs and whether or not these nodes are on a public cloud supplier or in an on-premises atmosphere. Kubecost additionally understands what these nodes price, and subsequently understands proportionally what the GPU prices. That’s true whether or not a enterprise makes use of one of many “large three” cloud suppliers, or self-provides node prices primarily based by itself non-public cloud configuration.
With these GPU prices in hand, the following step is to have a look at GPU utilization. Kubecost identifies price allocation primarily based not solely on GPUs requested, but additionally on GPU utilization, with a purpose to acknowledge idle capability. Kubecost additionally scrapes commonplace metrics, together with utilization info, offered by Nvidia software program. (We plan to broaden to AMD and extra GPU manufacturers.) By combining price and utilization info, Kubecost can decide GPU effectivity, which is among the largest questions in enterprise leaders’ minds as GPUs develop ever extra highly effective and dearer.
