Microsoft Azure has delivered industry-leading outcomes for AI inference workloads amongst cloud service suppliers in the latest MLPerf Inference outcomes printed publicly by MLCommons. The Azure outcomes have been achieved utilizing the brand new NC H100 v5 collection digital machines (VMs) powered by NVIDIA H100 NVL Tensor Core GPUs and strengthened the dedication from Azure to designing AI infrastructure that’s optimized for coaching and inferencing within the cloud.
The evolution of generative AI fashions
Fashions for generative AI are quickly increasing in dimension and complexity, reflecting a prevailing development within the {industry} towards ever-larger architectures. Trade-standard benchmarks and cloud-native workloads constantly push the boundaries, with fashions now reaching billions and even trillions of parameters. A primary instance of this development is the latest unveiling of Llama2, which boasts a staggering 70 billion parameters, marking it as MLPerf’s most vital check of generative AI thus far (determine 1). This monumental leap in mannequin dimension is clear when evaluating it to earlier {industry} requirements such because the Massive Language Mannequin GPT-J, which pales as compared with 10x fewer parameters. Such exponential progress underscores the evolving calls for and ambitions throughout the AI {industry}, as clients attempt to sort out more and more complicated duties and generate extra refined outputs.
Tailor-made particularly to deal with the dense or generative inferencing wants that fashions like Llama 2 require, the Azure NC H100 v5 VMs marks a big leap ahead in efficiency for generative AI functions. Its purpose-driven design ensures optimized efficiency, making it a super selection for organizations in search of to harness the facility of AI with reliability and effectivity. With the NC H100 v5-series, clients can anticipate enhanced capabilities with these new requirements for his or her AI infrastructure, empowering them to sort out complicated duties with ease and effectivity.
Nonetheless, the transition to bigger mannequin sizes necessitates a shift towards a distinct class of {hardware} that’s able to accommodating the massive fashions on fewer GPUs. This paradigm shift presents a novel alternative for high-end techniques, highlighting the capabilities of superior options just like the NC H100 v5 collection. Because the {industry} continues to embrace the period of mega-models, the NC H100 v5 collection stands prepared to satisfy the challenges of tomorrow’s AI workloads, providing unparalleled efficiency and scalability within the face of ever-expanding mannequin sizes.
Azure AI infrastucture
World-class infrastructure efficiency for AI workloads
Enhanced efficiency with purpose-built AI infrastructure
The NC H100 v5-series shines with purpose-built infrastructure, that includes a superior {hardware} configuration that yields outstanding efficiency good points in comparison with its predecessors. Every GPU inside this collection is provided with 94GB of HBM3 reminiscence. This substantial improve in reminiscence capability and bandwidth interprets in a 17.5% enhance in reminiscence dimension and a 64% enhance in reminiscence bandwidth over the earlier generations. . Powered by NVIDIA H100 NVL PCIe GPUs and 4th-generation AMD EPYC™ Genoa processors, these digital machines characteristic as much as 2 GPUs, alongside as much as 96 non-multithreaded AMD EPYC Genoa processor cores and 640 GiB of system reminiscence.
In at present’s announcement from MLCommons, the NC H100 v5 collection premiered efficiency leads to the MLPerf Inference v4.0 benchmark suite. Noteworthy amongst these achievements is a 46% efficiency achieve over competing merchandise outfitted with GPUs of 80GB of reminiscence (determine 2), solely based mostly on the spectacular 17.5% improve in reminiscence dimension (94 GB) of the NC H100 v5-series. This leap in efficiency is attributed to the collection’ skill to suit the massive fashions into fewer GPUs effectively. For smaller fashions like GPT-J with 6 billion parameters, there’s a notable 1.6x speedup from the earlier technology (NC A100 v4) to the brand new NC H100 v5. This enhancement is especially advantageous for purchasers with dense Inferencing jobs, because it permits them to run a number of duties in parallel with higher pace and effectivity whereas using fewer assets.
Efficiency delivering a aggressive edge
The rise in efficiency is essential not simply in comparison with earlier generations of comparable infrastructure options Within the MLPerf benchmarks outcomes, Azure’s NC H100 v5 collection digital machines outcomes are standout in comparison with different cloud computing submissions made. Notably, when in comparison with cloud choices with smaller reminiscence capacities per accelerator, akin to these with 16GB reminiscence per accelerator, the NC H100 v5 collection VMs exhibit a considerable efficiency enhance. With practically six instances the reminiscence per accelerator, Azure’s purpose-built AI infrastructure collection demonstrates a efficiency speedup of 8.6x to 11.6x (determine 3). This represents a efficiency improve of fifty% to 100% for each byte of GPU reminiscence, showcasing the unparalleled capability of the NC H100 v5 collection. These outcomes underscore the collection’ capability to guide the efficiency requirements in cloud computing, providing organizations a strong answer to deal with their evolving computational necessities.
In conclusion, the launch of the NC H100 v5 collection marks a big milestone in Azure’s relentless pursuit of innovation in cloud computing. With its excellent efficiency, superior {hardware} capabilities, and seamless integration with Azure’s ecosystem, the NC H100 v5 collection is revolutionizing the panorama of AI infrastructure, enabling organizations to totally leverage the potential of generative AI Inference workloads. The most recent MLPerf Inference v4.0 outcomes underscore the NC H100 v5 collection’ unparalleled capability to excel in probably the most demanding AI workloads, setting a brand new commonplace for efficiency within the {industry}. With its distinctive efficiency metrics and enhanced effectivity, the NC H100 v5 collection reaffirms its place as a frontrunner within the realm of AI infrastructure, empowering organizations to unlock new potentialities and obtain higher success of their AI initiatives. Moreover, Microsoft’s dedication, as introduced through the NVIDIA GPU Know-how Convention (GTC), to proceed innovating by introducing much more highly effective GPUs to the cloud, such because the NVIDIA Grace Blackwell GB200 Tensor Core GPUs, additional enhances the prospects for advancing AI capabilities and driving transformative change within the cloud computing panorama.
Study extra about Azure generative AI
