Nvidia Sets New Records on MLPerf for Upcoming H100 Chip
- By DSAITrends editors
- September 15, 2022
Nvidia’s last week touted some benchmarks from its upcoming H100 GPUs, which it says delivers up to 4.5 times more performance its previous-generation A100 GPUs to set new world records in inference workloads.
The MLPerf benchmark results is the first public demonstration of Nvidia’s upcoming H100 GPUs, which was first announced in March this year. To be clear, the inference benchmark Nvidia showed off is used to run learned models, which is different from machine learning (ML) process used to train new models.
The MLPerf benchmark
As explained by analyst Jack Gold of J.Gold Associates, MLPerf is an industry standard benchmark series that has broad inputs from a variety of companies. It models a variety of workloads, including natural language processing, speech recognition, image classification, medical imaging, and object detection.
Nvidia is understood to have used the MLPerf Inference v2.1 benchmark.
According to the charts released by Nvidia, the H100 did exceptionally well in the BERT-Large benchmark, which measures natural language processing (NLP). Nvidia says this is due to its Transform Engine, which in the real-world would allow it to shine when running large language models such as GPT-3.
The H100 GPU incorporates some 80 billion transistors that at 814 square millimeters stands at the edge of today’s chipmaking equipment. Under the hood, it features a new “SXM” connector to accommodate its higher power consumption that is expected to go as high between 500 and 600 watts per card.
The H100 is positioned as a data center and cloud GPU and potentially replace its A100 platform. However, its high power consumption might prevent existing systems based on the A100 from being upgraded.
The H100 is currently slated to be released “later this year”, though it is worth noting that earlier reports had cited a Q3 release timeframe.
Of course, Nvidia’s GPUs are not the game in town. Intel had previously touted its upcoming Habana Gaudi2 deep learning processors as outperforming the Nvidia A100. However, it remains to be seen how it would compete with the H100.
A new generation of AI chips is set to accelerate AI training and inference within the data center, as well as low-powered AI processors such as the Jetson AGX Orin for deployment at the edge.
Last week, we reported that the US has restricted the sale of advanced AI chips to China. This includes the Nvidia A100 and H100 GPUs, as well as the MI250 AI Accelerator family from rival chip maker AMD. Nvidia has since clarified in a second Securities and Exchange Commission filing that it would be allowed to continue development of the H100 in China.
Image credit: iStockphoto/Panuwat Sikham