Intel Launches Gaudi 3 AI Accelerator
- By Paul Mah
- April 10, 2024
Intel this week launched a new AI accelerator which it claims is 40% more power efficient with 50% faster inferencing than Nvidia’s H100 GPU, the current king of the AI chip market.
In addition, it will also offer 30% faster inferencing on Meta’s Llama 7B and 70B parameter models, as well as The Technology Innovation Institute (TII)’s Falcon 180B parameter models against Nvidia H200.
Intel Gaudi 3
Intel’s Gaudi 3 is a full-length PCIe 600-watt card, packed with 128GB of memory and a bandwidth of 3.7TB per second. Similar to Nvidia’s upcoming B200 GPU, Gaudi 3 is made up of two identical silicon dies joined by a high-bandwidth connection.
Under the hood, Gaudi 3 has more HBM memory than H100, but less than the H200 or AMD’s MI300. Its memory bandwidth is also superior to H100’s and uses less expensive HBM2e memory – which might help explain its relatively lower cost.
There are twenty-four 200 gigabit (Gb) Ethernet ports integrated into every Intel Gaudi 3 accelerator, which Intel says offers flexible and open-standard networking – a pointed reference to Nvidia’s NVLink technology.
“In the ever-evolving landscape of the AI market, a significant gap persists in the current offerings. Feedback from our customers and the broader market underscores a desire for increased choice,” said Justin Hotard, Intel executive vice president and general manager of its Data Center and AI Group.
“Enterprises weigh considerations such as availability, scalability, performance, cost, and energy efficiency. Intel Gaudi 3 stands out as the GenAI alternative presenting a compelling combination of price performance, system scalability, and time-to-value advantage.”
More options for AI
Intel’s announcement comes a month after Nvidia showed off its own next-generation AI platform called Blackwell to dramatically ramp up the computing capabilities needed to support AI. As we reported then, Nvidia claims its upcoming GPUs can train a GPT-4-level LLM with just a quarter of the chips and far less power.
Intel’s Gaudi enters an increasingly crowded field, with Google this week announcing the availability of its Cloud TPU v5p to developers. But while Intel and Google bring a wider range of options to public cloud providers and large enterprises, there is no doubt that Nvidia’s offerings are far more popular.
The Gaudi 3 accelerator will be available to original equipment manufacturers (OEMs) in the second quarter of 2024 while its add-on cards should ship before Nvidia’s latest Blackwell GPUs.
Image credit: Intel
Paul Mah
Paul Mah is the editor of DSAITrends, where he report on the latest developments in data science and AI. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose.