RISC-V AI Chips Are Joining GPU Race for AI Processing
- By Paul Mah
- March 02, 2022
A free and open-source instruction set architecture (ISA) is quietly gaining momentum and could well power a significant number of the estimated 25 billion AI chips that will be produced in 2027, according to a new report on IEEE Spectrum.
Designed at UC Berkeley in 2010, the RISC-V is based on reduced instruction set computer (RISC) principles. But why is it suited for AI, and what is fueling the growing interest in this relatively new architecture?
The road to AI
Currently, the majority of AI processing today is done by graphic processing units or GPUs. Originally developed to accelerate graphics processing in computers, GPUs have evolved to become a vital part of modern AI infrastructure thanks to their inherent effectiveness for AI workloads.
This can be attributed to its ease of scalability with the ability to install multiple GPUs in a single server, and sheer memory bandwidth from dedicated video memory. Another advantage lies with its architecture.
While general multi-core processors utilize a MIMD architecture, or multiple instructions, multiple data, the GPU uses a SIMD architecture, or single instruction, multiple data. The latter is ideal for deep learning, given how it requires the same process to be performed for numerous data items.
Today, GPUs used for AI currently range from consumer-level RTX graphic cards to the data center-centric Nvidia A100 Tensor Core GPUs used in Thailand’s upcoming supercomputer and Meta’s behemoth AI supercomputer – the latter will sport over 6,000 of these GPU.
Elsewhere, chipmaker AMD does offer GPUs designed for enterprise AI workloads, though they don’t have quite the stranglehold on the market that Nvidia has. Indeed, at least one machine learning (ML) researcher has opined that AMD’s ML libraries for its GPUs are not as robust.
Rise of the AI processors
With the growing importance of AI, other players have jumped aboard, designing custom chips to work exclusively with ML workloads. This is typically achieved with specialist application-specific integrated circuits (ASIC) chips built with a SIMD architecture.
For instance, Google makes ASIC-based called tensor processing units (TPUs) for deep learning workloads. Unlike Nvidia’s commercially-available GPUs, Google’s TPU is accessed primarily through its cloud, though stripped down versions known as “Edge TPUs” are available for purchase as part of a development board.
Not to be outdone, cloud giant AWS also makes its own AI chips – the first-gen Inferentia and second-gen Trainium chips. Similarly, Xilinx, which was acquired by AMD, makes high-performance AI processors for commercial customers.
With interest in AI processors soaring, at least a dozen AI startups have emerged over the last few years. And many of them are taking advantage of the open-source RISC-V architecture to leapfrog the competition and carve out a space in the increasingly lucrative segment estimated to be worth nearly USD129 billion in 2025.
The future has RISC-V in it
One such startup, Esperanto Technologies, utilized a modified RISC-V design with 1,092 cores into a system-on-a-chip (SoC) half the size of the popular A100 GPU from Nvidia.
As reported on IEEE Spectrum, the team created their own vector instructions to complement RISC-V’s efficient 47 instructions (A typical Intel desktop processor has close to a thousand instructions) to support machine learning math such as matrix multiplication.
The ET-SoC-1 from Esperanto is envisioned to accelerate AI in power-constrained data centers through expansion boards that fit into a standard peripheral component interconnect express (PCIe) slot. According to the report, each board can deliver 800 trillion operations per second.
What sets Esperanto’s solution apart is how each board uses multiple low-power SoC chips instead of a giant SoC. According to the AI chip maker, each ET-SoC-1 chip consumes 20 watts when performing a recommender-system benchmark neural network, or less than one-tenth of what the A100 GPU draws.
This allowed the team to place six chips for over 6,000 cores on a single AI accelerator card and still stay at around 120 watts.
And according to a report on All About Circuits last year, Esperanto claims an ET-SoC-1 outperforms the Nvidia A100 in both relative performance and energy efficiency running the MLPerf Deep Learning Recommendation Model benchmark.
To be clear, a high-performance chip by itself will not necessarily win the AI chip race. There are the software application programming interface and support from the wider ecosystem to consider – and which might well be the bigger hurdle to overcome.
Still, Esperanto is hardly the only startup touting an AI processor based on RISC-V; even Intel’s upcoming Mobileye EyeQ Ultra Chip for autonomous vehicles will sport 12 RISC-V cores. And as RISC-V adoption takes off, broader support is almost a certainty.
For now, Esperanto says samples of the ET-SoC-1 are in the hands of early partners.
Paul Mah is the editor of DSAITrends. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose. You can reach him at [email protected].
Image credit: iStockphoto/ALLVISIONN
Paul Mah
Paul Mah is the editor of DSAITrends, where he report on the latest developments in data science and AI. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose.