First Server-on-a-Chip for AI Systems Moves to Production
- By Paul Mah
- September 13, 2023
The world's first AI inference server-on-a-chip by AI startup firm NeuReality has been finalized and moved to production at a TSMC manufacturing facility after its design was successfully validated.
Touted as the world’s first "Network Addressable Processing Unit", or NAPU, the NR1 chip is designed to offload AI-optimized networking and data processing into hardware. It allows cloud providers or enterprises to do away with expensive, power-hungry CPUs to host AI processing hardware for a significantly improved TCO.
AI inference presents significant technical hurdles for the traditional CPU-focused computing architecture. Essentially, data is repeatedly transferred between the network, CPU, and AI hardware under software-based management and control.
This leads to numerous conflicts between parallel commands, limiting the AI hardware's efficiency, wasting valuable hardware resources, and raising costs and power usage.
NeuReality says that trials with its NR1 chip demonstrated 10 times the performance at the same cost compared to conventional CPU-centric systems. This promises higher performance, greater affordability, and easier-to-use data center infrastructure to unlock the full potential of AI inference.
As AI deployment gains momentum, NeuReality's solution could be the essential ingredient for an industry desperately seeking affordable access to cutting-edge AI inference infrastructure.
“In order for Inference-specific deep learning accelerators (DLA) to perform at full capacity, free of existing system bottlenecks and high overheads, our solution stack, coupled with any DLA technology out there, enables AI service requests to be processed faster and more efficiently," said Moshe Tanach, the co-founder and CEO of NeuReality.
“Function for function, hardware runs faster and parallelizes much more than software. As an industry, we’ve proven this model, offloading the deep learning processing function from CPUs to DLAs such as the GPU or ASIC solutions... NR1 offers an unprecedented competitive alternative to today’s general-purpose server solutions, setting a new standard for the direction our industry must take to fully support the AI Digital Age.”
“We are excited about our first generation NAPU product, proven, tested, and ready to move to manufacture. It’s full steam ahead as we reach this highly anticipated manufacturing stage with our TSMC partners. Our plan remains to start shipping products directly to customers by the end of the year,” said Tanach.
The Israeli company raised USD35 million last year to bring its product to the market in 2023.
Paul Mah is the editor of DSAITrends. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose. You can reach him at [email protected].
Image credit: iStockphoto/Tom Merton
Paul Mah
Paul Mah is the editor of DSAITrends, where he report on the latest developments in data science and AI. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose.