NEC Claims Its New LLMs Are Faster Than GPT-4
- By Paul Mah
- May 01, 2024
NEC Corporation says it has developed two new large language models (LLMs) that it claims deliver the same high performance as global LLMs but at more than ten times the inference speed.
Specifically, NEC says its “NEC cotomi Pro" model achieves performance comparable to OpenAI’s GPT-4 and Anthropic’s Claude 2, with a response time that is approximately 87% faster than GPT-4 using two Nvidia L40 GPUs. In addition, the faster "NEC cotomi Light" model has the same level of performance as global models such as “GPT-3.5-Turbo.”
When the system is used with an in-house document retrieval system using retrieval-augmented generation (RAG), NEC says it achieved a correct response rate higher than GPT-3.5 without fine-tuning and a correct response rate higher than GPT-4 after fine-tuning, with a response time that is approximately 93% faster.
As noted by NEC in its press release, larger AI models offer better performance but tend to respond slower. However, the Japanese firm says it had succeeded in improving both speed and performance with the development of an advanced new training method and architecture.
NEC credited the improvements to architectural innovations that enhanced both performance and speed, as well as a large Japanese dictionary (for tokenization) in the model. It did not offer details of its architectural innovations.
NEC cotomi Pro provided results more than five times faster than GPT-4 on a standard server with two GPUs. It also outperformed the speed of models such as “Gemini 1.0 Pro” and showed performance that is comparable to Claude 2 and GPT-4.
At the same time, "NEC cotomi Light” provided results that are more than 15 times faster than GPT-4 while outperforming large models such as “LLaMA2-70B” and demonstrating performance comparable to GPT-3.5-Turbo.
It is worth noting that LLM performance is a contentious issue due to the absence of a globally accepted methodology for testing AI models. LLMs can perform exceptionally well in certain niches but fare poorly when quizzed on other topics. Moreover, the state-of-the-art AI models that NEC tested its LLM against are hosted entirely in the cloud, so this is hardly an apples to apples comparison.
For now, NEC says it will continue to strengthen cooperation with partners and provide safe, secure and reliable AI services based on the expanded NEC cotomi lineup to help customers solve a wide range of complex challenges.
Image credit: iStock/jiefeng jiang
Paul Mah
Paul Mah is the editor of DSAITrends, where he report on the latest developments in data science and AI. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose.