Meta Creating New AI Model to Challenge GPT-4
- By Paul Mah
- September 13, 2023
Meta has been snapping up AI processors to create a new chatbot to rival OpenAI's GPT-4, according to the Wall Street Journal.
Specifically, Meta has been buying more Nvidia H100 AI-training chips and is beefing up its infrastructure to do it entirely in-house. It is expected to rank 2nd in the world next year in regards to its number of H100s, researchers said.
As we reported last year, Meta has already built the world’s largest AI supercomputer with 760 Nvidia DGX systems powered by a total of 6,080 A100 GPUs. The A100 is the predecessor to the even more powerful H100.
Training a new AI model
The AI will reportedly be primarily aimed at businesses as a tool for producing text and analysis, similar to ChatGPT enterprise. Training for the new large language model will start early in 2024.
According to the Journal, the new AI model is expected to be ready next year and is expected to be several times more powerful than Llama 2 which was released this year to compete with ChatGPT and Bard.
As we reported in July, LLaMa 2 was trained on 40% more data compared to Llama 1, and Meta says it outperforms open-source chat models “on most benchmarks”.
Unlike its predecessor, LLaMa 2 can be integrated into commercial products. It is also available in a variety of models that range in size from 7 to 70 billion parameters and are pretrained on 2 trillion tokens with a context length of 4,096.
Meta has increasingly shifted its focus toward AI, as its efforts at championing the metaverse have floundered.
However, it is facing a challenge to keep up with competitors like OpenAI and Microsoft. Internal memos from last year, obtained by Reuters, revealed that the company acknowledged a "significant gap" in AI research and investment.
Since the launch of OpenAI's ChatGPT late last year, businesses and enterprises have been drawn to the emerging generative AI market to acquire new capabilities and enhance their business processes.
Paul Mah is the editor of DSAITrends. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose. You can reach him at [email protected].
Image credit: iStockphoto/Ociacia
Paul Mah
Paul Mah is the editor of DSAITrends, where he report on the latest developments in data science and AI. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose.