Microsoft Reveals Tech Behind AI-powered Bing
- By Paul Mah
- March 01, 2023
Since announcing its plan to integrate the AI technology powering ChatGPT into its Bing search engine, Microsoft has been hard at work rolling out its new AI-infused Bing search engine around the world.
But how is it implemented, and what does the architecture look like? A recent LinkedIn post by Jordi Ribas, chief vice president at Microsoft for Search and AI shed some light on these questions.
Notably, Ribas revealed that the new Bing service is powered by a newer generation of the GPT model from OpenAI than what it has demonstrated publicly, and is ahead of GPT 3.5 which currently powers ChatGPT.
“The new model was much more powerful than GPT-3.5, which powers ChatGPT, and a lot more capable to synthesize, summarize, chat and create,” said Ribas, who also called it a breakthrough in large language models (LLMs).
Next Generation GPT
The team sought to overcome the inherent limits of LLMs trained with data from a given point in time by combining it with Bing’s back end. To enable that, the team developed a proprietary solution called “Prometheus” to combine fresh results and index from Bing with the “creative reasoning capabilities” from OpenAI’s most advanced GPT models.
All queries will funnel through a new Bing Orchestrator, which will forward basic queries to Bing directly. For Chat answers, queries will go to the Next Generation GPT which will generate multiple internal queries as needed to provide an “accurate and rich answer for the user query” within a given conversation context. This happens within a matter of milliseconds.
“Selecting the relevant internal queries and leveraging the respective Bing search results is a critical component of Prometheus since it provides relevant and fresh information to the model… the model reasons over the data provided by Bing and hence it’s grounded by Bing data.”
Finally, Prometheus will also integrate citations into sentences in the Chat answer so that users can easily click to access those sources and verify the information, says Ribas.
Microsoft determined that some queries are better served via traditional search results, while others might work better through Chat. This led to the development of a new UX that unified Search and Chat on a single interface, and which allows users to easily transition from Search to Chat – and back.
Improvements ahead
The Bing team is working hard to further reduce inaccuracies and prevent offensive and harmful content. This is why the Bing preview continues to collect user feedback to further improve Prometheus, says Ribas.
And it turns out that very long chat sessions can confuse the underlying chat models, resulting in less accurate outcomes or a tone that was not intended. It is for this reason that caps to chat sessions were introduced.
One upcoming improvement is to quadruple the amount of grounding data that is sent to the model to improve the Chat answer accuracy. In addition, Microsoft is also considering adding a toggle that gives users more control to optimize their experience depending on their tasks and goals.
According to Ribas, the incorporation of AI and other innovations has resulted in a search experience that “will never be the same”.
“Users of the new Bing will get more complete and contextual answers, which in some cases may save them hours of research in a search session, and that means more time back to spend on the things that matter. And we hope along the way you’ll also discover that search can be more interactive and fun,” he summed.
Paul Mah is the editor of DSAITrends. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose. You can reach him at [email protected].
Image credit: iStockphoto/Pogonici
Paul Mah
Paul Mah is the editor of DSAITrends, where he report on the latest developments in data science and AI. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose.