OpenAI’s Mira Murati: LLMs Show No Sign of Plateauing

Few figures stand out like Mira Murati, the CTO of OpenAI. Lauded by Microsoft CEO Satya Nadella for her ability to blend technical expertise with commercial insight and a visionary outlook, Murati was credited with helming transformative AI projects at OpenAI such as ChatGPT, DALL-E, and GPT-4.

At the Qualtrics X4 conference in May, she was interviewed by Gurdeep Pall, president of AI Strategy at Qualtrics on stage. She spoke at length about the potential of AI, what surprised her about GenAI, and what’s next at OpenAI, offering a glimpse into how OpenAI plans to navigate the future of AI.

How GenAI surprised

“From a technology perspective. We believed in scale, and we bet on the scaling paradigm. [It’s] this idea that you throw a ton of compute and data at this large language models, and that it will lead to emerging capabilities, that models will become more powerful and be able to do more things.”

And it turns out that even the researchers at OpenAI were astounded by the results: “It's one thing to sort of statistically predict the performance of the model and another to actually see the capabilities when you test them across different domains. When you see that the models can rhyme, do extremely well in biology tests or math tests - tests that we use to test people in colleges, that was a surprise.”

Just like how GPT-4 was significantly better than GPT-3.5 in reasoning across multiple domains, Murati says she expects the next generation of GenAI model to deliver another “step change” in the form of new capabilities.

The potential of AI

Murati says she is personally excited about the potential of AI in education and healthcare, given the opportunities to change the quality of life.

“[AI] can help us get to a place where, you know, most people have a high-quality of life, [where] they have free and high quality health care and education,” she said.

She pointed to how schools such as the Khan Academy and Carnegie Mellon are already using AI models from OpenAI to develop curricula personalized to individual students, a paradigm shift of having one teacher teaching 30 to 40 students.

“There's only so much capacity to teach in bespoke ways and motivate the students in a very personal way. But with AI tutors that's very achievable; to have a very personalized form of education and really help people learn and to enhance their creativity.”

What’s next at OpenAI

When asked what advancements we can look forward to with GenAI, Murati talked about multi-modality and also revealed something that observers had long pointed out: OpenAI isn’t offering a free version of ChatGPT for completely altruistic reasons but using the insights for reinforcement learning.

“As these models have the opportunity to interact with the world… interact with humans and get feedback… And we use that feedback to align the model to be more helpful and more useful… and allow us to make these technologies more steerable and useful,” she said.

“Expect a lot of multi-modality, both perception and generation coming into [new] models. That's important because we want to impute these models with a sense of the world and understanding of the world… similar to how humans interact with each other in ways that go beyond language. We want to [bake] that into AI models.”

Finally, expect Nvidia to sell many more GPUs in the foreseeable future, as OpenAI has no plans to let up on training ever-larger AI models.

“[We] will continue to push the scaling paradigm so models get more powerful as we put more compute, high-quality data, and more data into them. There is no evidence of the scaling paradigm stopping anytime soon; we should expect AI models to become much more powerful [just by scaling up].”

Image credit: iStock/sabeltp