Nobody Knows Why Large Language Models Can Do the Things They Do
- By Paul Mah
- March 13, 2024
Generative AI has proven to be an incredibly powerful and versatile tool for creating new content and solving complex problems. Since the release of ChatGPT, it has captured the attention of the world and sparked widespread interest and discussion about the future of AI.
The only problem? We don’t actually know how large language models (LLMs) do the jaw-dropping things that they have done, or what their true capabilities and limitations are.
A report on MIT Technology Review delved into the mysteries around generative AI, offering one of the best explanations of the mystery of LLMs yet.
Not adding up
For instance, researchers at OpenAI two years ago discovered a phenomenon where a model would seemingly fail to learn a task, and then get it all of a sudden. The phenomenon was termed “grokking”, and while the wider research community has much to say about it, there’s no consensus as to why it happens.
Grokking is but one of several odd phenomena that AI researchers are scratching their heads over. Indeed, many of those observations of modern AI models fly in the face of classical statistics, which had traditionally helped scientists explain how predictive models work.
And while there is no questioning the runaway success of deep learning technology powering today’s generative AI boom, nobody knows exactly how or why it works.
As noted by computer scientist, Mikhail Belkin, from the University of California, San Diego: “Obviously, we’re not completely ignorant. But our theoretical analysis is so far off what these models can do. Like, why can they learn language? I think this is very mysterious.”
Working backward
The best way to describe how researchers are working to unravel AI is that they are working backward. The following two quotes probably epitomized the current state of affairs best.
“Many people in the field often compare it to physics at the beginning of the 20th century. We have a lot of experimental results that we don’t completely understand, and often when you do an experiment it surprises you,” said Boaz Barak, a computer scientist at Harvard University and on secondment to OpenAI.
Indeed, professors teaching AI today are focusing on the how, not the why, says Hattie Zhou, an AI researcher at the University of Montreal and Apple Machine Learning Research.
“It was like, here is how you train these models and then here’s the result. But it wasn’t clear why this process leads to models that are capable of doing these amazing things,” she said. “[I assumed] scientists know what they’re doing. Like, they’d get the theories and then they’d build the models. That wasn’t the case at all.”
The issue of safety
Figuring generative AI models out would be a crucial step towards controlling more powerful future models, and help us build even better AI, of which progress has been fast – but unpredictable. Specifically, researchers are still arguing about what is possible or unachievable with today’s AI models.
Of course, the greatest issue is safety. After all, without a complete understanding of the theory behind the science, how can we predict the capabilities that may emerge? Put another way, one won't know what GPT-5 or GPT-6 is capable of until it is trained and tested. But what if it goes rogue?
For now, people like Barak are working on OpenAI’s superalignment team – set up by OpenAI’s chief scientist Ilya Sutskever, to figure out means of stopping a superintelligence from going rogue. One has to hope it works.
Image credit: DALL-E 3
Paul Mah
Paul Mah is the editor of DSAITrends, where he report on the latest developments in data science and AI. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose.