Meta Abruptly Pulls Galactica ‘Science’ Model After Launch
- By Paul Mah
- November 30, 2022
Meta unveiled a new language model called Galactica on November 15 but pulled the public demonstration just three days later after intense criticism.
Galactica was trained on a large corpus that comprises more than 360 million in-context citations and over 50 million unique references across a diverse set of sources. With 48 million examples of scientific articles, textbooks, and encyclopedias, Galactica was supposed to suggest citations and help discover related papers.
When I looked at examples offered on the Galactica site that I had domain knowledge of, I found that Galactica could explain programming code in plain English or translate a simple block of code between programming languages such as Python, JavaScript, and C++.
So why was it taken down?
The problem of large language models
It turns out that Galactica suffered from the same problem that afflicts all other large language models (LLM) – it didn’t understand the science behind what it was generating.
This meant that it couldn’t differentiate truth from falsehood and culminated in erroneous assertions. And as reported in the MIT Technology Review, it was also made up of fake papers that were sometimes attributed to real authors, including generating bogus wiki articles.
One quote in the MIT Technology Review report that summed up the situation came from Chirag Shah, an associate professor at the University of Washington.
Shah said: “I am both astounded and unsurprised by this new effort. When it comes to demoing these things, they look so fantastic, magical, and intelligent. But people still don’t seem to grasp that in principle such things can’t work the way we hype them up to.”
Some scientists have pushed back hard, too. Software engineer Grady Booch who co-developed the Unified Modeling Language wrote on Twitter: “Galactica is little more than statistical nonsense at scale. Amusing. Dangerous. And IMHO unethical.”
Why Galactica is dangerous
At its heart is how content from Galactica can be misleading in subtle ways yet written in an authoritative voice with jargon that is appropriately scientific. The danger is real that non-truths in highly specialized fields might not be spotted even by scientists not versed in that niche.
Crucially, fake papers pulled out of thin air by Galactica could end up being cited by other experts taken in by false assertions, unwittingly perpetuating and entrenching the falsehood.
Aaron Snoswell, a post-doctoral research fellow and Jean Burgess, a professor at the Queensland University of Technology, summed up their experience with Galactica: “We [asked] Galactica to explain technical concepts from our own fields of research. We found it would use all the right buzzwords, but get the actual details wrong – for example, mixing up the details of related but different algorithms.”
“Galactica takes this bias towards certainty, combines it with wrong answers and delivers responses with supreme overconfidence: hardly a recipe for trustworthiness in a scientific information service,” wrote Snoswell and Burgess.
A breakthrough AI model
To be clear, Galactica is a breakthrough in many ways. According to the white paper published by Meta, it outperforms the latest GPT-3 with LaTeX equations and outperforms DeepMind’s GPT-3 rival Chinchilla on mathematical MMLU, and Google’s 540-billion parameter PaLM model on MATH.
Galactica also outperforms BigScience’s open-source BLOOM model and OPT-175B on BIG-bench.
The team behind it believes that a large language model offers an advantage over search engines in that it can store, combine, and reason about scientific knowledge in ways traditional search engines cannot. They alluded to its potential to address information overload in science, and said it made the approach worth exploring.
“We believe these results demonstrate the potential for language models as a new interface for science,” wrote the authors in a paper released alongside Galactica.
You can read the paper “Galactica: A large language model for science” here (pdf).
Paul Mah is the editor of DSAITrends. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose. You can reach him at [email protected].
Image credit: iStockphoto/billyfoto
Paul Mah
Paul Mah is the editor of DSAITrends, where he report on the latest developments in data science and AI. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose.