Google Unveils Next-gen PaLM 2 Model
- By Paul Mah
- May 17, 2023
As expected, Google announced a slew of AI-powered products and features at its annual I/O conference last week, including its latest large language model, PaLM (Pathways Language Model) 2.
We wrote about the original PaLM when it was released last year, noting its impressive natural language understanding, the ability to fix broken C programs until they compile successfully, and high-quality explanations for unique jokes not found on the web.
More about PaLM 2
On its part, PaLM 2 is touted as faster and more efficient than previous models, with improved logic, mathematics, and common sense reasoning capabilities. Notably, the next generation PaLM 2 is smaller than PaLM, thanks to compute-optimal scaling by proportionately scaling the model size and training dataset size with each other.
According to Google, PaLM 2 is heavily trained on an improved dataset mix consisting of multilingual text from more than 100 languages. This significantly improves its ability to understand, generate, and translate nuanced text across a wide variety of languages.
The result is that PaLM 2 not only excels at popular programming languages like Python and JavaScript, but can also generate specialized code in languages like Prolog, Fortran and Verilog.
PaLM 2 comes in four sizes: Gecko, Otter, Bison and Unicorn, with Gecko being the smallest. Google says Gecko – the smallest – can work on mobile devices to power interactive applications. The multiple sizes offer ease of deployment, offering the potential to support entire classes of products without the need for Internet access.
So how does this new model perform? In examples cited by Google, PaLM 2 displayed an exceptional understanding of riddles and idioms, which require the ability to understand the ambiguous and figurative meaning of words, rather than the literal meaning. Moreover, it could find bugs in code, correct them, and explain what went wrong.
Speaking to The Verge, Google VP of research Zoubin Ghahramani explained PaLM 2 was also an improvement on earlier models in groundedness and attribution, though he acknowledged that there is still “a ways to go” when it comes to combating false information generated by AI.
You can read the announcement here, and more about PaLM 2 here. The PaLM 2 technical report is available here (pdf).
Paul Mah is the editor of DSAITrends. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose. You can reach him at [email protected].
Image credit: Google
Paul Mah
Paul Mah is the editor of DSAITrends, where he report on the latest developments in data science and AI. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose.