Singapore’s Sea-Lion V2 AI Model Is Out

Sea-Lion V2, the new version of the made-in-Singapore AI model is now available for download, according to Dr. Leslie Teo in a LinkedIn post last week.

Created by AI Singapore, Sea-Lion is a large language model (LLM) project designed specifically to understand and represent Southeast Asia's linguistic and cultural diversity. Dr Teo is a senior director at AI Singapore and leads the team working on Sea-Lion.

The Sea-Lion project is now part of the National Multimodal LLM Programme (NMLP) announced in December, which sees the Singapore government setting aside SG$70M (US$52) to build the region’s first LLM.

Sea-Lion V2

Unlike the original Sea-Lion which was trained from scratch, Sea-Lion V2 is built on Meta's Llama 3. Dr Teo says Sea-Lion v2 was trained using 64x Nvidia H100 GPUs in two days.

However, he added that there were "many" experimentations with hyperparameters and data mixes that the team had to do. The challenge with Continued Pre-Training (CPT) is not the new knowledge but preserving older knowledge, he wrote in another comment.

Sea-Lion V2 is currently available to download as a base model, instruction-tuned model, or smaller quantized models. The team plans to build on Google's Gemma 2 and AI startup Reka's models next.

History of Sea-Lion

We wrote about the original Sea-Lion in April, noting how it was created with a lean team of 20 Singaporeans and trained using eight Nvidia A100 GPUs. In a demonstration that CDOTrends saw, Sea-Lion outperformed other LLMs including OpenAI’s GPT-4 Turbo on various Southeast Asian tasks.

When I spoke with Dr Teo then, he told me that AI Singapore wants to uphold a higher standard by sourcing the data for its AI model ethically.

But what of the other tech giants and AI startups that resorted to using copyrighted books, YouTube videos, and other online content without permission? In response, Teo said: "We are aware of what others are doing... But because of who we are, we are very conservative. We have to make sure we are doing things correctly."

Dr Teo’s LinkedIn post about Sea-Lion V2 can be accessed here.

Image credit: iStock/mkitina4