LLM in Petite Size: Apple Thinks It’s the Future

The launch of Apple Intelligence shows an alternative route for tech companies to make AI more democratic and pervasive.

Ever since ChatGPT sprung into our consciousness, large language models or LLMs have shaped our collective imagination, promising everything from the perfect personal assistant to a step closer to the singularity.

One catch: it’s size. LLMs are, well, large. They’re the data-guzzling, energy-hungry giants of the AI world, demanding the power of massive data centers. The idea of squeezing one into your phone? Laughable — until now.   

Mini-me Language Models

Small language models, or SLMs, make perfect sense in today’s device-driven and autonomous-crazed world.

But SLMs aren’t just about convenience. They’re about privacy. No more sending your deepest thoughts and queries to some far-off server farm for harvesting. With an SLM, your conversations stay on your device, safe from prying eyes.

Initially, SLMs were diet versions of LLMs, capable of specific tasks but far from the all-knowing oracles we’ve come to expect. Think of them as specialized AI engineers, brilliant at troubleshooting a technical glitch but clueless when it comes to planning your daughter’s birthday party.

But SLMs have come a long way since then. Meta’s Llama 3 and Microsoft’s Phi 3 can run on laptops and smartphones. They don’t have to access all the training data in the model for reference. Instead, it only needs what AI engineers call model “weights,” floating points stored in multidimensional number lists called tensors.

In short, once trained, the weights (determined during the training phase) will guide the model in predicting the next most likely token using statistics. It’s like teaching a dog to fetch. You don’t need to show them every possible ball in the world; once they understand the concept of ‘fetch,’ they can apply it to any ball you throw.

This makes SLMs perfect for edge devices, our laptops, smartphones and other IoT devices. It also means better human efficiency and augmentation minus the cloud lag.

Shrink science

Fitting an AI model in the palm of your hand is no small feat. It’s like trying to cram a library into a matchbox. AI engineers have developed clever tricks to pull it off, like pruning, quantization and palletization.

Pruning involves packing those “weights” more efficiently. The other two methods involve having fewer bits per weight. Apple opted for the last two.

Compression has its trade-offs: it creates a lossy model, so model accuracy suffers. But just like music compression, we are getting better at maintaining model accuracy where it matters most. In Apple’s case, they used a better method to store tensor lookup tables using a new framework based on low-rank (LoRA) adapters. This allows them to claim near-lossless performance for their compressed models.

Apple’s almost 3 billion parameter on-device language model is also not alone. It will be paired with a larger server-based language model available with Private Cloud Compute, Apple’s cloud-based intelligence system for private AI processing.

Opening up new possibilities  

Apple may be late to the AI party, but it’s not just playing catch-up. When ChatGPT went public, many wondered whether “Hey Siri” would stop being part of our vocabulary. Companies like Google and Microsoft, shifted gears. The former fast-tracked their lab project into production while Microsoft became pals with Open AI.

Yet, Apple is now deftly playing its second-mover advantage. Sure, like any other tech company, LLMs will feature in the company’s near future. It has already announced that it will work with Open AI to integrate ChatGPT into its device OSes. However, with Apple Intelligence, the company is also charting its own course, focusing on the potential of on-device AI.

Focusing on SLMs is also strategically smart. Apple has always been realistic about its customers. Ever since Steve Jobs launched the first iPhone in 2007, they have been mostly consumers with an iPhone or iPad in hand.

The recent WWDC24 announcements show how the company is making the same devices more useful. They detail how Apple Intelligence will transform the iPad and iPhone into becoming more context-aware while running on Apple chips that the company controls.

This opens new possibilities. Imagine a world where your phone anticipates your needs, offering personalized suggestions based on your location, schedule, and preferences. No more generic responses or irrelevant recommendations.

Now, there are two

To be sure, SLMs will not replace LLMs anytime soon. But Apple betting its future on it shows that it is now a serious alternative — something that many other enterprises struggling to create an LLM use case will take note of.

It also hints at a future where we interact with an ecosystem of LLMs and SLMs. Such a scenario will blur the lines between the digital and physical worlds, introduce new types of applications that transcend real-world physics, and remodel how we interact with our environment.

Besides, SLMs can also help infrastructure engineers to meet their Scope 3 requirements. Not every prompt will need a LLM response that requires gallons of water for cooling.

For Apple, it’s a strategic masterstroke. They can now keep their captive audience on their platform and devices (which is what financial analysts care about). It also re-introduces Siri into our lives, which goes beyond telling us about the weather or playing our favorite song.

So the next time you whisper, “Hey Siri,” remember, you’re not just summoning a digital assistant. You’re poking a sleeping giant, one that Apple hopes will wake up and reshape the world, one personalized query at a time.

Who knows, maybe Siri will even finally get your coffee order right.

Image credit: Stockphoto/evgenyatamanenko