AI In Your Hands, Off-line (Phi-3, Llama 3)

What Happened? We already had the news that Meta released Llama 3, with the smallest model coming in at a particularly portable 8 billion parameters. But did you know that, at least for English-language prompts, it is competitive in a human-blind-graded leaderboard with the June ‘23 version of GPT-4?

And that today, we got phi-3.3B, which rivals Llama 3 8B, and which generates about 10 words/second on an iPhone 15, off-line?

• New lmsys leaderboard rankings (which are esteemed by Andrej Karpathy), show Llama 3 70B beating Claude 3 Opus, and Llama 3 8B beating June’s GPT-4 (should soon have it via Groq for SmartGPT 2.0).

• Phi-3 was released by Microsoft, with paper details on the Mini 3.3B model up to the 14B model. Benchmark performance was insane, with the 14B model (interestingly called ‘Medium’ - what else do they have?) handily beating GPT 3.5.

• While there are known issues with benchmarks, and reviews are still coming in for both model families, it is increasingly obvious datasets are the key for unlocking greater model performance.

So What? Having top-grade LLMs on our phones and laptops natively might not just embarrass Siri, they might unlock genuine use-cases across the world. Travelling somewhere remote and want first-aid tips with on signal? Basic tutoring needed in a developing-world province without reliable internet? Want to chat with models while you are on a subway without signal? Not as important as state-of-the-art models perhaps, but useful for many.

Moreover, the central path forward for significantly smarter top-level LLMs is now clear. Train on more, and better filtered, data, re-using the best data until performance is maximised, as Mark Zuckerberg reiterated this week.

Does It Change Everything? Rating = ⚂

Become a paying subscriber of Insider Essentials to get access to this post and other subscriber-only content.

Already a paying subscriber? Sign In