Full Title: 🌌 AI Drones Are Paving the Way for Autonomous Spaceflight
Highlights
If you’ve used LLMs long enough, you’ve probably come up against a pesky problem: Suddenly, your go-to prompt for completing a certain task no longer works. Or, maybe you’re trying out someone else’s prompt and you find that their method gives you a completely different result. (View Highlight)
In many cases, updates are to blame for those inconsistencies: Developers fine-tune their models to cut down on hallucinations, glitches, and other problems. But every time there’s an update, some of that work gets undone. As a result, an LLM might struggle with certain things — from reasoning problems to writing tasks — that it previously excelled at. (View Highlight)
What can we do about it? Apple researchers are trying to figure out a way to make the transition between old and new models easier to adjust to. It’s a tricky problem because negative flips — when an AI model gets wrong what it previously got right — are common even when an LLM’s training methods are kept exactly the same across generations. (View Highlight)
How does it work? The researchers created a set of metrics that can be used to spot differences between each version of an LLM — in this case, Meta’s open-source Llama 1 and Llama 2. Then, using that data, they taught a specialized LLM known as a compatibility model how to flag discrepancies on its own. This approach slashed inconsistencies by about 40%. (View Highlight)
What it means for you: Whenever there’s a new update, we often focus on a model’s number of parameters and tokens — or its performance across different benchmarks. But just as important is the experience of actually using that LLM. Apple’s research shows that tech companies are paying closer attention to the inconsistency problem and want to cut down on it. (View Highlight)
