Apple released analysis on current LLM/LRM capabilities. Fundamentally finding that models essentially “give up” rather than scale their thinking appropriately.

We get both “overthinking” and wild goose chases. Sophisticated pattern matching.

Apple released analysis on current LLM/LRM capabilities. Fundamentally finding that models essentially “give up” rather than scale their thinking appropriately.

We get both “overthinking” and wild goose chases. Sophisticated pattern matching.

https://ml-site.cdn-apple.com/papers/the-illusion-of-thinking.pdf

Their belief is that LLMs/LRMs will get better at doing very specific tasks and require some attention in the loop. Similar to RAG faults.

But again, this won’t scale nicely.