I did some deep diving on how AI/LLMs/chat products work recently, and now I can spot how this thread and the comparisons between "models" is completely nonsensical

Anthropic's CEO claims AI hallucinates less than humans.

Bold statement.

So I decided to test it by feeding the same FAKE theories to ChatGPT, Claude, and Gemini to see which one calls me out first.

The results shocked me 🧵 https://t.co/mBXGpZMaRj

Alex Vacca

I did some deep diving on how AI/LLMs/chat products work recently, and now I can spot how this thread and the comparisons between "models" is completely nonsensical https://x.com/itsalexvacca/status/1927393691267690922

Head of Product at @syky /digital-fashion. Love all dogs and especially my own. ex-Ledger

Decentralized open source consciousness | Values-aligned communities | citizen of Black Rock City

The UIs we use are not just “model”

They are: model + prompt + tools

Catching you in a lie has everything to do with the system prompt and the access to tools, not the model itself

Disagree - the post-training of the model, which is an immutable part of the model, has a lot to do with it

Sure, but use the APIs only and get *wildly* different answers to a question like this

Conversely, use Gemini API (which he says is worst) and adjust prompt and direct to use tools

Lemme know when you try it!

Might be pretty easy to do with Open Router UI

*for something that leverages tools to answer this particular question, like o3. 

Just send the same question to API only. I guarantee it’ll be extremely different