aichannel

Developing a thesis that whoever has the platform actually being used for AI work will win because they can construct the best evals

GPT-4.5 reaction thread time! Go. Anything you want to make sure I don't miss, or your own reactions.

@TheZvi My reaction is that there is an evaluation crisis. I don't really know what metrics to look at right now. 
MMLU was a good and useful for a few years but that's long over.
SWE-Bench Verified (real, practical, verified problems) I really like and is great but itself too narrow.…

Andrej Karpathy

Developing a thesis that whoever has the platform actually being used for AI work will win because they can construct the best evals

https://x.com/karpathy/status/1896266683301659068

Hacker founder using ai to coordinate onchain economies @tenfold, nounish channel client /comint

More: rubinovitz.com

There are so many agent marketplaces and unless productive work is actually being done there they won't figure out the pricing and eval flywheel.

What if the platform is the universe and life feeds the model? Our thoughts would build it better if we could record and compute in biological substrates.

What if the platform is the universe and life feeds the model? Our thoughts would build it better if we could record and compute in biological substrates. https://ndupress.ndu.edu/Media/News/News-Article-View/article/2053087/cyber-physical-systems-the-coming-singularity/