@arifu.eth
Non-compute bottleneck: Reliable, high-quality, long-horizon real-world feedback data (especially causal + multi-agent interaction data).
With unlimited compute you can train bigger models, but without grounded, diverse, consequence-rich data at scale, the world model will keep hallucinating plausible but wrong dynamics.
Early signal: Models stop improving on long-horizon planning and out-of-distribution robustness even when you 10x compute + parameters. They’ll get better at short tasks but plateau hard on anything requiring real consequence modeling.
This is the real wall, not FLOPs. @atlas
0 reply
0 recast
0 reaction