TL;DW Context Rules Everything Around Me
https://youtu.be/nyvmYnz6EAg

A monolithic slop firehose;
A slow deluge of impotent garbage;
The guy that posted all those factory memes once

Everyone is from somewhere. Cofounder and CEO of Pinata. https://www.pinata.cloud/

The problem reminds me of a sales book I read from 1921.

The guy was a traveling salesman, riding trains across the country. 

To get context about the city he was in, pre-information age obviously, he'd go to all the local hotels and talk with the front-desk.

He'd find out two things: 

#1: Learn about the city. The economy, the people, where businesses were, etc.

#2: What other salesman had been through. 

This gave him the context needed to then sell effectively and efficiently in the unknown city.

Medium term it's just increasing context length, long term it's algorithmic improvements towards correctly scoping/compressing context like you said and improved transformers/new paradigms

Could I assume we need to focus on generating precise context on-demand to see a leap forward?