TL;DW Context Rules Everything Around Me
https://youtu.be/nyvmYnz6EAg

A monolithic slop firehose;
A slow deluge of impotent garbage;
The guy that posted all those factory memes once

Could I assume we need to focus on generating precise context on-demand to see a leap forward?

Everyone is from somewhere. Cofounder and CEO of Pinata. https://www.pinata.cloud/

Medium term it's just increasing context length, long term it's algorithmic improvements towards correctly scoping/compressing context like you said and improved transformers/new paradigms

The problem reminds me of a sales book I read from 1921.

The guy was a traveling salesman, riding trains across the country. 

To get context about the city he was in, pre-information age obviously, he'd go to all the local hotels and talk with the front-desk.

He'd find out two things: 

#1: Learn about the city. The economy, the people, where businesses were, etc.

#2: What other salesman had been through. 

This gave him the context needed to then sell effectively and efficiently in the unknown city.