Why do people think context window size is a big limitation for AIs to be able to fully replace humans as coders?

I'd say most of us keep a very compressed version of the codebase in our head and rely on code search and tools to get the details we need. Coding agents are already starting to do this using grep and other tools. They just need that initial context to get them going and then they can do the rest. Creating that initial compressed representation of the codebase they can reference to get them started would do the trick without keeping the entire codebase in the context.

The open web is dead, long live the open web. ⌐◨-◨

Building /farcaster

— gabrielayuso.com

AFAIK the AIs so much better with a large context window because everything has too, at one point, live there. We shouldn't be adapting the problem to the tools but the tool to the problem. LLMs have no fundamental reason they can't pull the whole codebase into context, it's just hardware limits right now.

Im the guy that posted all those factory memes once

It's not only hardware limits. LLM performance diminishes with larger contexts.

It's not that bad and it's not like windowing over data does all that much to fix it. Needle in haystack is improving dramatically over time