That's why you don't throw the kitchen sink at models with large contexts.

The more specific the context the better.

That's also why few shot prompts work really well, it keeps the model focused with clear examples.

When coding I give models specific code that has examples I want to replicate, when applicable.


https://x.com/kimmonismus/status/1931380162274414772?t=Dad16ChnvnKokad2rSb7pQ&s=19

Chain of thought + RAG + few shot + mixture of experts + loops gets you very far

The open web is dead, long live the open web. ⌐◨-◨

Building /farcaster

— gabrielayuso.com

Same as us humans tbh. Maybe we don’t have million-token attention spans for a reason