pinata

These two images tell a fascinating story 

On the left, we have a server running on a GPU enabled machine that can run local AI models through Ollama, and with x402 monetize that usage

On the right, we have a request being made to that server using the OpenAI SDK and x402-fetch 

Distributed AI is near, and so is the blog post on this experiment

DX Engineer | Head of Developer Relations at /pinata | Building orbiter.host | stevedylan.dev/links | opinions are my own

why would you distribute the inference infra? what's the forcing function? def is not economical/private. what is it then? how does on-device come into play here? genuine question.

Admittedly it’s not a complete solution but conceptually it sets up a world where higher compute hardware is more accessible outside of large central providers, and for that hardware to be monetized.

Yeah this feels like a sketch of what automatic and self service AI looks like, not so much "decentralized" but rather something between federated and distributed.

Im the guy that posted all those factory memes once

Everyone is from somewhere. Cofounder and CEO of Pinata. https://www.pinata.cloud/

100%, to draw an allegory from jet engine development:
We know what the goal is, we know we can get there, but its a matter of tuning everything just right to get the loop to complete in a useful way. 

Some examples of issues that need solving:
Context length restrictions - Real time world knowledge is limited
Context Retrieval at long lengths - In task knowledge is limited
Model Speed (this is getting solved pretty effectively over time) - Feedback cycle during agentic execution
Vector-Goal Divergence - agents going off an tangents (looking at you 3.7 sonnet)
Model Alignment - Prompts are lossy, this compensates for that  

All of this can be compensated for through things like RAG and scaffolding and eventually these issues will be solved. 

Once they are then its just a matter of getting tool_calls wired up, for physical stuff see work @july is doing. 

Crypto then provides payment rails. 

Then fully agentic systems can emerge. 

10 years feels right.

I feel like we are stumbling around in the dark trying to figure out what the room is that we are in. We've stubbed our toe on a chair so far but don't know what else is around us.

If AI is going to be as dynamic as people think, access to computing definitely needs to have less boundaries but we don't have good examples of that today.

In 10 years, we will be able to point at Steve's example as a primitive example of everyday computing but we are probably missing some key components right now.