Btw, this is who you’re talking to with when using  www.elroy.computer

DX Engineer | Head of Developer Relations at /pinata | Building orbiter.host | stevedylan.dev/links | opinions are my own

I like baking /goodbread and building apps on web3 protocols. DevRel @ ENS Labs. gregskril.com

Any idea if you're running the 1b or 3b parameter Llama 3.2? I don't have a Jetson Nano so I'm not sure what it can reasonably handle.

I'm super curious how Gemma 3 1b or 4b compare for the sort of stuff you're doing. I don't think the Jetson Nano can run 4b, that might be too big, but it would be cool if it could.

At the moment I think I have it set to just llama3.2, so nothing crazy at all. I’ll need to experiment with some higher models and see what the latency is like.

Neat, is it TinyLlama 1.1b on a Jetson Nano?

Product designer | Pottery /made-by-marge | Building for devs & crypto /pinata