sidhant

@sidhant

64 Following

63 Followers

sidhant pfp

on-device inference is the future of AI-driven games. if core game logic runs off a paid API, your biggest fans become a liability. can’t charge per hour, you'd be punishing gamers that play the most. edge models let gamers pay once, play forever. like the Nintendo switch: portable, offline, open-ended.

0 reply

0 recast

9 reactions

sidhant pfp

what's the best tiny (<1B) LLM? torn between gemma and qwen for fine-tuning for tool calling on mobile phone processors. hearing interesting things about smollm too

0 reply

0 recast

0 reaction

sidhant pfp

trying to come up with a simple LLM driven video game idea. simple level of intelligence, something that can run on an edge model like Gemma, perhaps with some fine-tuning. game logic and visuals generated so it's more dynamic than what's possible with RNG and human generated assets.

0 reply

1 recast

8 reactions

sidhant pfp

who's tryna make this?

1 reply

0 recast

1 reaction

sidhant pfp

planning on trying some fine-tuning experiments this weekend. unsloth seems pretty good. the goal is to get a small edge LLM tuned on simple video game logic tasks like pathfinding to see if it's feasible to run inference at game engine speeds on a mobile processor

1 reply

0 recast

5 reactions

sidhant pfp

Apple intelligence gets a lot of hate (rightfully so) for overpromising and underdelivering, but I feel like it's super slept on too. embedded, purpose built edge LLMs for small simple tasks super fast. the best part is you're getting a little bit of intelligence without handing all your private data to a 3rd party.

3 replies

0 recast

19 reactions

sidhant pfp

all I needed was a better prompt! this is my most successful result so far trying to use gemma3n on-device to do more than just text generation!

1 reply

0 recast

2 reactions

sidhant pfp

I'm so obsessed with the idea that generative AI is the future of video games. the only problem is unit economics. gamers expect to spend $20-$60 one time for hours and hours of entertainment. nobody wants to pay a monthly sub. so: edge llms. run the game logic on device. no api fees. I've been playing around with this idea, my current experiment uses gemma3n running on my iphone to generate svg graphics. here’s it drawing a “puppy.” sadly this attempt ended up being a dead end. does anyone have better ideas for representing visual output from an edge llm? maybe some kind of json shape primitive -> sprite renderer thing?

4 replies

2 recasts

40 reactions

sidhant pfp

open source is a hell of a drug. i dropped an ios port of gemma 3n yesterday, just a few days after google released it (with android-only support), and already someone else jumped in, added image understanding, polished the UX, and is prepping a PR. insane. this is what it’s all about — no NDAs, no permission slips, just vibes and shipping. gemma 3n is a frontier edge model, and rn we’re watching a legit community ecosystem form around it in real time. that’s magic. huge thanks to @TheMagicIsInTheHole on reddit — here’s the screenshot they shared 👇

6 replies

2 recasts

46 reactions

sidhant pfp

🚀 just dropped the first-ever gemma 3n model running fully on ios — yes, on-device, no cloud. 📱 it’s slow (for now), but it benchmarks nearly on par with claude 3.7 sonnet for non-coding tasks. 🧠 gemma 3n is google’s new mobile-first frontier model, designed for edge devices. 🔧 i built a simple ios app to run it locally: 💬 features: on-device inference, real-time chat ui, streaming responses. ⚠️ it’s a bit slow, but it’s a start. 👀 try it out and see the future of on-device ai. https://github.com/sid9102/gemma3n-ios

3 replies

5 recasts

32 reactions