sidhant pfp

sidhant

@sidhant

64 Following
63 Followers
sidhant pfp
sidhant
@sidhant
reminds me of that old legal office on Rainey!
0 reply
0 recast
1 reaction
sidhant pfp
sidhant
@sidhant
on-device inference is the future of AI-driven games. if core game logic runs off a paid API, your biggest fans become a liability. can鈥檛 charge per hour, you'd be punishing gamers that play the most. edge models let gamers pay once, play forever. like the Nintendo switch: portable, offline, open-ended.
0 reply
0 recast
9 reactions
sidhant pfp
sidhant
@sidhant
what's the best tiny (<1B) LLM? torn between gemma and qwen for fine-tuning for tool calling on mobile phone processors. hearing interesting things about smollm too
0 reply
0 recast
0 reaction
sidhant pfp
sidhant
@sidhant
trying to come up with a simple LLM driven video game idea. simple level of intelligence, something that can run on an edge model like Gemma, perhaps with some fine-tuning. game logic and visuals generated so it's more dynamic than what's possible with RNG and human generated assets.
0 reply
1 recast
8 reactions
sidhant pfp
sidhant
@sidhant
who's tryna make this?
1 reply
0 recast
1 reaction
sidhant pfp
sidhant
@sidhant
I'm not ready to write my own transformers yet! maybe some day
1 reply
0 recast
1 reaction
sidhant pfp
sidhant
@sidhant
planning on trying some fine-tuning experiments this weekend. unsloth seems pretty good. the goal is to get a small edge LLM tuned on simple video game logic tasks like pathfinding to see if it's feasible to run inference at game engine speeds on a mobile processor
1 reply
0 recast
5 reactions
sidhant pfp
sidhant
@sidhant
the end result is kinda useless yeah. but I'm glad at least one major OS is shipping privacy focused edge LLMs instead of stealing all your data like Google and Microsoft
2 replies
0 recast
2 reactions
sidhant pfp
sidhant
@sidhant
Apple intelligence gets a lot of hate (rightfully so) for overpromising and underdelivering, but I feel like it's super slept on too. embedded, purpose built edge LLMs for small simple tasks super fast. the best part is you're getting a little bit of intelligence without handing all your private data to a 3rd party.
3 replies
0 recast
19 reactions
sidhant pfp
sidhant
@sidhant
this is pretty wild! thanks for the link, I had no clue this was possible
1 reply
0 recast
2 reactions
sidhant pfp
sidhant
@sidhant
is there a diffusion model capable of running on a cellphone with decent results?
1 reply
0 recast
1 reaction
sidhant pfp
sidhant
@sidhant
how pricey would the fee have to be to justify all the tool calls? I pay $50+ a month for my coding tools. that would only be justified for massive AAA blockbuster type games (and even those tend to be $60-$70 one time). I wanna keep it lean and target the indie space, sell a small simple game for $10-$20. even if it only costs me 1垄 per tool call, that's infinitely more expensive than running on the player's hardware for free.
1 reply
0 recast
0 reaction
sidhant pfp
sidhant
@sidhant
it should be illegal to label this mix of corn syrup, citric acid and various petrochemical flavorings "lemonade"
0 reply
0 recast
1 reaction
sidhant pfp
sidhant
@sidhant
it's even more insane than that. "Gemma 3n models use selective parameter activation technology to reduce resource requirements. This technique allows the models to operate at an effective size of 2B and 4B parameters, which is lower than the total number of parameters they contain."
1 reply
0 recast
2 reactions
sidhant pfp
sidhant
@sidhant
for the most part the results are still garbage, but every now and then there's a good one!
0 reply
0 recast
1 reaction
sidhant pfp
sidhant
@sidhant
all I needed was a better prompt! this is my most successful result so far trying to use gemma3n on-device to do more than just text generation!
1 reply
0 recast
2 reactions
sidhant pfp
sidhant
@sidhant
update: I'm getting way better results with the JSON shape renderer approach. might actually be able to get this to work with better prompting!
0 reply
1 recast
0 reaction
sidhant pfp
sidhant
@sidhant
update: I'm getting way better results with the JSON shape renderer approach. might actually be able to get this to work with better prompting!
0 reply
1 recast
0 reaction
sidhant pfp
sidhant
@sidhant
not yet! I assume they'd be faster but less "smart"
0 reply
0 recast
1 reaction
sidhant pfp
sidhant
@sidhant
nope, I don't think google has made that available yet
1 reply
0 recast
2 reactions