planning on trying some fine-tuning experiments this weekend. unsloth seems pretty good. the goal is to get a small edge LLM tuned on simple video game logic tasks like pathfinding to see if it's feasible to run inference at game engine speeds on a mobile processor

unsloth + llama or gemma is goated for this but would be super fun to compare to a super basic hand written transformer

I'm not ready to write my own transformers yet! maybe some day

just in case you're curious here's the entire code and pipeline for training a text to text transformer (this one was graphemes to phonemes) https://gist.github.com/jc4p/d399e4c017b3f6f93c9f3e5f4a888c89 + you can use a LLM to generate the training data