what hardware should i buy to run LLMs locally?

naive question, i know, but i don't have a good sense of where to start. are there useful public guides for deciding? the state of the art is changing rapidly, so figured i'd ask here first...

soldier of Good. 🪖🥰 
producer × DJ. 🎧 🪩 
working on /campfire. 🫴🪵🔥
host of /downshift ⚙️ + /dww 🎧 +  /gtd 🛠️.
https://linktr.ee/0xdownshift

Applied AI, databases, data, vibe coding (sic.), open source, community/devrel, remote. 

itinerant businessperson. globalist.

Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia

yeah feel free to holler here or in DMs. the fact that you also have another use helps lighten the budget load. the key is knowing what you're trying to achieve with a local llm.

awesome pointers!

i love the mac studio, but wonder if it's cost efficient $/flop (i would also use it for audio production, tho)

i'm still early in my research but will follow up when i have a better idea on model size. i don't have a firm budget just yet 

ty 🙏🏼

have you looked at ollama? it can run even on reasonably simple machines, but it is slow if cpu only. you could get a good mac studio, or even string some machines together with exolabs. guess i hae to ask - what is the budget, and what kind of model sizes are you planning to run

Mac mini M4 or 2 running with Exo.
A (extreme) example below 
 https://x.com/victortaelin/status/1901683454321791066?s=46&t=K4GIh0DK9OyTnjBcntN7qA

Best resource for running LLMs at home

https://www.reddit.com/r/LocalLLaMA

Dad, geek, conversation liquidity provider. vrypan.net | @l--o--l | $lemon3 | @wdimapp

i dabble with ai using open identity data from crypto https://alexpaden.tech

prev startups & racecars.

living @ network school

ᴡɪʀᴇᴅ ꜰᴏʀ ᴡᴏɴᴅᴇʀ, ᴛʀʏɪɴɢ ᴛᴏ ᴍᴀᴋᴇ ꜱᴇɴꜱᴇ ᴏꜰ ᴛʜᴇ ᴡᴏʀʟᴅ . ɪ ᴀꜱᴋ ʙᴇᴄᴀᴜꜱᴇ ɪ ᴄᴀʀᴇ. ɪ ᴄᴀꜱᴛ ᴛᴏ ꜰᴇᴇʟ ʟᴇꜱꜱ ᴀʟᴏɴᴇ.

Why do I feel like @alexpaden or @vrypan.eth can give an accurate answer to this

planting causal loops; harvesting reflexivity

You can daisy chain the Mac minis also I can’t remember the app for that @bleu.eth

I use a maxed out Mac Studio which is the best non gpu option iirc the cheapest ram too