Minimum requirements of running local LLM's

bolt.diy [bolt.diy] Issues and Troubleshooting
1

Hi everyone!
I am really loving how oTToDev works and this community, so first of all thanks!

It works fine for me when using API from Claude Sonnet 3.5 for example, but for me the advantage is using local LLMs to avoid pricy payments. However, my MAC with 16 GB RAM is not strong enough to run the bigger models, and when I use the smaller ones it hallucinates and the preview is working on/off. Therefore I want to invest in a MAC strong enough to run for example qwen 2.5-coder 32b to really take oTToDev locally.
Does anyone have some guidelines on the minimum requirements to get oTToDev running well with bigger local LLMs?

All the best,
Claus

1 Like
2

I dropped your question into Poe and here are what it suggests.

I would have used my OttoDev :wink: install to find you an answer however I am still trying to get my install up and running. My issue is something to do with it finding my Ollama. I’ll post for some help as soon as I get back to my pc.

Hope it helps.

3

Thanks a lot, @Doug! Just what I needed :slight_smile:

4

The thing is Ollama doesn’t run what Mac is better at: MLX. Only Lm Studio as far as I know and investigating…