From my testing, local “Distilled” models of DeepSeek-R1 don’t work with Artifacts (the bolt terminal and etc.) because they are not “Instruct” fine-tuned. You’ll want to try a different model. If you have a massive hardware constraints (or want to run an LLM on an RPI for example), you can try the smallest “Instruct” model (3B Parameters) I have seen work with bolt (and useable):
And for the size, the current best 7B Parameter LLM on the Hugging Face Open LLM Leader Board:
| Rank | Type | Model | Average | IFEval | BBH | MATH | GPQA | MURS | MMLU-Pro | CO2 Cost |
|---|---|---|---|---|---|---|---|---|---|---|
| #708 | prithivMLmods | QwQ-LCoT2-7B-Instruct | 28.57% | 55.61% | 34.37% | 22.21% | 6.38% | 15.75% | 37.13% | 1.37 kg |
You can install them for Ollama with:
ollama run hf.co/mradermacher/QwQ-LCoT-3B-Instruct-GGUF:Q4_K_M
ollama run hf.co/mradermacher/QwQ-LCoT2-7B-Instruct-GGUF:Q4_K_M
P.S. I do personally plan on creating a “Instruct” fine-tune set of the DeepSeek-R1 “Distilled” (1.5B - 32B) models to support Bolt.diy, but I’m still figuring out a few things and I will release them on both Hugging Face and Ollama. Maybe also write a how-to doc.
Lastly, I’d also go through your Bolt.diy settings to turn off all the extra stuff (like providers) that you aren’t going to use, and enable optimized prompts, etc.
Hope that helps!