Hello,
I installed Bolt.diy on my computer and configured it to use the qwen2.5 model running on my computer, but when I write a prompt, the response time is very slow. When I write the same prompt to qwen2.5 in the terminal, I get a much faster response.
Thanks for your help in advance.
Hi,
I dont think you compare the right things here. You probably just used the same user prompts, but bolt has its own system prompts in the back which are also sent with to do its “magic”.
I am also no expert on this but if you take a look e.g. in file “app/routes/api.enhancer.ts” you see some stuff:
Other then that bolt has also write all output in the webcontainer where the files are, what your terminal doesnt have to do.
thats correct. and this is for enhancing the prompt.
the actual system prompt for regular operation is much larger.
its located at app/lib/.server/llm/prompts.ts
it takes 4k context alone. so if you want to have a fair comparison use this prompt in the system prompt of ollama
then write your query to compare the performance
