Bolt.new <> oTToDev Comparison

bolt.diy [bolt.diy] Issues and Troubleshooting
1

Hi guys, I apologize in advance for this probably unfair comparison, but I just want to better understand the expectations for output from some of the models.

Here is a short video of a comparison between bolt.new commercial and oTToDev fork using Qwen2.5 Coder 32B Instruct - API, Providers, Stats | OpenRouter cloud API:

Please do let me know how to make a better comparison. What model is bolt.new using?

2

Bolt.new uses Claude 3.5 Sonnet. I read about it on their website, possibly on the pricing page a while ago. You probably won’t get the same results, but let us know with another video what you get. The current version of Bolt.new (local) still has many features to be implemented and seems broken, but that’s part of the development process and will improve over time.

Using other models might not even open the preview window due to issues like hallucination and other factors. A new pull request with agent support is being tested and should streamline how instructions are handled, though its success depends on the model’s quality. Just a heads up, Claude 3.5 Sonnet is very expensive.

1 Like
3

Thanks for the feedback, @Champ2050! I fully understand that this is still a lot of work in development. I’m just trying to get a bearing on what to expect at the moment :slight_smile:

For some reason I can’t use OpenRouter API key for Claude? There is no error message in CMD but in the browser I get:

chunk-TLFX46WQ.js?v=728f4d57:7399 Warning: Encountered two children with the same key, `anthropic/claude-3.5-sonnet`. Keys should be unique so that components maintain their identity across updates. Non-unique keys may cause children to be duplicated and/or omitted — the behavior is unsupported and could change in a future version.

I only added one API key to the env file for OpenRouter. I did try creating an account at Anthropic but for whatever reason they declined my card and so I can’t be bothered lol.

1 Like
4

I’m facing similar issues with Anthropic, especially with cards. I also use OpenRouter from time to time. While I’m not sure how to fix this, there are a few steps you can try to troubleshoot.

  • Verify the Basics: Make sure all the basic settings are entered correctly.
  • Check the .env.local File: Ensure the API key is included in this file. I assume it’s there, as your video shows a successful run with the Qwen 32B instruct model.
  • Double-Check Browser Settings: In your browser, both dropdowns should be set correctly:
    • The first dropdown should be set to OpenRouter.
    • The second dropdown should be set to the Sonnet 3.5 model.
  • Check OpenRouter Connection: Ensure that OpenRouter is properly connecting to the 3.5 Sonnet model.
  • Troubleshooting with Other Models: Try using other models through OpenRouter. If only the 3.5 Sonnet model isn’t working, it could be that OpenRouter is undergoing maintenance.
5

Yeah it currently works with the Qwen 32B instruct and a few other models, but none of the Anthropic models. The error I shared it actually repeats in the browser dev tools for every model provider…

I had two keys created in OpenRouter and thought that could be why, but still same problem after deleting the second one I created.

6

Many models show errors on Openrouter, and I’ve seen it myself. Try the Haiku model, it worked for me yesterday on Openrouter.

7

I did manage to get the anthropic/claude-3-haiku (OpenRouter) to respond and build something but was never able to show the app in the preview.

8

Same here, we’re in the same situation, but let’s hope things improve soon with the preview issue through the pending agentic PR.

9

the main issue is the qwen no matter how good at coding it is it cant compare to Sonnet , because Sonnet is not only good at coding but other factors 2, like logic, and what type of design etc!

u can try this with qwen coder and give it like 4 to 5 steps make it clear what you want, that will help you get better results!

2 Likes
10

There are known reliability issues in prompts/AI outputs.

Some that come to mind are:

  1. it outputs code in wrong syntax so oTToDev fails to pick it up to send to preview
  2. When chat is long there could be other degradations too
  3. It asks to run commands in terminal that fail and it never knows

All of that gets better or worse depending on the model

1 Like