So, I jumped in with both feet and in less than an hour ws well over 5 bucks in tokens on 3.7. I expected that. But, knowing myself, I wanted to test out some local models. SO, I tried a few through LMstudio and Ollama. I wasn’t aable to get anythign to build though. I primarily tried 14b models that I had (deepseek and Gwen 2.5)
Wondering what your experience has been?
I personally tried a lot of models, but found not a single one I would use to build stuff with it. Even if they manage to build code, its not good quality compared to the commercial/cloud ones. As there are also free cloud ones, I still dont see the usecase for using local ones.
But I think in the future / as soon as we got better system prompts for different models, there will be some, which perform better.
I have to agree whole heartedly. The problem with free ones are the limitations reaching quotas. like, halfway through an app, and easily a buck or 2 in tokens, and you have a half done program. BUT. like I said, I’m new. I haven’t fully got the hang of it yet. I’m coming from (still using) Websim which has come a LONG way in the last year. it’s nearly identical from the user’s perspectivel (not the GUI, I mean in no-code style prompt building) What would be KILLER is being able to download my websim hosted apps and retrofit them to be locally run apps (personal PC style)
my next question whold be… Which of the online 200k+CTX models works well enough and doesn’t cost 10 bucks to build and complete an app? lol I know. complexity. I’m a terrible judge of it. I DID publish my 1st github from it though. it’s rough. i t’s not good. but it worked… once. it’s a dual AI discussion app. it uses LM studio and OLLAMA to alternate back and forth.
That’s the idea. turns out it’'s MUCH more complex than I thought but I got a lot of it working… mostly
