I have had some good success using Qwen2.5-coder:32b with oTToDev. However it consistently hallucinates the version number of a package, eg; concurrently@^7.6.5 I remind oTToDev that that version does not exist and suggest a real version, which it is happy to substitute, until my next feature request, and it re-hallucinates the old version again.
I imagine the large commercial LLMs don’t suffer from this (as much), but I’m wondering, and wanted to start a discussion about, if such boilerplate stuff could be maintained in RAG or some such. Human programmers don’t try to remember a bunch of version numbers off the top of their heads, they’ll refer to a template or coding standards doc (or copy a previous project).
Thoughts?
1 Like
I believe that package management should be implemented using a tool: whenever a package interaction is requested, the LLM should revert to getting this information via an API e. G. from npmjs.org
2 Likes
I can’t parse that. What is an “all e. G.” ?
It should have been „api“, autocorrect-issue
Ah, of course.
I agree the data should be sourced from there, but perhaps not for every time the system builds a new project.
There could be an Agent that periodically updates a package list (or lists) of compatible packages. Note: this could be broader than just npm, it might know about resources such as Font Awesome, etc.
1 Like
yes i have seen this issue… this can be solved once project template is added.
also you can at the starting prompt tell the llm to only use npm commands to update the dependencies and not up manually update the package.json
see if that makes any difference
1 Like