Appreciate that there is now a settings tab. I use an env file for all my API keys so I don’t really understand the purpose of Settings->Providers. It would make more sense to me if it worked with the env file. What I mean exactly is:
if the env contains an API key for HuggingFace (for example) then it should be enabled automatically.
the providers tab could allow the user to enter the API key for a provider which would be saved to the env file and that provider automatically enabled.
allow the user to modify / update the provider details (API key and endpoint), saved to the env file
remove API key entry functionality on the chat interface
Without the settings it would always show all providers, even if there is no API key set within the .env file, but if you just show all that have one, you are not able to set an API key via the UI (temporary).
I think its not a big deal / you dont need it when you host on your local system and can change fast the .env file, but when you host on netlify etc. its not the fast to change something. Even when using the bolt-instance for multiple users, which should not use your API key for every provider.
Considering this, you also do not want a save to .env-file function
Introduction of the providers tab was a surprise. Previously (as a user of env file) all providers were visible but their corresponding models were not unless an API key was provided.
With the providers tab, it is possible for me to enable a provider that I don’t have a key for and for their models to be “selectable” in the chat ui. It is also possible for me to shut off a provider that I do have a key for and therefore prevent the provider from appearing as a selectable option in the UI.
So, yes. I don’t understand the purpose of the providers tab. I don’t understand the problem that this apparently solves. When it first appeared, all it did was confuse me because the default selection for providers was “disabled”.
The user still needs to enter the API key via the UI or via the env file. In the past, that “enabled” the provider and caused the model list to be populated for that provider. The provider tab merely provides a way to prevent providers from being displayed in the UI with no consideration for whether the user has provided a key for a provider or not.
I agree, but I like the settings and provider list, but all API keys should also be managed there and not in code. I’m not sure I like adding them in the model window drop down, because it has issues when you switch models (you have to change it each time).
Allow people to add to the list from the front end, put keys in there and etc. Add ability to add or remove from the list (with reset, export, etc). Then PR’s wouldn’t need to be made to add new ones, could cleanup the repo.
So it needs to be revamped further, but I think it’s a step in the right direction.
Apparently there are some potential security issues that are above my pay grade. I think these need to be worked through in slower time without losing sight of UX.
As a target for the future, I would like to see the providers tab become the single place where all users populate the keys and providers they want to use and leave the ability for users to select applicable providers and models on the chat UI.
Adding correction to @aliasfox, we discussed in dm and tested that you dont need to reset the key when you switch llm it automatically adapt to selected provider, and if you have set key for one provider and change the provider from dropdown, the api also gets updated to the latest provider selected
