Here are a few things I ran into trying to get this to work with Ollama:
is_ollama = “localhost” in base_url.lower() really screws things up when accessing Ollama outside Docker requires the base URL to be: http://host.docker.internal:11434/v1. Instead of looking for localhost, maybe look for “11434” or just allow us to choose our provider from a dropdown.
For me, using Ollama locally is great for saving on costs while sacrificing speed. I don’t have a blazing-fast PC, so when trying to add the Pydantic AI Documentation to Supabase, I left it run for an hour and it was just a mess. I think this task should be separate and we should be able to select a different model and provider. Why are we forced to use the same base for everything?
Revamping the environment variable set up is the next thing on my list. I’m going to make it so you can select your provider first and then provide your base URL which will solve #1. And then I also want to separate the embedding model from the LLMs like you saying for #2!
If you’re going to do that, can I make an additional suggestion: when you pick a provider and enter URLs and models for that provider, save those URLs and models per provider so we can quickly switch back and forth between providers without having to keep changing the URLs. This would allow us to switch providers on the fly depending on our needs.