Model Confusion

Archon
1

Finally got around to lighting off Archon today. Nice work Cole!

I started off using docker. As usual, I found an excuse to dump that quickly. In this case, it was probably 50% my fault.

Basically, the reason was the environment setup and by that I mean the use of LLM’s, Reasoner_Model, Primary_Model etc.

I initially set up:
BASE_URL and LLM_API_KEY for Anthropic.
OPENAI_API_KEY
REASONER_MODEL as o3-mini
PRIMARY_MODEL as claude
EMBEDDING_MODEL as text-embedding3-small

So what was most confusing was which models were used for what. For example, which model is used when crawling pydantic docs? (was why I booted docker, so I could debug in python).

It would be useful to include a description of what roles parameters / models are used for. For instance, does BASE_URL, LLM_API_KEY and PRIMARY_MODEL all tie together? Is that what is used in crawling of docs AND for coding? Is that model also used in any agents that are created?

Meanwhile, does anyone have any suggestions for what works best in what role?

2 Likes
2

I appreciate the feedback! It’s actually up next on my todo list to make the environment variable setup way more comprehensive. The PRIMARY MODEL is the one that is used to crawl the Pydantic AI docs btw. And I do want to call out in the help bubbles for each environment variable where they are used too.

I like using OpenRouter so I have access to both o3-mini (for my reasoner) and Claude 3.7 Sonnet for my primary model.

2 Likes
3

what model you use for embedding
and what is the base url of ollama embedding

4

Since you’re asking about ollama, the embeddings model you can use with ollama can be found here: Embedding models · Ollama Search

The endpoint is http://localhost:11434/api/embed

Most ollama users would use the nomic-embed-text model but my experience with that has been very, very slow on lower spec GPU/CPU.