I’m having the same issue on my Mac. I’m running Ollama directly on the host so that I can make use of the GPU. When running oTToDev in the container I needed to set
OLLAMA_API_BASE_URL=http://host.docker.internal:11434
If I open a terminal in the oTToDev container I can:
first time running a ps command I get:
→ curl {OLLAMA_API_BASE_URL}/api/ps
{“models”:}
but then running:
→ curl {OLLAMA_API_BASE_URL}/api/chat -d '{
- “model”: “dolphin_2_9_2_lc:latest”*
}’
{“model”:“dolphin_2_9_2_lc:latest”,“created_at”:“2024-11-19T21:49:37.519538Z”,“message”:{“role”:“assistant”,“content”:“”},“done_reason”:“load”,“done”:true}
Now ps works
→ curl {OLLAMA_API_BASE_URL}/api/ps
{“models”:[{“name”:“dolphin_2_9_2_lc:latest”,“model”:“dolphin_2_9_2_lc:latest”,“size”:102690629632,“digest”:“f586f65be437e2fca804b550a90715e227ec9106d6d67cffc3bd9a7553f7a782”,“details”:{“parent_model”:“”,“format”:“gguf”,“family”:“qwen2”,“families”:[“qwen2”],“parameter_size”:“72.7B”,“quantization_level”:“Q4_0”},“expires_at”:“2024-11-19T13:54:37.519822-08:00”,“size_vram”:102690629632}]}
But refreshing the web page still won’t find any models
eventually the model unloads and I get back to:
→ curl {OLLAMA_API_BASE_URL}/api/ps
{“models”:}