I can get google to work, but when I hit ollama running through systemd on the local ubuntu system, I can get a model list but the inference never happens for some reason.
Under google I can get it to make simple things, but I get this message.
Note that in this WebContainer environment, pip is not available, and we’re limited to Python’s standard library.
I put this as the ollama base url in settings: http://nomnom:11434
note: nomnom.draeician.com is the hostname of the machine running the ollama service and is in dns
I have two issues:
- the docker container not having pip or the ability to use it from the docker installation instructions
- Ollama listing models, but not running inference
When sending a prompt to ollama I get this on the docker console.
app-dev-1 | INFO LLMManager Found 62 cached models for Ollama
app-dev-1 | INFO stream-text Sending llm call to Ollama with model deepseek-coder-v2:latest
app-dev-1 | ERROR api.chat TypeError: Cannot read properties of undefined (reading ‘replace’)
app-dev-1 | at OllamaProvider.getModelInstance (/app/app/lib/modules/llm/providers/ollama.ts:59:34)
app-dev-1 | at Module.streamText (/app/app/lib/.server/llm/stream-text.ts:156:21)
app-dev-1 | at processTicksAndRejections (node:internal/process/task_queues:95:5)
app-dev-1 | at chatAction (/app/app/routes/api.chat.ts:116:20)
app-dev-1 | at Object.callRouteAction (/app/node_modules/.pnpm/@remix-run+server-runtime@2.15.0_typescript@5.7.2/node_modules/@remix-run/server-runtime/dist/data.js:36:16)
app-dev-1 | at /app/node_modules/.pnpm/@remix-run+router@1.21.0/node_modules/@remix-run/router/router.ts:4899:19
app-dev-1 | at callLoaderOrAction (/app/node_modules/.pnpm/@remix-run+router@1.21.0/node_modules/@remix-run/router/router.ts:4963:16)
app-dev-1 | at async Promise.all (index 0)
app-dev-1 | at defaultDataStrategy (/app/node_modules/.pnpm/@remix-run+router@1.21.0/node_modules/@remix-run/router/router.ts:4772:17)
app-dev-1 | at callDataStrategyImpl (/app/node_modules/.pnpm/@remix-run+router@1.21.0/node_modules/@remix-run/router/router.ts:4835:17)
app-dev-1 | at callDataStrategy (/app/node_modules/.pnpm/@remix-run+router@1.21.0/node_modules/@remix-run/router/router.ts:3992:19)
app-dev-1 | at submit (/app/node_modules/.pnpm/@remix-run+router@1.21.0/node_modules/@remix-run/router/router.ts:3755:21)
app-dev-1 | at queryImpl (/app/node_modules/.pnpm/@remix-run+router@1.21.0/node_modules/@remix-run/router/router.ts:3684:22)
app-dev-1 | at Object.queryRoute (/app/node_modules/.pnpm/@remix-run+router@1.21.0/node_modules/@remix-run/router/router.ts:3629:18)
app-dev-1 | at handleResourceRequest (/app/node_modules/.pnpm/@remix-run+server-runtime@2.15.0_typescript@5.7.2/node_modules/@remix-run/server-runtime/dist/server.js:402:20)
app-dev-1 | at requestHandler (/app/node_modules/.pnpm/@remix-run+server-runtime@2.15.0_typescript@5.7.2/node_modules/@remix-run/server-runtime/dist/server.js:156:18)
app-dev-1 | at /app/node_modules/.pnpm/@remix-run+dev@2.15.0_@remix-run+react@2.15.0_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_3djlhh3t6jbfog2cydlrvgreoy/node_modules/@remix-run/dev/dist/vite/cloudflare-proxy-plugin.js:70:25