Linux Install - Bolt trying to access wrong Model

General
1

I have been trying to setup bolt with Ollama and qwen2.5-coder:7b and I can run Ollama and get the prompts to work correctly but bolt is failing to connect to it.

It looks like its still trying to connect to claude-sonnet somehow - If I go to the serverip:11434 or curl it I get a message saying Ollama is running.

[+] Running 1/0
:heavy_check_mark: Container boltnew-any-llm-bolt-ai-dev-1 Created 0.0s
Attaching to boltnew-any-llm-bolt-ai-dev-1
boltnew-any-llm-bolt-ai-dev-1 |
boltnew-any-llm-bolt-ai-dev-1 | > bolt@ dev /app
boltnew-any-llm-bolt-ai-dev-1 | > remix vite:dev “–host” “0.0.0.0”
boltnew-any-llm-bolt-ai-dev-1 |
boltnew-any-llm-bolt-ai-dev-1 |
boltnew-any-llm-bolt-ai-dev-1 | ➜ Local: localhost:5173/
boltnew-any-llm-bolt-ai-dev-1 | ➜ Network: 172.18.0.2:5173/
boltnew-any-llm-bolt-ai-dev-1 | APICallError [AI_APICallError]: Not Found
boltnew-any-llm-bolt-ai-dev-1 | at /app/node_modules/.pnpm/@ai-sdk+provider-utils@1.0.20_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.js:505:14
boltnew-any-llm-bolt-ai-dev-1 | at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
boltnew-any-llm-bolt-ai-dev-1 | at async postToApi (/app/node_modules/.pnpm/@ai-sdk+provider-utils@1.0.20_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.js:398:28)
boltnew-any-llm-bolt-ai-dev-1 | at async OllamaChatLanguageModel.doStream (/app/node_modules/.pnpm/ollama-ai-provider@0.15.2_zod@3.23.8/node_modules/ollama-ai-provider/dist/index.js:485:50)
boltnew-any-llm-bolt-ai-dev-1 | at async fn (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3938:23)
boltnew-any-llm-bolt-ai-dev-1 | at async file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
boltnew-any-llm-bolt-ai-dev-1 | at async retryWithExponentialBackoff (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:86:12)
boltnew-any-llm-bolt-ai-dev-1 | at async startStep (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3903:13)
boltnew-any-llm-bolt-ai-dev-1 | at async fn (file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3977:11)
boltnew-any-llm-bolt-ai-dev-1 | at async file:///app/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
boltnew-any-llm-bolt-ai-dev-1 | at async chatAction (/app/app/routes/api.chat.ts:64:20)
boltnew-any-llm-bolt-ai-dev-1 | at async Object.callRouteAction (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/data.js:37:16)
boltnew-any-llm-bolt-ai-dev-1 | at async /app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4612:21
boltnew-any-llm-bolt-ai-dev-1 | at async callLoaderOrAction (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4677:16)
boltnew-any-llm-bolt-ai-dev-1 | at async Promise.all (index 1)
boltnew-any-llm-bolt-ai-dev-1 | at async callDataStrategyImpl (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4552:17)
boltnew-any-llm-bolt-ai-dev-1 | at async callDataStrategy (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:4041:19)
boltnew-any-llm-bolt-ai-dev-1 | at async submit (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3900:21)
boltnew-any-llm-bolt-ai-dev-1 | at async queryImpl (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3858:22)
boltnew-any-llm-bolt-ai-dev-1 | at async Object.queryRoute (/app/node_modules/.pnpm/@remix-run+router@1.17.0/node_modules/@remix-run/router/dist/router.cjs.js:3827:18)
boltnew-any-llm-bolt-ai-dev-1 | at async handleResourceRequest (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/server.js:413:20)
boltnew-any-llm-bolt-ai-dev-1 | at async requestHandler (/app/node_modules/.pnpm/@remix-run+server-runtime@2.10.0_typescript@5.5.2/node_modules/@remix-run/server-runtime/dist/server.js:156:18)
boltnew-any-llm-bolt-ai-dev-1 | at async /app/node_modules/.pnpm/@remix-run+dev@2.10.0@remix-run+react@2.10.2_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_qwyxqdhnwp3srgtibfrlais3ge/node_modules/@remix-run/dev/dist/vite/cloudflare-proxy-plugin.js:70:25 {
boltnew-any-llm-bolt-ai-dev-1 | cause: undefined,
boltnew-any-llm-bolt-ai-dev-1 | url: ’ 192.168.20.200:11434/api/chat’,
boltnew-any-llm-bolt-ai-dev-1 | requestBodyValues: {
boltnew-any-llm-bolt-ai-dev-1 | format: undefined,
boltnew-any-llm-bolt-ai-dev-1 | model: ‘claude-3-5-sonnet-latest’,
boltnew-any-llm-bolt-ai-dev-1 | options: { num_ctx: 32768, num_predict: 8000, temperature: 0 },
boltnew-any-llm-bolt-ai-dev-1 | messages: [
boltnew-any-llm-bolt-ai-dev-1 | [Object], [Object],
boltnew-any-llm-bolt-ai-dev-1 | [Object], [Object],
boltnew-any-llm-bolt-ai-dev-1 | [Object], [Object],
boltnew-any-llm-bolt-ai-dev-1 | [Object], [Object],
boltnew-any-llm-bolt-ai-dev-1 | [Object], [Object]
boltnew-any-llm-bolt-ai-dev-1 | ],
boltnew-any-llm-bolt-ai-dev-1 | tools: undefined
boltnew-any-llm-bolt-ai-dev-1 | },
boltnew-any-llm-bolt-ai-dev-1 | statusCode: 404,
boltnew-any-llm-bolt-ai-dev-1 | responseHeaders: {
boltnew-any-llm-bolt-ai-dev-1 | ‘content-length’: ‘78’,
boltnew-any-llm-bolt-ai-dev-1 | ‘content-type’: ‘application/json; charset=utf-8’,
boltnew-any-llm-bolt-ai-dev-1 | date: ‘Wed, 20 Nov 2024 19:41:40 GMT’
boltnew-any-llm-bolt-ai-dev-1 | },
boltnew-any-llm-bolt-ai-dev-1 | responseBody: ‘{“error”:“model \“claude-3-5-sonnet-latest\” not found, try pulling it first”}’,
boltnew-any-llm-bolt-ai-dev-1 | isRetryable: false,
boltnew-any-llm-bolt-ai-dev-1 | data: undefined,
boltnew-any-llm-bolt-ai-dev-1 | [Symbol(vercel.ai.error)]: true,
boltnew-any-llm-bolt-ai-dev-1 | [Symbol(vercel.ai.error.AI_APICallError)]: true
boltnew-any-llm-bolt-ai-dev-1 | }

Any ideas?
edit: Had to replace all HTTP with blank spaces for posting rules. But they are still there.

2

I have it running currently and am getting responses back from prompts but in the code preview section all I see is

"Failed to spawn bolt shell

Worker.postMessage: The WebAssembly.Memory object cannot be serialized. The Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policy HTTP headers can be used to enable this.

"

3

Could you try selecting a model after choosing Ollama instead of using the first one it gives? I think I might know what the issue is but I’m curious what happens with that.

4

Hey Cole -

I dont get anything in the Ollama dropdown. I can run any of the ollama models directly from the command prompt without issue. I’ve got several version of the models installed on the server and they appear just fine when I run ‘ollama list’.

NAME ID SIZE MODIFIED
qwen2.5-mod:7b da53dcc7862f 4.7 GB 17 minutes ago
qwen2.5:7b da53dcc7862f 4.7 GB 17 minutes ago
qwen2.5-coder:7b 2b0496514337 4.7 GB 48 minutes ago
qwen2.5-coder-extra-ctx:7b c002283697a0 4.7 GB About an hour ago
qwen2.5-extra-ctx:7b da53dcc7862f 4.7 GB 2 hours ago

5

image
image1851×1004 30.1 KB

6

Are you using Chrome Canary as your browser? The wasm error in the terminal related to COOP/COEP is a flag for me. That could be throwing a hard error that prevents the model loader from ever populating.

If not, definitely something to be looked into.

7

I wasnt but I just tried to run it in Canary and am getting the same error, the model loader populates correctly for models that aren’t Ollama. This is running on a local server, but we are accessing it via the local network.

8

image
image
image