Adding new llms from together.ai or deepinfra

bolt.diy
29

I got it to work without all of that, just called the API Endpoint directly and modified a few lines of code to add a “GitHub” provider to Bolt.diy:

I didn’t realize it was so tedious to add new providers, but it wasn’t hard per se:

Add to .\bolt.diy\app\utils\constants.ts

  {
    name: 'GitHub',
    staticModels: [
      { name: 'gpt-4o', label: 'OpenAI 4o', provider: 'GitHub', maxTokenAllowed: 8192 },
    ],
    getApiKeyLink: 'https://github.com/settings/tokens',
  },

Add to .\bolt.diy\app\lib\.server\llm\api-key.ts

  // Fall back to environment variables
  switch (provider) {
    ...
    case 'GitHub':
      return env.GITHUB_API_KEY || cloudflareEnv.GITHUB_API_KEY ;

Add to .\bolt.diy\app\lib\.server\llm\model.ts

export function getGitHubModel(apiKey: OptionalApiKey, model: string) {
  const openai = createOpenAI({
    baseURL: 'https://models.inference.ai.azure.com',
    apiKey,
  });

  return openai(model);
}

And…

  switch (provider) {
    ...
    case 'GitHub':
      return getGitHubModel(apiKey, model);

Added GITHUB_API_KEY to .env.local

And a one-shot prompt worked!

image
image1920×1080 177 KB

I might tidy this up and submit a PR.

Thanks!

P.S. It would be nice if adding providers were handled more dynamically, instead of having to make a static entry for each. I mean it’s very redundant and unnecessary (and I personally don’t like writing code that way). But what can you do? And it be nice if the baseURL was just in the provider list, by provider along with the “getApiKeyLink” (could be called completely dynamically, reduce code, and improve maintaining/adding providers).

Edit: I provided the relative path to each file to edit, and confirmed that either Fine-Grained or Classic GitHub API tokens work fine.