Together.ai offering 100$ free credits and deepseek R1

Hi! Via a video of Matthew Berman on youtube I was made aware of together.ai. Similar to openrouter.ai they wrap models. Because I used up the 10$ credits of openrouter and wanted to play with R1 I registered. Simply to try it…

To my surprise I got greeted with 100$ free credits and an api key.

So I extended my ai_provider_helper to support PureOpenAi, OpenRouter, TogetherAi

from langchain_openai import ChatOpenAI
from os import getenv
from enum import Enum
#from langchain_together import ChatTogether

class AiWrapperProvider(Enum):
    PureOpenAi = 10
    OpenRouter = 20
    TogetherAi = 30

def get_ai_provider(model: str = "openai/gpt-4o", provider: AiWrapperProvider = AiWrapperProvider.OpenRouter) -> ChatOpenAI:
    """
    Returns an instance of ChatOpenAI based on the specified provider.

    :param model: The model to use (default: "openai/gpt-4o").
    :param provider: The AI wrapper provider to use (default: OpenRouter).
    :return: ChatOpenAI instance.
    """
    if provider == AiWrapperProvider.PureOpenAi:
        model = model.replace("openai/", "")
        return ChatOpenAI(model=model)

    elif provider == AiWrapperProvider.OpenRouter:
        return ChatOpenAI(model=model,
                          openai_api_key=getenv("OPENROUTER_API_KEY"),
                          openai_api_base=getenv("OPENROUTER_BASE_URL"))

    elif provider == AiWrapperProvider.TogetherAi:
        return ChatOpenAI(model=model,
                    openai_api_key=getenv("TOGETHER_API_KEY"),
                    openai_api_base=getenv("TOGETHER_BASE_URL"))
    else:
        raise ValueError("Invalid AI wrapper provider specified.")

In your .env file you need:
TOGETHER_API_KEY=PuT-It-HeRe
TOGETHER_BASE_URL=https://api.together.xyz/v1

You might also need to pip install langchain-together package

This is the llm initializer for openrouter changes to:
llm = get_ai_provider(model="openai/gpt-4o", provider=AiWrapperProvider.OpenRouter)

And this is deepseek with TogetherAi:
llm = get_ai_provider(model="deepseek-ai/DeepSeek-R1", provider=AiWrapperProvider.TogetherAi)

I eventually ran into the error:

deepseek-ai/DeepSeek-R1 is not supported for JSON mode/function calling

… which I need for my project so I didn’t deep dive but thought maybe interesting to some. They do offer a json mode of some sort but I think it is now called structured output so I think they might be outdated but not sure.

Sidenote: Here are the docs on how to integrate them via langchain package addon Integrations but it behaved the same with my implementation above and that one is closer to OpenRouter.

4 Likes

Deepseek R1 isnt one of the allowed llms as per the hackathon though. Can someone from the moderator team confirm?

1 Like

This is so cool, thanks for sharing!

@udd Good question - R1 is allowed! Initially the rules said to stick with cheaper LLMs so I ruled out reasoning LLMs like o1, but that’s only because the cheaper reasoning LLMs like R1 weren’t out when I wrote the rules! Oh how fast AI moves haha. R1 is certainly cheep enough to use!

The main thing though is R1 takes longer since it “thinks”, so you just have to think about the user experience on the Live Agent Studio and make sure the workflow doesn’t take too long to return a response to the user. Anything more than a minute is too long!

1 Like

Sound great. How you got the 100$ credit i just create a account in https://www.together.ai/ and only got 1$
maybe for your responses in the start form i put this:

deepseek R1 doesnt support function calling
here is the issue discussed with workarounds,

regular deepseek-chat does however

2 Likes

I signed up with his code on but only got $1.00 credit. Must be over already?

I think the decimal . is so small that someone misread ?

anyhow as for deepseek , continuously timing out today … guess that happens if you crash stockmarkets and are allover the news worldwide

1 Like

Try kluster.ai, they are offering $100 worth of Api credits and they are hosting R1 model.