Skip to main content
`Ctrl`+`K`

# DeepInfra

`DeepInfra` provides several LLMs.

This notebook goes over how to use Langchain with DeepInfra.

## Imports

    import os
    from langchain.llms import DeepInfra
    from langchain import PromptTemplate, LLMChain

## Set the Environment API Key

Make sure to get your API key from DeepInfra. You have to Login and get a new token.

You are given a 1 hour free of serverless GPU compute to test different models. (see here) You can print your token with `deepctl auth token`

    # get a new token: https://deepinfra.com/login?from=%2Fdash

    from getpass import getpass

    DEEPINFRA_API_TOKEN = getpass()

    os.environ["DEEPINFRA_API_TOKEN"] = DEEPINFRA_API_TOKEN

## Create the DeepInfra instance

Make sure to deploy your model first via `deepctl deploy create -m google/flat-t5-xl` (see here)

    llm = DeepInfra(model_id="DEPLOYED MODEL ID")

## Create a Prompt Template

We will create a prompt template for Question and Answer.

    template = """Question: {question}

    Answer: Let's think step by step."""

    prompt = PromptTemplate(template=template, input_variables=["question"])

## Initiate the LLMChain

    llm_chain = LLMChain(prompt=prompt, llm=llm)

## Run the LLMChain

Provide a question and run the LLMChain.

    question = "What NFL team won the Super Bowl in 2015?"

    llm_chain.run(question)