# Calling LLMs

Mastra has direct LLM support through the `LLM` class. Note that this class is pretty barebones: you can specify a model and a user prompt, but you do not have the ability to use system prompts, tools, or workflows.

## Basic Setup

```ts showLineNumbers filename="index.ts" copy
import { Mastra, type ModelConfig } from "@mastra/core";

async function main() {
  // Configure your model

  const mastra = new Mastra({});

  const modelConfig: ModelConfig = {
    provider: "OPEN_AI",
    name: "gpt-4o",
    toolChoice: "auto",
  };

  const llm = mastra.llm;

  const response = await llm.text({
    messages: [
      {
        role: "user",
        content: "What is machine learning?",
      },
    ],
    model: modelConfig,
  });
  console.log(response.text);
}

main();
```

The code for this example can be found [here](https://github.com/mastra-ai/mastra/tree/main/examples/llm).

## Available Models

You can choose from several providers and models:

- **OpenAI models**: 'gpt-4', 'gpt-4-turbo', 'gpt-3.5-turbo', 'gpt-4o', 'gpt-4o-mini'

- **Anthropic models**: 'claude-3-5-sonnet-20241022', 'claude-3-5-sonnet-20240620', 'claude-3-5-haiku-20241022', 'claude-3-opus-20240229', 'claude-3-sonnet-20240229', 'claude-3-haiku-20240307'

- **Google Gemini models**: 'gemini-1.5-pro-latest', 'gemini-1.5-pro', 'gemini-1.5-flash-latest', 'gemini-1.5-flash'

- **Groq models**: 'llama3-groq-70b-8192-tool-use-preview', 'llama3-groq-8b-8192-tool-use-preview', 'gemma2-9b-it', 'gemma-7b-it'

- **Perplexity models**: 'llama-3.1-sonar-small-128k-online', 'llama-3.1-sonar-large-128k-online', 'llama-3.1-sonar-huge-128k-online', 'llama-3.1-sonar-small-128k-chat'

- **TogetherAI models**: 'codellama/CodeLlama-34b-Instruct-hf', 'upstage/SOLAR-10.7B-Instruct-v1.0', 'mistralai/Mixtral-8x7B-v0.1', 'WhereIsAI/UAE-Large-V1'

- **LM Studio models**: 'qwen2-7b-instruct-4bit', 'qwen2-math-1.5b', 'qwen2-0.5b', 'aya-23-8b', 'mistral-7b-v0.3'

- **Baseten models**: 'llama-3.1-70b-instruct', 'qwen2.5-7b-math-instruct', 'qwen2.5-14b-instruct', 'qwen2.5-32b-coder-instruct'

- **Fireworks models**: 'llama-3.1-405b-instruct', 'llama-3.1-70b-instruct', 'llama-3.1-8b-instruct', 'llama-3.2-3b-instruct'

- **Mistral models**: 'pixtral-large-latest', 'mistral-large-latest', 'mistral-small-latest', 'ministral-3b-latest'

- **X Grok models**: 'grok-beta', 'grok-vision-beta'

- **Cohere models**: 'command-r-plus'

- **Azure models**: 'gpt-35-turbo-instruct'

- **Amazon models**: 'amazon-titan-tg1-large', 'amazon-titan-text-express-v1', 'anthropic-claude-3-5-sonnet-20241022-v2:0'

- **Anthropic Vertex models**: 'claude-3-5-sonnet@20240620', 'claude-3-opus@20240229', 'claude-3-sonnet@20240229', 'claude-3-haiku@20240307'

## Usage Examples

### Text Generation

```typescript copy
const response = await llm.text({
  messages: [
    {
      role: "user",
      content: "What is machine learning?",
    },
  ],
  model: modelConfig,
});

console.log(response.text);
```

### Streaming Responses

```typescript copy
const stream = await llm.stream({
  messages: [
    {
      role: "system",
      content: "You are a helpful assistant",
    },
    {
      role: "user",
      content: "Explain quantum computing",
    },
  ],
  model: modelConfig,
  onStepFinish: (step) => {
    console.log("Step completed:", step);
  },
  maxSteps: 3,
});
```

## Environment Setup

Make sure to set your API keys. If you don't have an API key for an LLM provider, you can get one from the following services:

- [OpenAI](https://platform.openai.com/). Env variable: `OPENAI_API_KEY`
- [Anthropic](https://console.anthropic.com/settings/keys). Env variable: `ANTHROPIC_API_KEY`
- [Google Gemini](https://ai.google.dev/gemini-api/docs). Env variable: `GOOGLE_GENERATIVE_AI_API_KEY`
- [Groq](https://console.groq.com/keys). Env variable: `GROQ_API_KEY`

If you don't have an account with these providers, you can sign up and get an API key. OpenAI and Anthropic require a credit card to get an API key. Gemini does not and has a generous free tier for its API.
