---
title: Writing your first LLM app
description: Crafting Your First LLM Application with Agenta
---

In this tutorial, we'll lead you through the process of creating your first Language Learning Model (LLM) application using Agenta. Our aim is to build an application capable of producing a compelling startup pitch based on the startup's name and core idea. By the end of the tutorial, we'll have our app set up locally, ready for iterative refinement in the playground. You'll have the ability to modify parameters, branch out new variants, and systematically assess different versions in Agenta.

Let's begin.

## Installation

Run `pip install agenta` to install the Agenta CLI.

## 1. Project Initialization

Start by creating an empty project in an empty directory.

```bash
mkdir my-first-app; cd my-first-app
agenta init
```

Follow the prompts to initialize your project. Make sure to select `start with an empty project` when prompted.

Now create a virtual environment:

```bash
python3 -m venv env
source env/bin/activate
```

Finally let's create a requirements.txt and add within it our dependencies

```
langchain
agenta
python-dotenv
openai
```

Install the dependencies:

```bash
pip install -r requirements.txt
```

Now we're ready to start writing our app.

## 2. Write a Simple LLM App

Create a file called `app.py` - the main script that Agenta SDK expects. This simple script takes a startup name and idea, and generates a pitch using langchain:

```python

from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate

startup_name = "agenta"
startup_idea = "the open-source developer-first llmops platform"

default_prompt = """
    please write a short linkedin message (2 SENTENCES MAX) to an investor pitching the following startup:
    startup name: {startup_name}
    startup idea: {startup_idea}"""

#template is a float, it is used to decide what sampling temperature to use. Default to 0.7
temperature = 0.5

llm = OpenAI(temperature=temperature)
prompt = PromptTemplate(
    input_variables=["startup_name", "startup_idea"],
    template=prompt_template)

chain = LLMChain(llm=llm, prompt=prompt)
output = chain.run(startup_name=startup_name, startup_idea=startup_idea)
print(output)
```

## 3. Encapsulate Your Code in a Function

Wrap your code into a function named `generate`. This function will be the primary function Agenta calls, taking in strings as inputs and returning a string as an output:

```python
from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate


default_prompt = """
    please write a short linkedin message (2 SENTENCES MAX) to an investor pitching the following startup:
    startup name: {startup_name}
    startup idea: {startup_idea}"""


def generate(startup_name, startup_idea):
    #template is a float, it is used to decide what sampling temperature to use. Default to 0.7
    temperature = 0.5
    llm = OpenAI(temperature=temperature)
    prompt = PromptTemplate(
        input_variables=["startup_name", "startup_idea"],
        template=prompt_template)

    chain = LLMChain(llm=llm, prompt=prompt)
    output = chain.run(startup_name=startup_name, startup_idea=startup_idea)
    return output

```

## 4. Add the decorator

We'll now transform our plain Python function into an Agenta app. Use the Agenta decorator `@post` for this purpose. The decorator tells Agenta that this function is the main function in the app, and it specifies the inputs (and later parameters) that the app takes.

Note here that we made sure to add the type annotations to the function arguments. This is required for Agenta to be able to understand the inputs and outputs of the function.

```python

from agenta import post
from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
import os


default_prompt = """
    please write a short linkedin message (2 SENTENCES MAX) to an investor pitching the following startup:
    startup name: {startup_name}
    startup idea: {startup_idea}"""


@post
def generate(startup_name: str, startup_idea: str) -> str:
    #template is a float, it is used to decide what sampling temperature to use. Default to 0.7
    temperature = 0.5
    llm = OpenAI(temperature=temperature)
    prompt = PromptTemplate(
        input_variables=["startup_name", "startup_idea"],
        template=prompt_template)

    chain = LLMChain(llm=llm, prompt=prompt)
    output = chain.run(startup_name=startup_name, startup_idea=startup_idea)
    return output
```

## 5. Parameter Addition

To iterate and fine-tune this app in the playground, we need to expose parameters. Accomplish this by adding additional arguments in the app function:

```python
from agenta import post, TextParam, FloatParam
from agenta.types import MultipleChoiceParam
from dotenv import load_dotenv
from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate


default_prompt = """
    please write a short linkedin message (2 SENTENCES MAX) to an investor pitching the following startup:
    startup name: {startup_name}
    startup idea: {startup_idea}"""


@post
def generate(
    startup_name: str,
    startup_idea: str,
    prompt_template: TextParam = default_prompt,
    temperature: FloatParam = 0.5,
    model: MultipleChoiceParam = MultipleChoiceParam(
        choices = ["text-davinci-003", "gpt-3.5-turbo", "gpt-4"]
    )
) -> str:
    llm = OpenAI(temperature=temperature)
    prompt = PromptTemplate(
        input_variables=["startup_name", "startup_idea"],
        template=prompt_template)

    chain = LLMChain(llm=llm, prompt=prompt)
    output = chain.run(startup_name=startup_name, startup_idea=startup_idea)
    return output
```

In this example, `startup_name` and `startup_idea` are required inputs, while `prompt`, `temperature` and `model` are optional parameters with default values.
The types TextParam, FloatParam and MultipleChoiceParam are used to indicate these are parameters rather than inputs, and they will be made available for tuning in the Agenta UI.
The type MultipleChoiceParam will allow the user to select from multiple choices of models, by default the model to use is `text-davinci-003` since a default model to use wasn't specified.
To specify a default model, modify the parameter to this:

```python
model: MultipleChoiceParam = MultipleChoiceParam(
    "text-davinci-003",
    ["gpt-3.5-turbo", "gpt-4"]
)
```

This will result to having the app use the `text-davinci-003` model.

## 6. Text Generation Function - `call_llm`

The provided code snippet below defines a function `call_llm` that uses the chosen language model to generate text output based on the provided parameters.
The function takes inputs such as model, temperature, and prompt, along with additional keyword arguments (kwargs) required for text generation.

If the chosen model is `"text-davinci-003"`, it uses the OpenAI class from the LangChain library to run an LLMChain with the given parameters, producing text output based on the input prompt and the provided keyword arguments.

If the model is either `"gpt-3.5-turbo"` or `"gpt-4"`, it uses the ChatOpenAI class from the LangChain library to generate text output in a conversational format.
The function prepares a list of messages with a single HumanMessage containing the content from the input prompt formatted with the provided keyword arguments. The generated text output is then returned.

```python

def call_llm(model, temperature, prompt, **kwargs):
    if model == "text-davinci-003":
        llm = OpenAI(model=model, temperature=temperature)
        chain = LLMChain(llm=llm, prompt=prompt)
        output = chain.run(**kwargs)
    elif model in ["gpt-3.5-turbo", "gpt-4"]:
        chat = ChatOpenAI(model=model,
                          temperature=temperature)
        messages = [
            HumanMessage(content=prompt.format(**kwargs))
        ]
        output = chat(messages).content
    return output
```

This function facilitates the text generation process using different language models, providing the necessary flexibility for users to choose the appropriate model for their specific needs.

## 7. Integration of the `call_llm` function

The `call_llm` function is seamlessly integrated into the `generate` app to provide text generation capabilities based on user inputs and selected language models.

```python
# other imports here
from agenta.types import MultipleChoiceParam
from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage


def call_llm(model, temperature, prompt, **kwargs):
    if model == "text-davinci-003":
        llm = OpenAI(model=model, temperature=temperature)
        chain = LLMChain(llm=llm, prompt=prompt)
        output = chain.run(**kwargs)
    elif model in ["gpt-3.5-turbo", "gpt-4"]:
        chat = ChatOpenAI(model=model,
                          temperature=temperature)
        message = HumanMessage(content=prompt.format(**kwargs))
        output = chat(message).content
    return output


@post
def generate(
    startup_name: str,
    startup_idea: str,
    prompt_template: TextParam = default_prompt,
    temperature: FloatParam = 0.5,
    model: MultipleChoiceParam = MultipleChoiceParam(
        choices = ["text-davinci-003", "gpt-3.5-turbo", "gpt-4"]
    )
) -> str:
    prompt = PromptTemplate(
        input_variables=["startup_name", "startup_idea"],
        template=prompt_template)

    output = call_llm(
        model=model, temperature=temperature, prompt=prompt,
        startup_name=startup_name, startup_idea=startup_idea
    )
    return output
```

The `generate` app function accepts parameters such as `startup_name`, `startup_idea`, `prompt_template`, `temperature`, and `model`.
It initializes a `PromptTemplate` with `startup_name` and `startup_idea` as input variables, along with the provided `prompt_template`.
The `call_llm` function is then invoked with the specified `model`, `temperature`, `prompt`, and keyword arguments (`startup_name` and `startup_idea`).

The function utilizes the chosen language model to generate a LinkedIn message based on the `prompt_template` and input data (`startup_name` and `startup_idea`).
The generated LinkedIn message is returned as the output of the generate app.

**NOTE:** The `call_llm` function significantly enhances the capabilities of the generate app, providing a flexible and powerful mechanism for text generation with different language models.
Its seamless integration within the app allows users to easily fine-tune text generation based on their specific requirements and chosen models.

## 8. API Key Integration

To use openai or other APIs, you'll need appropriate keys. Create an empty `.env` file and add your keys to it. Then, read the keys from the environment in your code:

```bash
OPENAI_API_KEY=sk-xxxxxxx
```

We just need to read the keys from the environement and use them in our code.

```python
from agenta import post, TextParam, FloatParam
from dotenv import load_dotenv
from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
from agenta.types import MultipleChoiceParam
from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage


default_prompt = """
    please write a short linkedin message (2 SENTENCES MAX) to an investor pitching the following startup:
    startup name: {startup_name}
    startup idea: {startup_idea}"""


def call_llm(model, temperature, prompt, **kwargs):
    if model == "text-davinci-003":
        llm = OpenAI(model=model, temperature=temperature)
        chain = LLMChain(llm=llm, prompt=prompt)
        output = chain.run(**kwargs)
    elif model in ["gpt-3.5-turbo", "gpt-4"]:
        chat = ChatOpenAI(model=model,
                          temperature=temperature)
        message = HumanMessage(content=prompt.format(**kwargs))
        output = chat(message).content
    return output


@post
def generate(
    startup_name: str,
    startup_idea: str,
    prompt_template: TextParam = default_prompt,
    temperature: FloatParam = 0.5,
    model: MultipleChoiceParam = MultipleChoiceParam(
        choices = ["text-davinci-003", "gpt-3.5-turbo", "gpt-4"]
    )
) -> str:
    load_dotenv()
    prompt = PromptTemplate(
        input_variables=["startup_name", "startup_idea"],
        template=prompt_template)

    output = call_llm(
        model=model, temperature=temperature, prompt=prompt,
        startup_name=startup_name, startup_idea=startup_idea
    )
    return output
```

## 9. Run the code in CLI

The `@post` decorator allows the function to run in the Command Line Interface (CLI). Execute it using the command:

```bash
python app.py agenta "the open-source developer-first llmops platform"

Hi there, I'm excited to tell you about Agenta, the open source Dev First LLMOPS Platform. It's a revolutionary way to simplify and streamline the development process, and I'm confident it will be a game changer in the industry. Let's chat soon!
```

If no default value was provided in the model (MultiChoiceParam) parameter, the code would still work because the app is using the first model in the list of models provided.
If you would like to specify which model to use, kindly provide the name of the model and run the command again:

```bash
python app agenta "the open-source developer-first llmops platform" --model gpt-3.5-turbo

Hi [Investor's Name], I wanted to introduce you to Agenta, an open-source developer-first llmops platform. We believe it has the potential to revolutionize the way developers manage their llmops processes. Would love to discuss further if you're interested.
```

## 10. Deploying to Agenta

Our app is still local. To make it available in the playground, add it to Agenta by running:

```bash
agenta variant serve --file_name app.py
```

This commands build a docker image of your app and push it to the Agenta registry. It also creates a new variant for your app in the Agenta UI. You can now go to the Agenta UI and start playing with your app.

## 11. Iterative Improvement in the Playground

Your app is now accessible in the Agenta UI at [http://localhost](http://localhost). Interact with your app, experiment with different parameters, and fine-tune your app's behavior to achieve desired results.

## Conclusion

Congratulations! You've crafted and deployed your first LLM app with Agenta. You can now experiment and refine it further in the playground.
