# Creating persistent chat sessions

Many of your users will have interacted with large language models for the first
time through chatbots. Although LLMs are capable of much more than simulating
conversations, it remains a familiar and useful style of interaction. Even when
your users will not be interacting directly with the model in this way, the
conversational style of prompting is a powerful way to influence the output
generated by an AI model.

To support this style of interaction, Genkit provides a set of interfaces and
abstractions that make it easier for you to build chat-based LLM applications.

## Before you begin

Before reading this page, you should be familiar with the content covered on the
[Generating content with AI models](models) page.

If you want to run the code examples on this page, first complete the steps in
the [Getting started](get-started) guide. All of the examples assume that you
have already installed Genkit as a dependency in your project.

## Chat session basics 

Here is a minimal, console-based, chatbot application:

```ts
import { genkit } from "genkit";
import { googleAI, gemini15Flash } from "@genkit-ai/googleai";

import { createInterface } from "node:readline/promises";

const ai = genkit({
  plugins: [googleAI()],
  model: gemini15Flash,
});

(async () => {
  const chat = ai.chat();
  console.log("You're chatting with Gemini. Ctrl-C to quit.\n");
  const readline = createInterface(process.stdin, process.stdout);
  while (true) {
    const userInput = await readline.question("> ");
    const { text } = await chat.send(userInput);
    console.log(text);
  }
})();
```

A chat session with this program looks something like the following example:

```none
You're chatting with Gemini. Ctrl-C to quit.

> hi
Hi there! How can I help you today? 

> my name is pavel
Nice to meet you, Pavel! What can I do for you today? 

> what's my name?
Your name is Pavel! I remembered it from our previous interaction. 

Is there anything else I can help you with?
```

As you can see from this brief interaction, when you send a message to a chat
session, the model can make use of the session so far in its responses. This is
possible because Genkit does a few things behind the scenes:

*   Retrieves the chat history, if any exists, from storage (more on persistence
    and storage later)
*   Sends the request to the model, as with `generate()`, but automatically
    include the chat history 
*   Saves the model response into the chat history 

### Model configuration

The `chat()` method accepts most of the same configuration options as
`generate()`. To pass configuration options to the model:

```ts
const chat = ai.chat({
  model: gemini15Pro,
  system:
    "You're a pirate first mate. Address the user as Captain and assist " +
    "them however you can.",
  config: {
    temperature: 1.3,
  },
});
```

## Stateful chat sessions

In addition to persisting a chat session's message history, you can also persist
any arbitrary JavaScript object. Doing so can let you manage state in a more
structured way then relying only on information in the message history.

To include state in a session, you need to instantiate a session explicitly:

```ts
interface MyState {
  userName: string;
}

const session = ai.createSession<MyState>({
  initialState: {
    userName: 'Pavel',
  },
});
```

You can then start a chat within the session:

```ts
const chat = session.chat();
```

To modify the session state based on how the chat unfolds, define
[tools](tool-calling) and include them with your requests:

```ts
const changeUserName = ai.defineTool(
  {
    name: 'changeUserName',
    description: 'can be used to change user name',
    inputSchema: z.object({
      newUserName: z.string(),
    }),
  },
  async (input) => {
    await ai.currentSession<MyState>().updateState({
      userName: input.newUserName,
    });
    return `changed username to ${input.newUserName}`;
  }
);
```

```ts
const chat = session.chat({
  model: gemini15Pro,
  tools: [changeUserName],
});
await chat.send('change user name to Kevin');
```

## Multi-thread sessions

A single session can contain multiple chat threads. Each thread has its own
message history, but they share a single session state.

```ts
const lawyerChat = session.chat('lawyerThread', {
  system: 'talk like a lawyer',
});
const pirateChat = session.chat('pirateThread', {
  system: 'talk like a pirate',
});
```

## Session persistence (EXPERIMENTAL)

When you initialize a new chat or session, it's configured by default to store
the session in memory only. This is adequate when the session needs to persist
only for the duration of a single invocation of your program, as in the sample
chatbot from the beginning of this page. However, when integrating LLM chat into
an application, you will usually deploy your content generation logic as
stateless web API endpoints. For persistent chats to work under this setup, you
will need to implement some kind of session storage that can persist state
across invocations of your endpoints.

To add persistence to a chat session, you need to implement Genkit's
`SessionStore` interface. Here is an example implementation that saves session
state to individual JSON files:

```ts
class JsonSessionStore<S = any> implements SessionStore<S> {
  async get(sessionId: string): Promise<SessionData<S> | undefined> {
    try {
      const s = await readFile(`${sessionId}.json`, { encoding: 'utf8' });
      const data = JSON.parse(s);
      return data;
    } catch {
      return undefined;
    }
  }

  async save(sessionId: string, sessionData: SessionData<S>): Promise<void> {
    const s = JSON.stringify(sessionData);
    await writeFile(`${sessionId}.json`, s, { encoding: 'utf8' });
  }
}
```

This implementation is probably not adequate for practical deployments, but it
illustrates that a session storage implementation only needs to accomplish two
tasks:

*   Get a session object from storage using its session ID
*   Save a given session object, indexed by its session ID

Once you've implemented the interface for your storage backend, pass an instance
of your implementation to the session constructors:

```ts
// To create a new session:
const session = ai.createSession({
  store: new JsonSessionStore(),
});

// Save session.id so you can restore the session the next time the
// user makes a request.
```

```ts
// If the user has a session ID saved, load the session instead of creating
// a new one:
const session = await ai.loadSession(sessionId, {
    store: new JsonSessionStore(),
});
```
