---
sidebar_position: 11
---

# Jlama
[Jlama Project](https://github.com/tjake/Jlama)

## Project setup

To install langchain4j to your project, add the following dependency:

For Maven project `pom.xml`

```xml

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j</artifactId>
    <version>0.36.2</version>
</dependency>

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-jlama</artifactId>
    <version>0.36.2</version>
</dependency>

<dependency>
    <groupId>com.github.tjake</groupId>
    <artifactId>jlama-native</artifactId>
    <!-- for faster inference. supports linux-x86_64, macos-x86_64/aarch_64, windows-x86_64 
       Use https://github.com/trustin/os-maven-plugin to detect os and arch -->
    <classifier>${os.detected.name}-${os.detected.arch}</classifier>
    <version>${jlama.version}</version> <!-- Version from langchain4j-jlama pom -->
</dependency>

```

For Gradle project `build.gradle`

```groovy
implementation 'dev.langchain4j:langchain4j:{your-version}'
implementation 'dev.langchain4j:langchain4j-jlama:{your-version}'
```

### Model Selection
You can use most safetensor models on [HuggingFace](https://huggingface.co/models?library=safetensors&sort=trending) and specify them using the `owner/model-name` format.
Jlama maintains a list of pre-quantized popular models under http://huggingface.co/tjake

Models that use the following architecture are supported:
- Gemma Models
- Llama Models
- Mistral Models
- Mixtral Models
- GPT-2 Models
- BERT Models

## Chat Completion
The chat models allow you to generate human-like responses with a model fined-tuned on conversational data.

### Synchronous
Create a class and add the following code.

```java
import dev.langchain4j.model.chat.ChatLanguageModel;
import dev.langchain4j.model.jlama.JlamaChatModel;

public class HelloWorld {
    public static void main(String[] args) {
        ChatLanguageModel model = JlamaChatLanguageModel.builder()
                .modelName("tjake/TinyLlama-1.1B-Chat-v1.0-Jlama-Q4")
                .build();

        String response = model.generate("Say 'Hello World'");
        System.out.println(response);
    }
}
```
Running the program will generate a variant of the following output

```plaintext
Hello World! How can I assist you today?
```

### Streaming
Create a class and add the following code.

```java
import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.model.StreamingResponseHandler;
import dev.langchain4j.model.jlama.JlamaStreamingChatModel;
import dev.langchain4j.model.output.Response;

import java.util.concurrent.CompletableFuture;

public class HelloWorld {
    public static void main(String[] args) {
        StreamingChatLanguageModel model = JlamaStreamingChatLanguageModel.builder()
                .modelName("tjake/TinyLlama-1.1B-Chat-v1.0-Jlama-Q4")
                .build();

        CompletableFuture<Response<AiMessage>> futureResponse = new CompletableFuture<>();         
        model.generate("Tell me a joke about Java", new StreamingResponseHandler() {
            @Override
            public void onNext(String token) {
                System.out.print(token);
            }

            @Override
            public void onComplete(Response<AiMessage> response) {
                futureResponse.complete(response);
            }

            @Override
            public void onError(Throwable error) {
                futureResponse.completeExceptionally(error);
            }    
        });

        futureResponse.join();
    }
}
```
You will receive each chunk of text (token) as it is generated by the LLM on the `onNext` method.

You can see that output below is streamed in real-time.

```plaintext
"Why do Java developers wear glasses? Because they can't C#"
```

Of course, you can combine Jlama chat completion with other features like [Set Model Parameters](/tutorials/model-parameters) and [Chat Memory](/tutorials/chat-memory) to get more accurate responses.

In [Chat Memory](/tutorials/chat-memory) you will learn how to pass along your chat history, so the LLM knows what has been said before. If you don't pass the chat history, like in this simple example, the LLM will not know what has been said before, so it won't be able to correctly answer the second question ('What did I just ask?').

A lot of parameters are set behind the scenes, such as timeout, model type and model parameters.
In [Set Model Parameters](/tutorials/model-parameters) you will learn how to set these parameters explicitly.


Jlama has some special model parameters that you can set 

 - `modelCachePath` parameter, which allows you to specify a path to a directory where the model will be cached once downloaded. Default is `~/.jlama`.
 - `workingDirectory` parameter, which allows you to keep a persistent ChatMemory on disk for a given model instance. This is faster than using Chat Memory.
 - `quantizeModelAtRuntime` parameter, which will quantize the model at runtime. The current quantization is always Q4. You can also pre-quantize the model using jlama project tools (See [Jlama Project](https://github.com/tjake/jlama) for more information).

### Function Calling
Jlama supports function calling for models that support it (Mistral, Llama-3.1, etc). 
See [Jlama Examples](https://github.com/langchain4j/langchain4j-examples/tree/main/jlama-examples)

### JSON mode
Jlama does not support JSON mode (yet). But you can always ask the model nicely to return JSON.

## Examples

- [Jlama Examples](https://github.com/langchain4j/langchain4j-examples/tree/main/jlama-examples)
