Skip to main content
`Ctrl`+`K`

# GPT4All

GitHub:nomic-ai/gpt4all an ecosystem of open-source chatbots trained on a massive collections of clean assistant data including code, stories and dialogue.

This example goes over how to use LangChain to interact with `GPT4All` models.

    %pip install pyllamacpp > /dev/null

    Note: you may need to restart the kernel to use updated packages.

    from langchain import PromptTemplate, LLMChain
    from langchain.llms import GPT4All
    from langchain.callbacks.base import CallbackManager
    from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

    template = """Question: {question}

    Answer: Let's think step by step."""

    prompt = PromptTemplate(template=template, input_variables=["question"])

## Specify Model

To run locally, download a compatible ggml-formatted model. For more info, visit https://github.com/nomic-ai/pyllamacpp

For full installation instructions go here.

The GPT4All Chat installer needs to decompress a 3GB LLM model during the installation process!

Note that new models are uploaded regularly - check the link above for the most recent `.bin` URL

    local_path = './models/gpt4all-lora-quantized-ggml.bin'  # replace with your desired local file path

Uncomment the below block to download a model. You may want to update `url` to a new version.

    # import requests

    # from pathlib import Path
    # from tqdm import tqdm

    # Path(local_path).parent.mkdir(parents=True, exist_ok=True)

    # # Example model. Check https://github.com/nomic-ai/pyllamacpp for the latest models.
    # url = 'https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-quantized-ggml.bin'

    # # send a GET request to the URL to download the file. Stream since it's large
    # response = requests.get(url, stream=True)

    # # open the file in binary mode and write the contents of the response to it in chunks
    # # This is a large file, so be prepared to wait.
    # with open(local_path, 'wb') as f:
    #     for chunk in tqdm(response.iter_content(chunk_size=8192)):
    #         if chunk:
    #             f.write(chunk)

    # Callbacks support token-wise streaming
    callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])
    # Verbose is required to pass to the callback manager
    llm = GPT4All(model=local_path, callback_manager=callback_manager, verbose=True)

    llm_chain = LLMChain(prompt=prompt, llm=llm)

    question = "What NFL team won the Super Bowl in the year Justin Bieber was born?"

    llm_chain.run(question)