langchain/docs/modules/models/llms/integrations/gpt4all.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "# GPT4All\n",
                "\n",
                "[GitHub:nomic-ai/gpt4all](https://github.com/nomic-ai/gpt4all) an ecosystem of open-source chatbots trained on a massive collections of clean assistant data including code, stories and dialogue.\n",
                "\n",
                "This example goes over how to use LangChain to interact with `GPT4All` models."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {
                "tags": []
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Note: you may need to restart the kernel to use updated packages.\n"
                    ]
                }
            ],
            "source": [
                "%pip install pygpt4all > /dev/null"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {
                "tags": []
            },
            "outputs": [],
            "source": [
                "from langchain import PromptTemplate, LLMChain\n",
                "from langchain.llms import GPT4All\n",
                "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "metadata": {
                "tags": []
            },
            "outputs": [],
            "source": [
                "template = \"\"\"Question: {question}\n",
                "\n",
                "Answer: Let's think step by step.\"\"\"\n",
                "\n",
                "prompt = PromptTemplate(template=template, input_variables=[\"question\"])"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Specify Model\n",
                "\n",
                "To run locally, download a compatible ggml-formatted model. For more info, visit https://github.com/nomic-ai/pygpt4all\n",
                "\n",
                "For full installation instructions go [here](https://gpt4all.io/index.html).\n",
                "\n",
                "The GPT4All Chat installer needs to decompress a 3GB LLM model during the installation process!\n",
                "\n",
                "Note that new models are uploaded regularly - check the link above for the most recent `.bin` URL"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "local_path = './models/ggml-gpt4all-l13b-snoozy.bin'  # replace with your desired local file path"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Uncomment the below block to download a model. You may want to update `url` to a new version."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# import requests\n",
                "\n",
                "# from pathlib import Path\n",
                "# from tqdm import tqdm\n",
                "\n",
                "# Path(local_path).parent.mkdir(parents=True, exist_ok=True)\n",
                "\n",
                "# # Example model. Check https://github.com/nomic-ai/pygpt4all for the latest models.\n",
                "# url = 'http://gpt4all.io/models/ggml-gpt4all-l13b-snoozy.bin'\n",
                "\n",
                "# # send a GET request to the URL to download the file. Stream since it's large\n",
                "# response = requests.get(url, stream=True)\n",
                "\n",
                "# # open the file in binary mode and write the contents of the response to it in chunks\n",
                "# # This is a large file, so be prepared to wait.\n",
                "# with open(local_path, 'wb') as f:\n",
                "#     for chunk in tqdm(response.iter_content(chunk_size=8192)):\n",
                "#         if chunk:\n",
                "#             f.write(chunk)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# Callbacks support token-wise streaming\n",
                "callbacks = [StreamingStdOutCallbackHandler()]\n",
                "# Verbose is required to pass to the callback manager\n",
                "llm = GPT4All(model=local_path, callbacks=callbacks, verbose=True)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "llm_chain = LLMChain(prompt=prompt, llm=llm)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "question = \"What NFL team won the Super Bowl in the year Justin Bieber was born?\"\n",
                "\n",
                "llm_chain.run(question)"
            ]
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3 (ipykernel)",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.11.2"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 4
}