langchain/docs/modules/models/llms/integrations/gpt4all.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GPT4All\n",
    "\n",
    "[GitHub:nomic-ai/gpt4all](https://github.com/nomic-ai/gpt4all) an ecosystem of open-source chatbots trained on a massive collections of clean assistant data including code, stories and dialogue.\n",
    "\n",
    "This example goes over how to use LangChain to interact with `GPT4All` models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install pyllamacpp > /dev/null"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain import PromptTemplate, LLMChain\n",
    "from langchain.llms import GPT4All\n",
    "from langchain.callbacks.base import CallbackManager\n",
    "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "template = \"\"\"Question: {question}\n",
    "\n",
    "Answer: Let's think step by step.\"\"\"\n",
    "\n",
    "prompt = PromptTemplate(template=template, input_variables=[\"question\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Specify Model\n",
    "\n",
    "To run locally, download a compatible ggml-formatted model. For more info, visit https://github.com/nomic-ai/pyllamacpp\n",
    "\n",
    "For full installation instructions go [here](https://gpt4all.io/index.html).\n",
    "\n",
    "The GPT4All Chat installer needs to decompress a 3GB LLM model during the installation process!\n",
    "\n",
    "Note that new models are uploaded regularly - check the link above for the most recent `.bin` URL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "local_path = './models/gpt4all-lora-quantized-ggml.bin'  # replace with your desired local file path"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Uncomment the below block to download a model. You may want to update `url` to a new version."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import requests\n",
    "\n",
    "# from pathlib import Path\n",
    "# from tqdm import tqdm\n",
    "\n",
    "# Path(local_path).parent.mkdir(parents=True, exist_ok=True)\n",
    "\n",
    "# # Example model. Check https://github.com/nomic-ai/pyllamacpp for the latest models.\n",
    "# url = 'https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-quantized-ggml.bin'\n",
    "\n",
    "# # send a GET request to the URL to download the file. Stream since it's large\n",
    "# response = requests.get(url, stream=True)\n",
    "\n",
    "# # open the file in binary mode and write the contents of the response to it in chunks\n",
    "# # This is a large file, so be prepared to wait.\n",
    "# with open(local_path, 'wb') as f:\n",
    "#     for chunk in tqdm(response.iter_content(chunk_size=8192)):\n",
    "#         if chunk:\n",
    "#             f.write(chunk)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Callbacks support token-wise streaming\n",
    "callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])\n",
    "# Verbose is required to pass to the callback manager\n",
    "llm = GPT4All(model=local_path, callback_manager=callback_manager, verbose=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm_chain = LLMChain(prompt=prompt, llm=llm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "question = \"What NFL team won the Super Bowl in the year Justin Bieber was born?\"\n",
    "\n",
    "llm_chain.run(question)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}