{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# GPT4All\n", "\n", "[GitHub:nomic-ai/gpt4all](https://github.com/nomic-ai/gpt4all) an ecosystem of open-source chatbots trained on a massive collections of clean assistant data including code, stories and dialogue.\n", "\n", "This example goes over how to use LangChain to interact with `GPT4All` models." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ "%pip install gpt4all > /dev/null" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "tags": [] }, "outputs": [], "source": [ "from langchain import PromptTemplate, LLMChain\n", "from langchain.llms import GPT4All\n", "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "tags": [] }, "outputs": [], "source": [ "template = \"\"\"Question: {question}\n", "\n", "Answer: Let's think step by step.\"\"\"\n", "\n", "prompt = PromptTemplate(template=template, input_variables=[\"question\"])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Specify Model\n", "\n", "To run locally, download a compatible ggml-formatted model. \n", " \n", "**Download option 1**: The [gpt4all page](https://gpt4all.io/index.html) has a useful `Model Explorer` section:\n", "\n", "* Select a model of interest\n", "* Download using the UI and move the `.bin` to the `local_path` (noted below)\n", "\n", "For more info, visit https://github.com/nomic-ai/gpt4all.\n", "\n", "--- \n", "\n", "**Download option 2**: Uncomment the below block to download a model. \n", "\n", "* You may want to update `url` to a new version, whih can be browsed using the [gpt4all page](https://gpt4all.io/index.html)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "local_path = (\n", " \"./models/ggml-gpt4all-l13b-snoozy.bin\" # replace with your desired local file path\n", ")\n", "\n", "# import requests\n", "\n", "# from pathlib import Path\n", "# from tqdm import tqdm\n", "\n", "# Path(local_path).parent.mkdir(parents=True, exist_ok=True)\n", "\n", "# # Example model. Check https://github.com/nomic-ai/gpt4all for the latest models.\n", "# url = 'http://gpt4all.io/models/ggml-gpt4all-l13b-snoozy.bin'\n", "\n", "# # send a GET request to the URL to download the file. Stream since it's large\n", "# response = requests.get(url, stream=True)\n", "\n", "# # open the file in binary mode and write the contents of the response to it in chunks\n", "# # This is a large file, so be prepared to wait.\n", "# with open(local_path, 'wb') as f:\n", "# for chunk in tqdm(response.iter_content(chunk_size=8192)):\n", "# if chunk:\n", "# f.write(chunk)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Callbacks support token-wise streaming\n", "callbacks = [StreamingStdOutCallbackHandler()]\n", "\n", "# Verbose is required to pass to the callback manager\n", "llm = GPT4All(model=local_path, callbacks=callbacks, verbose=True)\n", "\n", "# If you want to use a custom model add the backend parameter\n", "# Check https://docs.gpt4all.io/gpt4all_python.html for supported backends\n", "llm = GPT4All(model=local_path, backend=\"gptj\", callbacks=callbacks, verbose=True)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "llm_chain = LLMChain(prompt=prompt, llm=llm)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "question = \"What NFL team won the Super Bowl in the year Justin Bieber was born?\"\n", "\n", "llm_chain.run(question)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.16" } }, "nbformat": 4, "nbformat_minor": 4 }