{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Replicate\n", "\n", ">[Replicate](https://replicate.com/blog/machine-learning-needs-better-tools) runs machine learning models in the cloud. We have a library of open-source models that you can run with a few lines of code. If you're building your own machine learning models, Replicate makes it easy to deploy them at scale.\n", "\n", "This example goes over how to use LangChain to interact with `Replicate` [models](https://replicate.com/explore)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# magics to auto-reload external modules in case you are making changes to langchain while working on this notebook\n", "%load_ext autoreload\n", "%autoreload 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To run this notebook, you'll need to create a [replicate](https://replicate.com) account and install the [replicate python client](https://github.com/replicate/replicate-python)." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "scrolled": true, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collecting replicate\n", " Using cached replicate-0.9.0-py3-none-any.whl (21 kB)\n", "Requirement already satisfied: packaging in /root/Source/github/docugami.langchain/libs/langchain/.venv/lib/python3.9/site-packages (from replicate) (23.1)\n", "Requirement already satisfied: pydantic>1 in /root/Source/github/docugami.langchain/libs/langchain/.venv/lib/python3.9/site-packages (from replicate) (1.10.9)\n", "Requirement already satisfied: requests>2 in /root/Source/github/docugami.langchain/libs/langchain/.venv/lib/python3.9/site-packages (from replicate) (2.28.2)\n", "Requirement already satisfied: typing-extensions>=4.2.0 in /root/Source/github/docugami.langchain/libs/langchain/.venv/lib/python3.9/site-packages (from pydantic>1->replicate) (4.5.0)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /root/Source/github/docugami.langchain/libs/langchain/.venv/lib/python3.9/site-packages (from requests>2->replicate) (3.1.0)\n", "Requirement already satisfied: idna<4,>=2.5 in /root/Source/github/docugami.langchain/libs/langchain/.venv/lib/python3.9/site-packages (from requests>2->replicate) (3.4)\n", "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /root/Source/github/docugami.langchain/libs/langchain/.venv/lib/python3.9/site-packages (from requests>2->replicate) (1.26.16)\n", "Requirement already satisfied: certifi>=2017.4.17 in /root/Source/github/docugami.langchain/libs/langchain/.venv/lib/python3.9/site-packages (from requests>2->replicate) (2023.5.7)\n", "Installing collected packages: replicate\n", "Successfully installed replicate-0.9.0\n" ] } ], "source": [ "!poetry run pip install replicate" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "tags": [] }, "outputs": [], "source": [ "# get a token: https://replicate.com/account\n", "\n", "from getpass import getpass\n", "\n", "REPLICATE_API_TOKEN = getpass()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "tags": [] }, "outputs": [], "source": [ "import os\n", "\n", "os.environ[\"REPLICATE_API_TOKEN\"] = REPLICATE_API_TOKEN" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "tags": [] }, "outputs": [], "source": [ "from langchain.llms import Replicate\n", "from langchain.prompts import PromptTemplate\nfrom langchain.chains import LLMChain" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Calling a model\n", "\n", "Find a model on the [replicate explore page](https://replicate.com/explore), and then paste in the model name and version in this format: model_name/version.\n", "\n", "For example, here is [`LLama-V2`](https://replicate.com/a16z-infra/llama13b-v2-chat)." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'1. Dogs do not have the ability to operate complex machinery like cars.\\n2. Dogs do not have human-like intelligence or cognitive abilities to understand the concept of driving.\\n3. Dogs do not have the physical ability to use their paws to press pedals or turn a steering wheel.\\n4. Therefore, a dog cannot drive a car.'" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "llm = Replicate(\n", " model=\"a16z-infra/llama13b-v2-chat:df7690f1994d94e96ad9d568eac121aecf50684a0b0963b25a41cc40061269e5\",\n", " model_kwargs={\"temperature\": 0.75, \"max_length\": 500, \"top_p\": 1},\n", ")\n", "prompt = \"\"\"\n", "User: Answer the following yes/no question by reasoning step by step. Can a dog drive a car?\n", "Assistant:\n", "\"\"\"\n", "llm(prompt)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As another example, for this [dolly model](https://replicate.com/replicate/dolly-v2-12b), click on the API tab. The model name/version would be: `replicate/dolly-v2-12b:ef0e1aefc61f8e096ebe4db6b2bacc297daf2ef6899f0f7e001ec445893500e5`\n", "\n", "Only the `model` param is required, but we can add other model params when initializing.\n", "\n", "For example, if we were running stable diffusion and wanted to change the image dimensions:\n", "\n", "```\n", "Replicate(model=\"stability-ai/stable-diffusion:db21e45d3f7023abc2a46ee38a23973f6dce16bb082a930b0c49861f96d1e5bf\", input={'image_dimensions': '512x512'})\n", "```\n", " \n", "*Note that only the first output of a model will be returned.*" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "tags": [] }, "outputs": [], "source": [ "llm = Replicate(\n", " model=\"replicate/dolly-v2-12b:ef0e1aefc61f8e096ebe4db6b2bacc297daf2ef6899f0f7e001ec445893500e5\"\n", ")" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'No, dogs lack some of the brain functions required to operate a motor vehicle. They cannot focus and react in time to accelerate or brake correctly. Additionally, they do not have enough muscle control to properly operate a steering wheel.\\n\\n'" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "prompt = \"\"\"\n", "Answer the following yes/no question by reasoning step by step. \n", "Can a dog drive a car?\n", "\"\"\"\n", "llm(prompt)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can call any replicate model using this syntax. For example, we can call stable diffusion." ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [], "source": [ "text2image = Replicate(\n", " model=\"stability-ai/stable-diffusion:db21e45d3f7023abc2a46ee38a23973f6dce16bb082a930b0c49861f96d1e5bf\",\n", " model_kwargs={\"image_dimensions\": \"512x512\"},\n", ")" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'https://pbxt.replicate.delivery/bqQq4KtzwrrYL9Bub9e7NvMTDeEMm5E9VZueTXkLE7kWumIjA/out-0.png'" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "image_output = text2image(\"A cat riding a motorcycle by Picasso\")\n", "image_output" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The model spits out a URL. Let's render it." ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: Pillow in /Users/bagatur/langchain/.venv/lib/python3.9/site-packages (9.5.0)\n", "\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.2.1\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n" ] } ], "source": [ "!poetry run pip install Pillow" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from PIL import Image\n", "import requests\n", "from io import BytesIO\n", "\n", "response = requests.get(image_output)\n", "img = Image.open(BytesIO(response.content))\n", "\n", "img" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Streaming Response\n", "You can optionally stream the response as it is produced, which is helpful to show interactivity to users for time-consuming generations. See detailed docs on [Streaming](https://python.langchain.com/docs/modules/model_io/models/llms/how_to/streaming_llm) for more information." ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1. Dogs do not have the physical ability to operate a vehicle." ] } ], "source": [ "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\n", "\n", "llm = Replicate(\n", " streaming=True,\n", " callbacks=[StreamingStdOutCallbackHandler()],\n", " model=\"a16z-infra/llama13b-v2-chat:df7690f1994d94e96ad9d568eac121aecf50684a0b0963b25a41cc40061269e5\",\n", " model_kwargs={\"temperature\": 0.75, \"max_length\": 500, \"top_p\": 1},\n", ")\n", "prompt = \"\"\"\n", "User: Answer the following yes/no question by reasoning step by step. Can a dog drive a car?\n", "Assistant:\n", "\"\"\"\n", "_ = llm(prompt)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Stop Sequences\n", "You can also specify stop sequences. If you have a definite stop sequence for the generation that you are going to parse with anyway, it is better (cheaper and faster!) to just cancel the generation once one or more stop sequences are reached, rather than letting the model ramble on till the specified `max_length`. Stop sequences work regardless of whether you are in streaming mode or not, and Replicate only charges you for the generation up until the stop sequence." ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Raw output:\n", " There are several ways to learn Python, and the best method for you will depend on your learning style and goals. Here are a few suggestions:\n", "\n", "1. Online tutorials and courses: Websites such as Codecademy, Coursera, and edX offer interactive coding lessons and courses that can help you get started with Python. These courses are often designed for beginners and cover the basics of Python programming.\n", "2. Books: There are many books available that can teach you Python, ranging from introductory texts to more advanced manuals. Some popular options include \"Python Crash Course\" by Eric Matthes, \"Automate the Boring Stuff with Python\" by Al Sweigart, and \"Python for Data Analysis\" by Wes McKinney.\n", "3. Videos: YouTube and other video platforms have a wealth of tutorials and lectures on Python programming. Many of these videos are created by experienced programmers and can provide detailed explanations and examples of Python concepts.\n", "4. Practice: One of the best ways to learn Python is to practice writing code. Start with simple programs and gradually work your way up to more complex projects. As you gain experience, you'll become more comfortable with the language and develop a better understanding of its capabilities.\n", "5. Join a community: There are many online communities and forums dedicated to Python programming, such as Reddit's r/learnpython community. These communities can provide support, resources, and feedback as you learn.\n", "6. Take online courses: Many universities and organizations offer online courses on Python programming. These courses can provide a structured learning experience and often include exercises and assignments to help you practice your skills.\n", "7. Use a Python IDE: An Integrated Development Environment (IDE) is a software application that provides an interface for writing, debugging, and testing code. Popular Python IDEs include PyCharm, Visual Studio Code, and Spyder. These tools can help you write more efficient code and provide features such as code completion, debugging, and project management.\n", "\n", "\n", "Which of the above options do you think is the best way to learn Python?\n", "Raw output runtime: 25.27470933299992 seconds\n", "Stopped output:\n", " There are several ways to learn Python, and the best method for you will depend on your learning style and goals. Here are some suggestions:\n", "Stopped output runtime: 25.77039254200008 seconds\n" ] } ], "source": [ "import time\n", "\n", "llm = Replicate(\n", " model=\"a16z-infra/llama13b-v2-chat:df7690f1994d94e96ad9d568eac121aecf50684a0b0963b25a41cc40061269e5\",\n", " model_kwargs={\"temperature\": 0.01, \"max_length\": 500, \"top_p\": 1},\n", ")\n", "\n", "prompt = \"\"\"\n", "User: What is the best way to learn python?\n", "Assistant:\n", "\"\"\"\n", "start_time = time.perf_counter()\n", "raw_output = llm(prompt) # raw output, no stop\n", "end_time = time.perf_counter()\n", "print(f\"Raw output:\\n {raw_output}\")\n", "print(f\"Raw output runtime: {end_time - start_time} seconds\")\n", "\n", "start_time = time.perf_counter()\n", "stopped_output = llm(prompt, stop=[\"\\n\\n\"]) # stop on double newlines\n", "end_time = time.perf_counter()\n", "print(f\"Stopped output:\\n {stopped_output}\")\n", "print(f\"Stopped output runtime: {end_time - start_time} seconds\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Chaining Calls\n", "The whole point of langchain is to... chain! Here's an example of how do that." ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "from langchain.chains import SimpleSequentialChain" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, let's define the LLM for this model as a flan-5, and text2image as a stable diffusion model." ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [], "source": [ "dolly_llm = Replicate(\n", " model=\"replicate/dolly-v2-12b:ef0e1aefc61f8e096ebe4db6b2bacc297daf2ef6899f0f7e001ec445893500e5\"\n", ")\n", "text2image = Replicate(\n", " model=\"stability-ai/stable-diffusion:db21e45d3f7023abc2a46ee38a23973f6dce16bb082a930b0c49861f96d1e5bf\"\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First prompt in the chain" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [], "source": [ "prompt = PromptTemplate(\n", " input_variables=[\"product\"],\n", " template=\"What is a good name for a company that makes {product}?\",\n", ")\n", "\n", "chain = LLMChain(llm=dolly_llm, prompt=prompt)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Second prompt to get the logo for company description" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [], "source": [ "second_prompt = PromptTemplate(\n", " input_variables=[\"company_name\"],\n", " template=\"Write a description of a logo for this company: {company_name}\",\n", ")\n", "chain_two = LLMChain(llm=dolly_llm, prompt=second_prompt)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Third prompt, let's create the image based on the description output from prompt 2" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [], "source": [ "third_prompt = PromptTemplate(\n", " input_variables=[\"company_logo_description\"],\n", " template=\"{company_logo_description}\",\n", ")\n", "chain_three = LLMChain(llm=text2image, prompt=third_prompt)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now let's run it!" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[1m> Entering new SimpleSequentialChain chain...\u001b[0m\n", "\u001b[36;1m\u001b[1;3mColorful socks could be named after a song by The Beatles or a color (yellow, blue, pink). A good combination of letters and digits would be 6399. Apple also owns the domain 6399.com so this could be reserved for the Company.\n", "\n", "\u001b[0m\n", "\u001b[33;1m\u001b[1;3mA colorful sock with the numbers 3, 9, and 99 screen printed in yellow, blue, and pink, respectively.\n", "\n", "\u001b[0m\n", "\u001b[38;5;200m\u001b[1;3mhttps://pbxt.replicate.delivery/P8Oy3pZ7DyaAC1nbJTxNw95D1A3gCPfi2arqlPGlfG9WYTkRA/out-0.png\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "https://pbxt.replicate.delivery/P8Oy3pZ7DyaAC1nbJTxNw95D1A3gCPfi2arqlPGlfG9WYTkRA/out-0.png\n" ] } ], "source": [ "# Run the chain specifying only the input variable for the first chain.\n", "overall_chain = SimpleSequentialChain(\n", " chains=[chain, chain_two, chain_three], verbose=True\n", ")\n", "catchphrase = overall_chain.run(\"colorful socks\")\n", "print(catchphrase)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "response = requests.get(\n", " \"https://replicate.delivery/pbxt/682XgeUlFela7kmZgPOf39dDdGDDkwjsCIJ0aQ0AO5bTbbkiA/out-0.png\"\n", ")\n", "img = Image.open(BytesIO(response.content))\n", "img" ] } ], "metadata": { "kernelspec": { "display_name": "poetry-venv", "language": "python", "name": "poetry-venv" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.1" }, "vscode": { "interpreter": { "hash": "a0a0263b650d907a3bfe41c0f8d6a63a071b884df3cfdc1579f00cdc1aed6b03" } } }, "nbformat": 4, "nbformat_minor": 4 }