{ "cells": [ { "cell_type": "markdown", "id": "517a9fd4", "metadata": {}, "source": [ "# BabyAGI with Tools\n", "\n", "This notebook builds on top of [baby agi](baby_agi.html), but shows how you can swap out the execution chain. The previous execution chain was just an LLM which made stuff up. By swapping it out with an agent that has access to tools, we can hopefully get real reliable information" ] }, { "cell_type": "markdown", "id": "556af556", "metadata": {}, "source": [ "## Install and Import Required Modules" ] }, { "cell_type": "code", "execution_count": 1, "id": "c8a354b6", "metadata": {}, "outputs": [], "source": [ "import os\n", "from collections import deque\n", "from typing import Dict, List, Optional, Any\n", "\n", "from langchain import LLMChain, OpenAI, PromptTemplate\n", "from langchain.embeddings import OpenAIEmbeddings\n", "from langchain.llms import BaseLLM\n", "from langchain.vectorstores.base import VectorStore\n", "from pydantic import BaseModel, Field\n", "from langchain.chains.base import Chain" ] }, { "cell_type": "markdown", "id": "09f70772", "metadata": {}, "source": [ "## Connect to the Vector Store\n", "\n", "Depending on what vectorstore you use, this step may look different." ] }, { "cell_type": "code", "execution_count": 2, "id": "794045d4", "metadata": {}, "outputs": [], "source": [ "%pip install faiss-cpu > /dev/null\n", "%pip install google-search-results > /dev/null\n", "from langchain.vectorstores import FAISS\n", "from langchain.docstore import InMemoryDocstore" ] }, { "cell_type": "code", "execution_count": 3, "id": "6e0305eb", "metadata": {}, "outputs": [], "source": [ "# Define your embedding model\n", "embeddings_model = OpenAIEmbeddings()\n", "# Initialize the vectorstore as empty\n", "import faiss\n", "\n", "embedding_size = 1536\n", "index = faiss.IndexFlatL2(embedding_size)\n", "vectorstore = FAISS(embeddings_model.embed_query, index, InMemoryDocstore({}), {})" ] }, { "cell_type": "markdown", "id": "0f3b72bf", "metadata": {}, "source": [ "## Define the Chains\n", "\n", "BabyAGI relies on three LLM chains:\n", "- Task creation chain to select new tasks to add to the list\n", "- Task prioritization chain to re-prioritize tasks\n", "- Execution Chain to execute the tasks\n", "\n", "\n", "NOTE: in this notebook, the Execution chain will now be an agent." ] }, { "cell_type": "code", "execution_count": 4, "id": "bf4bd5cd", "metadata": {}, "outputs": [], "source": [ "class TaskCreationChain(LLMChain):\n", " \"\"\"Chain to generates tasks.\"\"\"\n", "\n", " @classmethod\n", " def from_llm(cls, llm: BaseLLM, verbose: bool = True) -> LLMChain:\n", " \"\"\"Get the response parser.\"\"\"\n", " task_creation_template = (\n", " \"You are an task creation AI that uses the result of an execution agent\"\n", " \" to create new tasks with the following objective: {objective},\"\n", " \" The last completed task has the result: {result}.\"\n", " \" This result was based on this task description: {task_description}.\"\n", " \" These are incomplete tasks: {incomplete_tasks}.\"\n", " \" Based on the result, create new tasks to be completed\"\n", " \" by the AI system that do not overlap with incomplete tasks.\"\n", " \" Return the tasks as an array.\"\n", " )\n", " prompt = PromptTemplate(\n", " template=task_creation_template,\n", " input_variables=[\n", " \"result\",\n", " \"task_description\",\n", " \"incomplete_tasks\",\n", " \"objective\",\n", " ],\n", " )\n", " return cls(prompt=prompt, llm=llm, verbose=verbose)" ] }, { "cell_type": "code", "execution_count": 5, "id": "b6488ffe", "metadata": {}, "outputs": [], "source": [ "class TaskPrioritizationChain(LLMChain):\n", " \"\"\"Chain to prioritize tasks.\"\"\"\n", "\n", " @classmethod\n", " def from_llm(cls, llm: BaseLLM, verbose: bool = True) -> LLMChain:\n", " \"\"\"Get the response parser.\"\"\"\n", " task_prioritization_template = (\n", " \"You are an task prioritization AI tasked with cleaning the formatting of and reprioritizing\"\n", " \" the following tasks: {task_names}.\"\n", " \" Consider the ultimate objective of your team: {objective}.\"\n", " \" Do not remove any tasks. Return the result as a numbered list, like:\"\n", " \" #. First task\"\n", " \" #. Second task\"\n", " \" Start the task list with number {next_task_id}.\"\n", " )\n", " prompt = PromptTemplate(\n", " template=task_prioritization_template,\n", " input_variables=[\"task_names\", \"next_task_id\", \"objective\"],\n", " )\n", " return cls(prompt=prompt, llm=llm, verbose=verbose)" ] }, { "cell_type": "code", "execution_count": 46, "id": "b43cd580", "metadata": {}, "outputs": [], "source": [ "from langchain.agents import ZeroShotAgent, Tool, AgentExecutor\n", "from langchain import OpenAI, SerpAPIWrapper, LLMChain\n", "\n", "todo_prompt = PromptTemplate.from_template(\n", " \"You are a planner who is an expert at coming up with a todo list for a given objective. Come up with a todo list for this objective: {objective}\"\n", ")\n", "todo_chain = LLMChain(llm=OpenAI(temperature=0), prompt=todo_prompt)\n", "search = SerpAPIWrapper()\n", "tools = [\n", " Tool(\n", " name=\"Search\",\n", " func=search.run,\n", " description=\"useful for when you need to answer questions about current events\",\n", " ),\n", " Tool(\n", " name=\"TODO\",\n", " func=todo_chain.run,\n", " description=\"useful for when you need to come up with todo lists. Input: an objective to create a todo list for. Output: a todo list for that objective. Please be very clear what the objective is!\",\n", " ),\n", "]\n", "\n", "\n", "prefix = \"\"\"You are an AI who performs one task based on the following objective: {objective}. Take into account these previously completed tasks: {context}.\"\"\"\n", "suffix = \"\"\"Question: {task}\n", "{agent_scratchpad}\"\"\"\n", "prompt = ZeroShotAgent.create_prompt(\n", " tools,\n", " prefix=prefix,\n", " suffix=suffix,\n", " input_variables=[\"objective\", \"task\", \"context\", \"agent_scratchpad\"],\n", ")" ] }, { "cell_type": "markdown", "id": "3ad996c5", "metadata": {}, "source": [ "### Define the BabyAGI Controller\n", "\n", "BabyAGI composes the chains defined above in a (potentially-)infinite loop." ] }, { "cell_type": "code", "execution_count": 47, "id": "0ada0636", "metadata": {}, "outputs": [], "source": [ "def get_next_task(\n", " task_creation_chain: LLMChain,\n", " result: Dict,\n", " task_description: str,\n", " task_list: List[str],\n", " objective: str,\n", ") -> List[Dict]:\n", " \"\"\"Get the next task.\"\"\"\n", " incomplete_tasks = \", \".join(task_list)\n", " response = task_creation_chain.run(\n", " result=result,\n", " task_description=task_description,\n", " incomplete_tasks=incomplete_tasks,\n", " objective=objective,\n", " )\n", " new_tasks = response.split(\"\\n\")\n", " return [{\"task_name\": task_name} for task_name in new_tasks if task_name.strip()]" ] }, { "cell_type": "code", "execution_count": 48, "id": "d35250ad", "metadata": {}, "outputs": [], "source": [ "def prioritize_tasks(\n", " task_prioritization_chain: LLMChain,\n", " this_task_id: int,\n", " task_list: List[Dict],\n", " objective: str,\n", ") -> List[Dict]:\n", " \"\"\"Prioritize tasks.\"\"\"\n", " task_names = [t[\"task_name\"] for t in task_list]\n", " next_task_id = int(this_task_id) + 1\n", " response = task_prioritization_chain.run(\n", " task_names=task_names, next_task_id=next_task_id, objective=objective\n", " )\n", " new_tasks = response.split(\"\\n\")\n", " prioritized_task_list = []\n", " for task_string in new_tasks:\n", " if not task_string.strip():\n", " continue\n", " task_parts = task_string.strip().split(\".\", 1)\n", " if len(task_parts) == 2:\n", " task_id = task_parts[0].strip()\n", " task_name = task_parts[1].strip()\n", " prioritized_task_list.append({\"task_id\": task_id, \"task_name\": task_name})\n", " return prioritized_task_list" ] }, { "cell_type": "code", "execution_count": 49, "id": "e3f1840c", "metadata": {}, "outputs": [], "source": [ "def _get_top_tasks(vectorstore, query: str, k: int) -> List[str]:\n", " \"\"\"Get the top k tasks based on the query.\"\"\"\n", " results = vectorstore.similarity_search_with_score(query, k=k)\n", " if not results:\n", " return []\n", " sorted_results, _ = zip(*sorted(results, key=lambda x: x[1], reverse=True))\n", " return [str(item.metadata[\"task\"]) for item in sorted_results]\n", "\n", "\n", "def execute_task(\n", " vectorstore, execution_chain: LLMChain, objective: str, task: str, k: int = 5\n", ") -> str:\n", " \"\"\"Execute a task.\"\"\"\n", " context = _get_top_tasks(vectorstore, query=objective, k=k)\n", " return execution_chain.run(objective=objective, context=context, task=task)" ] }, { "cell_type": "code", "execution_count": 50, "id": "1e978938", "metadata": {}, "outputs": [], "source": [ "class BabyAGI(Chain, BaseModel):\n", " \"\"\"Controller model for the BabyAGI agent.\"\"\"\n", "\n", " task_list: deque = Field(default_factory=deque)\n", " task_creation_chain: TaskCreationChain = Field(...)\n", " task_prioritization_chain: TaskPrioritizationChain = Field(...)\n", " execution_chain: AgentExecutor = Field(...)\n", " task_id_counter: int = Field(1)\n", " vectorstore: VectorStore = Field(init=False)\n", " max_iterations: Optional[int] = None\n", "\n", " class Config:\n", " \"\"\"Configuration for this pydantic object.\"\"\"\n", "\n", " arbitrary_types_allowed = True\n", "\n", " def add_task(self, task: Dict):\n", " self.task_list.append(task)\n", "\n", " def print_task_list(self):\n", " print(\"\\033[95m\\033[1m\" + \"\\n*****TASK LIST*****\\n\" + \"\\033[0m\\033[0m\")\n", " for t in self.task_list:\n", " print(str(t[\"task_id\"]) + \": \" + t[\"task_name\"])\n", "\n", " def print_next_task(self, task: Dict):\n", " print(\"\\033[92m\\033[1m\" + \"\\n*****NEXT TASK*****\\n\" + \"\\033[0m\\033[0m\")\n", " print(str(task[\"task_id\"]) + \": \" + task[\"task_name\"])\n", "\n", " def print_task_result(self, result: str):\n", " print(\"\\033[93m\\033[1m\" + \"\\n*****TASK RESULT*****\\n\" + \"\\033[0m\\033[0m\")\n", " print(result)\n", "\n", " @property\n", " def input_keys(self) -> List[str]:\n", " return [\"objective\"]\n", "\n", " @property\n", " def output_keys(self) -> List[str]:\n", " return []\n", "\n", " def _call(self, inputs: Dict[str, Any]) -> Dict[str, Any]:\n", " \"\"\"Run the agent.\"\"\"\n", " objective = inputs[\"objective\"]\n", " first_task = inputs.get(\"first_task\", \"Make a todo list\")\n", " self.add_task({\"task_id\": 1, \"task_name\": first_task})\n", " num_iters = 0\n", " while True:\n", " if self.task_list:\n", " self.print_task_list()\n", "\n", " # Step 1: Pull the first task\n", " task = self.task_list.popleft()\n", " self.print_next_task(task)\n", "\n", " # Step 2: Execute the task\n", " result = execute_task(\n", " self.vectorstore, self.execution_chain, objective, task[\"task_name\"]\n", " )\n", " this_task_id = int(task[\"task_id\"])\n", " self.print_task_result(result)\n", "\n", " # Step 3: Store the result in Pinecone\n", " result_id = f\"result_{task['task_id']}\"\n", " self.vectorstore.add_texts(\n", " texts=[result],\n", " metadatas=[{\"task\": task[\"task_name\"]}],\n", " ids=[result_id],\n", " )\n", "\n", " # Step 4: Create new tasks and reprioritize task list\n", " new_tasks = get_next_task(\n", " self.task_creation_chain,\n", " result,\n", " task[\"task_name\"],\n", " [t[\"task_name\"] for t in self.task_list],\n", " objective,\n", " )\n", " for new_task in new_tasks:\n", " self.task_id_counter += 1\n", " new_task.update({\"task_id\": self.task_id_counter})\n", " self.add_task(new_task)\n", " self.task_list = deque(\n", " prioritize_tasks(\n", " self.task_prioritization_chain,\n", " this_task_id,\n", " list(self.task_list),\n", " objective,\n", " )\n", " )\n", " num_iters += 1\n", " if self.max_iterations is not None and num_iters == self.max_iterations:\n", " print(\n", " \"\\033[91m\\033[1m\" + \"\\n*****TASK ENDING*****\\n\" + \"\\033[0m\\033[0m\"\n", " )\n", " break\n", " return {}\n", "\n", " @classmethod\n", " def from_llm(\n", " cls, llm: BaseLLM, vectorstore: VectorStore, verbose: bool = False, **kwargs\n", " ) -> \"BabyAGI\":\n", " \"\"\"Initialize the BabyAGI Controller.\"\"\"\n", " task_creation_chain = TaskCreationChain.from_llm(llm, verbose=verbose)\n", " task_prioritization_chain = TaskPrioritizationChain.from_llm(\n", " llm, verbose=verbose\n", " )\n", " llm_chain = LLMChain(llm=llm, prompt=prompt)\n", " tool_names = [tool.name for tool in tools]\n", " agent = ZeroShotAgent(llm_chain=llm_chain, allowed_tools=tool_names)\n", " agent_executor = AgentExecutor.from_agent_and_tools(\n", " agent=agent, tools=tools, verbose=True\n", " )\n", " return cls(\n", " task_creation_chain=task_creation_chain,\n", " task_prioritization_chain=task_prioritization_chain,\n", " execution_chain=agent_executor,\n", " vectorstore=vectorstore,\n", " **kwargs,\n", " )" ] }, { "cell_type": "markdown", "id": "05ba762e", "metadata": {}, "source": [ "### Run the BabyAGI\n", "\n", "Now it's time to create the BabyAGI controller and watch it try to accomplish your objective." ] }, { "cell_type": "code", "execution_count": 51, "id": "3d220b69", "metadata": {}, "outputs": [], "source": [ "OBJECTIVE = \"Write a weather report for SF today\"" ] }, { "cell_type": "code", "execution_count": 52, "id": "8a8e5543", "metadata": {}, "outputs": [], "source": [ "llm = OpenAI(temperature=0)" ] }, { "cell_type": "code", "execution_count": 53, "id": "3d69899b", "metadata": {}, "outputs": [], "source": [ "# Logging of LLMChains\n", "verbose = False\n", "# If None, will keep on going forever\n", "max_iterations: Optional[int] = 3\n", "baby_agi = BabyAGI.from_llm(\n", " llm=llm, vectorstore=vectorstore, verbose=verbose, max_iterations=max_iterations\n", ")" ] }, { "cell_type": "code", "execution_count": 54, "id": "f7957b51", "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[95m\u001b[1m\n", "*****TASK LIST*****\n", "\u001b[0m\u001b[0m\n", "1: Make a todo list\n", "\u001b[92m\u001b[1m\n", "*****NEXT TASK*****\n", "\u001b[0m\u001b[0m\n", "1: Make a todo list\n", "\n", "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3mThought: I need to gather data on the current weather conditions in SF\n", "Action: Search\n", "Action Input: Current weather conditions in SF\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3mHigh 67F. Winds WNW at 10 to 15 mph. Clear to partly cloudy.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I need to make a todo list\n", "Action: TODO\n", "Action Input: Write a weather report for SF today\u001b[0m\n", "Observation: \u001b[33;1m\u001b[1;3m\n", "\n", "1. Research current weather conditions in San Francisco\n", "2. Gather data on temperature, humidity, wind speed, and other relevant weather conditions\n", "3. Analyze data to determine current weather trends\n", "4. Write a brief introduction to the weather report\n", "5. Describe current weather conditions in San Francisco\n", "6. Discuss any upcoming weather changes\n", "7. Summarize the weather report\n", "8. Proofread and edit the report\n", "9. Submit the report\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", "Final Answer: A weather report for SF today should include research on current weather conditions in San Francisco, gathering data on temperature, humidity, wind speed, and other relevant weather conditions, analyzing data to determine current weather trends, writing a brief introduction to the weather report, describing current weather conditions in San Francisco, discussing any upcoming weather changes, summarizing the weather report, proofreading and editing the report, and submitting the report.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "\u001b[93m\u001b[1m\n", "*****TASK RESULT*****\n", "\u001b[0m\u001b[0m\n", "A weather report for SF today should include research on current weather conditions in San Francisco, gathering data on temperature, humidity, wind speed, and other relevant weather conditions, analyzing data to determine current weather trends, writing a brief introduction to the weather report, describing current weather conditions in San Francisco, discussing any upcoming weather changes, summarizing the weather report, proofreading and editing the report, and submitting the report.\n", "\u001b[95m\u001b[1m\n", "*****TASK LIST*****\n", "\u001b[0m\u001b[0m\n", "2: Gather data on temperature, humidity, wind speed, and other relevant weather conditions\n", "3: Analyze data to determine current weather trends\n", "4: Write a brief introduction to the weather report\n", "5: Describe current weather conditions in San Francisco\n", "6: Discuss any upcoming weather changes\n", "7: Summarize the weather report\n", "8: Proofread and edit the report\n", "9: Submit the report\n", "1: Research current weather conditions in San Francisco\n", "\u001b[92m\u001b[1m\n", "*****NEXT TASK*****\n", "\u001b[0m\u001b[0m\n", "2: Gather data on temperature, humidity, wind speed, and other relevant weather conditions\n", "\n", "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3mThought: I need to search for the current weather conditions in SF\n", "Action: Search\n", "Action Input: Current weather conditions in SF\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3mHigh 67F. Winds WNW at 10 to 15 mph. Clear to partly cloudy.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I need to make a todo list\n", "Action: TODO\n", "Action Input: Create a weather report for SF today\u001b[0m\n", "Observation: \u001b[33;1m\u001b[1;3m\n", "\n", "1. Gather current weather data for SF, including temperature, wind speed, humidity, and precipitation.\n", "2. Research historical weather data for SF to compare current conditions.\n", "3. Analyze current and historical data to determine any trends or patterns.\n", "4. Create a visual representation of the data, such as a graph or chart.\n", "5. Write a summary of the weather report, including key findings and any relevant information.\n", "6. Publish the weather report on a website or other platform.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", "Final Answer: Today in San Francisco, the temperature is 67F with winds WNW at 10 to 15 mph. The sky is clear to partly cloudy.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "\u001b[93m\u001b[1m\n", "*****TASK RESULT*****\n", "\u001b[0m\u001b[0m\n", "Today in San Francisco, the temperature is 67F with winds WNW at 10 to 15 mph. The sky is clear to partly cloudy.\n", "\u001b[95m\u001b[1m\n", "*****TASK LIST*****\n", "\u001b[0m\u001b[0m\n", "3: Research current weather conditions in San Francisco\n", "4: Compare the current weather conditions in San Francisco to the average for this time of year.\n", "5: Identify any potential weather-related hazards in the area.\n", "6: Research any historical weather patterns in San Francisco.\n", "7: Analyze data to determine current weather trends\n", "8: Include any relevant data from nearby cities in the report.\n", "9: Include any relevant data from the National Weather Service in the report.\n", "10: Include any relevant data from local news sources in the report.\n", "11: Include any relevant data from online weather sources in the report.\n", "12: Include any relevant data from local meteorologists in the report.\n", "13: Include any relevant data from local weather stations in the report.\n", "14: Include any relevant data from satellite images in the report.\n", "15: Describe current weather conditions in San Francisco\n", "16: Discuss any upcoming weather changes\n", "17: Write a brief introduction to the weather report\n", "18: Summarize the weather report\n", "19: Proofread and edit the report\n", "20: Submit the report\n", "\u001b[92m\u001b[1m\n", "*****NEXT TASK*****\n", "\u001b[0m\u001b[0m\n", "3: Research current weather conditions in San Francisco\n", "\n", "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3mThought: I need to search for current weather conditions in San Francisco\n", "Action: Search\n", "Action Input: Current weather conditions in San Francisco\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3mTodaySun 04/09 High 67 · 1% Precip. ; TonightSun 04/09 Low 49 · 9% Precip. ; TomorrowMon 04/10 High 64 · 11% Precip.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", "Final Answer: Today in San Francisco, the high temperature is 67 degrees with 1% chance of precipitation. The low temperature tonight is 49 degrees with 9% chance of precipitation. Tomorrow's high temperature is 64 degrees with 11% chance of precipitation.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "\u001b[93m\u001b[1m\n", "*****TASK RESULT*****\n", "\u001b[0m\u001b[0m\n", "Today in San Francisco, the high temperature is 67 degrees with 1% chance of precipitation. The low temperature tonight is 49 degrees with 9% chance of precipitation. Tomorrow's high temperature is 64 degrees with 11% chance of precipitation.\n", "\u001b[91m\u001b[1m\n", "*****TASK ENDING*****\n", "\u001b[0m\u001b[0m\n" ] }, { "data": { "text/plain": [ "{'objective': 'Write a weather report for SF today'}" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "baby_agi({\"objective\": OBJECTIVE})" ] }, { "cell_type": "code", "execution_count": null, "id": "898a210b", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.1" } }, "nbformat": 4, "nbformat_minor": 5 }