forked from Archives/langchain
bcce9a3a92
The current example in https://python.langchain.com/en/latest/modules/agents/plan_and_execute.html has inconsistent reasoning step (observing 28 years and thinking it's 26 years): ``` Observation: 28 years Thought:Based on my search, Gigi Hadid's current age is 26 years old. Action: { "action": "Final Answer", "action_input": "Gigi Hadid's current age is 26 years old." } ``` Guessing this is model noise. Rerunning seems to give correct answer of 28 years.
342 lines
10 KiB
Plaintext
342 lines
10 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "406483c4",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Plan and Execute\n",
|
||
"\n",
|
||
"Plan and execute agents accomplish an objective by first planning what to do, then executing the sub tasks. This idea is largely inspired by [BabyAGI](https://github.com/yoheinakajima/babyagi) and then the [\"Plan-and-Solve\" paper](https://arxiv.org/abs/2305.04091).\n",
|
||
"\n",
|
||
"The planning is almost always done by an LLM.\n",
|
||
"\n",
|
||
"The execution is usually done by a separate agent (equipped with tools)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "91192118",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Imports"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "6ccd1dc5",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from langchain.chat_models import ChatOpenAI\n",
|
||
"from langchain.experimental.plan_and_execute import PlanAndExecute, load_agent_executor, load_chat_planner\n",
|
||
"from langchain.llms import OpenAI\n",
|
||
"from langchain import SerpAPIWrapper\n",
|
||
"from langchain.agents.tools import Tool\n",
|
||
"from langchain import LLMMathChain"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0b10d200",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Tools"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "3c00f724",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"search = SerpAPIWrapper()\n",
|
||
"llm = OpenAI(temperature=0)\n",
|
||
"llm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)\n",
|
||
"tools = [\n",
|
||
" Tool(\n",
|
||
" name = \"Search\",\n",
|
||
" func=search.run,\n",
|
||
" description=\"useful for when you need to answer questions about current events\"\n",
|
||
" ),\n",
|
||
" Tool(\n",
|
||
" name=\"Calculator\",\n",
|
||
" func=llm_math_chain.run,\n",
|
||
" description=\"useful for when you need to answer questions about math\"\n",
|
||
" ),\n",
|
||
"]"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ce38ae84",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Planner, Executor, and Agent"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "0ab2cadd",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"model = ChatOpenAI(temperature=0)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "7b2419f2",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"planner = load_chat_planner(model)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "ed9f518b",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"executor = load_agent_executor(model, tools, verbose=True)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"id": "36943178",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"agent = PlanAndExecute(planner=planner, executer=executor, verbose=True)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8be9f1bd",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Run Example"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 10,
|
||
"id": "4891062e",
|
||
"metadata": {
|
||
"scrolled": false
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\n",
|
||
"\n",
|
||
"\u001b[1m> Entering new PlanAndExecute chain...\u001b[0m\n",
|
||
"steps=[Step(value=\"Search for Leo DiCaprio's girlfriend on the internet.\"), Step(value='Find her current age.'), Step(value='Raise her current age to the 0.43 power using a calculator or programming language.'), Step(value='Output the result.'), Step(value=\"Given the above steps taken, respond to the user's original question.\\n\\n\")]\n",
|
||
"\n",
|
||
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
|
||
"\u001b[32;1m\u001b[1;3mAction:\n",
|
||
"```\n",
|
||
"{\n",
|
||
" \"action\": \"Search\",\n",
|
||
" \"action_input\": \"Who is Leo DiCaprio's girlfriend?\"\n",
|
||
"}\n",
|
||
"``` \n",
|
||
"\n",
|
||
"\u001b[0m\n",
|
||
"Observation: \u001b[36;1m\u001b[1;3mDiCaprio broke up with girlfriend Camila Morrone, 25, in the summer of 2022, after dating for four years. He's since been linked to another famous supermodel – Gigi Hadid. The power couple were first supposedly an item in September after being spotted getting cozy during a party at New York Fashion Week.\u001b[0m\n",
|
||
"Thought:\u001b[32;1m\u001b[1;3mBased on the previous observation, I can provide the answer to the current objective. \n",
|
||
"Action:\n",
|
||
"```\n",
|
||
"{\n",
|
||
" \"action\": \"Final Answer\",\n",
|
||
" \"action_input\": \"Leo DiCaprio is currently linked to Gigi Hadid.\"\n",
|
||
"}\n",
|
||
"```\n",
|
||
"\u001b[0m\n",
|
||
"\n",
|
||
"\u001b[1m> Finished chain.\u001b[0m\n",
|
||
"*****\n",
|
||
"\n",
|
||
"Step: Search for Leo DiCaprio's girlfriend on the internet.\n",
|
||
"\n",
|
||
"Response: Leo DiCaprio is currently linked to Gigi Hadid.\n",
|
||
"\n",
|
||
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
|
||
"\u001b[32;1m\u001b[1;3mAction:\n",
|
||
"```\n",
|
||
"{\n",
|
||
" \"action\": \"Search\",\n",
|
||
" \"action_input\": \"What is Gigi Hadid's current age?\"\n",
|
||
"}\n",
|
||
"```\n",
|
||
"\u001b[0m\n",
|
||
"Observation: \u001b[36;1m\u001b[1;3m28 years\u001b[0m\n",
|
||
"Thought:\u001b[32;1m\u001b[1;3mPrevious steps: steps=[(Step(value=\"Search for Leo DiCaprio's girlfriend on the internet.\"), StepResponse(response='Leo DiCaprio is currently linked to Gigi Hadid.'))]\n",
|
||
"\n",
|
||
"Current objective: value='Find her current age.'\n",
|
||
"\n",
|
||
"Action:\n",
|
||
"```\n",
|
||
"{\n",
|
||
" \"action\": \"Search\",\n",
|
||
" \"action_input\": \"What is Gigi Hadid's current age?\"\n",
|
||
"}\n",
|
||
"```\n",
|
||
"\n",
|
||
"\u001b[0m\n",
|
||
"Observation: \u001b[36;1m\u001b[1;3m28 years\u001b[0m\n",
|
||
"Thought:\u001b[32;1m\u001b[1;3mPrevious steps: steps=[(Step(value=\"Search for Leo DiCaprio's girlfriend on the internet.\"), StepResponse(response='Leo DiCaprio is currently linked to Gigi Hadid.')), (Step(value='Find her current age.'), StepResponse(response='28 years'))]\n",
|
||
"\n",
|
||
"Current objective: None\n",
|
||
"\n",
|
||
"Action:\n",
|
||
"```\n",
|
||
"{\n",
|
||
" \"action\": \"Final Answer\",\n",
|
||
" \"action_input\": \"Gigi Hadid's current age is 28 years.\"\n",
|
||
"}\n",
|
||
"```\n",
|
||
"\n",
|
||
"\u001b[0m\n",
|
||
"\n",
|
||
"\u001b[1m> Finished chain.\u001b[0m\n",
|
||
"*****\n",
|
||
"\n",
|
||
"Step: Find her current age.\n",
|
||
"\n",
|
||
"Response: Gigi Hadid's current age is 28 years.\n",
|
||
"\n",
|
||
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
|
||
"\u001b[32;1m\u001b[1;3mAction:\n",
|
||
"```\n",
|
||
"{\n",
|
||
" \"action\": \"Calculator\",\n",
|
||
" \"action_input\": \"28 ** 0.43\"\n",
|
||
"}\n",
|
||
"```\n",
|
||
"\u001b[0m\n",
|
||
"\n",
|
||
"\u001b[1m> Entering new LLMMathChain chain...\u001b[0m\n",
|
||
"28 ** 0.43\u001b[32;1m\u001b[1;3m\n",
|
||
"```text\n",
|
||
"28 ** 0.43\n",
|
||
"```\n",
|
||
"...numexpr.evaluate(\"28 ** 0.43\")...\n",
|
||
"\u001b[0m\n",
|
||
"Answer: \u001b[33;1m\u001b[1;3m4.1906168361987195\u001b[0m\n",
|
||
"\u001b[1m> Finished chain.\u001b[0m\n",
|
||
"\n",
|
||
"Observation: \u001b[33;1m\u001b[1;3mAnswer: 4.1906168361987195\u001b[0m\n",
|
||
"Thought:\u001b[32;1m\u001b[1;3mThe next step is to provide the answer to the user's question.\n",
|
||
"\n",
|
||
"Action:\n",
|
||
"```\n",
|
||
"{\n",
|
||
" \"action\": \"Final Answer\",\n",
|
||
" \"action_input\": \"Gigi Hadid's current age raised to the 0.43 power is approximately 4.19.\"\n",
|
||
"}\n",
|
||
"```\n",
|
||
"\n",
|
||
"\u001b[0m\n",
|
||
"\n",
|
||
"\u001b[1m> Finished chain.\u001b[0m\n",
|
||
"*****\n",
|
||
"\n",
|
||
"Step: Raise her current age to the 0.43 power using a calculator or programming language.\n",
|
||
"\n",
|
||
"Response: Gigi Hadid's current age raised to the 0.43 power is approximately 4.19.\n",
|
||
"\n",
|
||
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
|
||
"\u001b[32;1m\u001b[1;3mAction:\n",
|
||
"```\n",
|
||
"{\n",
|
||
" \"action\": \"Final Answer\",\n",
|
||
" \"action_input\": \"The result is approximately 4.19.\"\n",
|
||
"}\n",
|
||
"```\n",
|
||
"\u001b[0m\n",
|
||
"\n",
|
||
"\u001b[1m> Finished chain.\u001b[0m\n",
|
||
"*****\n",
|
||
"\n",
|
||
"Step: Output the result.\n",
|
||
"\n",
|
||
"Response: The result is approximately 4.19.\n",
|
||
"\n",
|
||
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
|
||
"\u001b[32;1m\u001b[1;3mAction:\n",
|
||
"```\n",
|
||
"{\n",
|
||
" \"action\": \"Final Answer\",\n",
|
||
" \"action_input\": \"Gigi Hadid's current age raised to the 0.43 power is approximately 4.19.\"\n",
|
||
"}\n",
|
||
"```\n",
|
||
"\u001b[0m\n",
|
||
"\n",
|
||
"\u001b[1m> Finished chain.\u001b[0m\n",
|
||
"*****\n",
|
||
"\n",
|
||
"Step: Given the above steps taken, respond to the user's original question.\n",
|
||
"\n",
|
||
"\n",
|
||
"\n",
|
||
"Response: Gigi Hadid's current age raised to the 0.43 power is approximately 4.19.\n",
|
||
"\u001b[1m> Finished chain.\u001b[0m\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"\"Gigi Hadid's current age raised to the 0.43 power is approximately 4.19.\""
|
||
]
|
||
},
|
||
"execution_count": 10,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"agent.run(\"Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "aa3ec998",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": []
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.11.3"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|