blob42/langchain
1
0
Fork 0
forked from Archives/langchain
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Fix age inconsistency in plan and execute Jupyter notebook example (#4814)

Browse Source 
The current example in
https://python.langchain.com/en/latest/modules/agents/plan_and_execute.html
has inconsistent reasoning step (observing 28 years and thinking it's 26
years):

```
Observation: 28 years
Thought:Based on my search, Gigi Hadid's current age is 26 years old. 
Action:
{
  "action": "Final Answer",
  "action_input": "Gigi Hadid's current age is 26 years old."
}
```

Guessing this is model noise. Rerunning seems to give correct answer of
28 years.
This commit is contained in:
keenangraham 2023-05-16 15:27:27 -07:00 committed by GitHub
parent 61f9c52fc7
commit bcce9a3a92
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 21 additions and 42 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

63
docs/modules/agents/plan_and_execute.ipynb
View File

@ -194,14 +194,18 @@
"\n", "\n",
"\u001b[0m\n", "\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m28 years\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m28 years\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mBased on my search, Gigi Hadid's current age is 26 years old. \n", "Thought:\u001b[32;1m\u001b[1;3mPrevious steps: steps=[(Step(value=\"Search for Leo DiCaprio's girlfriend on the internet.\"), StepResponse(response='Leo DiCaprio is currently linked to Gigi Hadid.')), (Step(value='Find her current age.'), StepResponse(response='28 years'))]\n",
"\n",
"Current objective: None\n",
"\n",
"Action:\n", "Action:\n",
"```\n", "```\n",
"{\n", "{\n",
" \"action\": \"Final Answer\",\n", " \"action\": \"Final Answer\",\n",
" \"action_input\": \"Gigi Hadid's current age is 26 years old.\"\n", " \"action_input\": \"Gigi Hadid's current age is 28 years.\"\n",
"}\n", "}\n",
"```\n", "```\n",
"\n",
"\u001b[0m\n", "\u001b[0m\n",
"\n", "\n",
"\u001b[1m> Finished chain.\u001b[0m\n", "\u001b[1m> Finished chain.\u001b[0m\n",
@ -209,64 +213,39 @@
"\n", "\n",
"Step: Find her current age.\n", "Step: Find her current age.\n",
"\n", "\n",
"Response: Gigi Hadid's current age is 26 years old.\n", "Response: Gigi Hadid's current age is 28 years.\n",
"\n", "\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction:\n", "\u001b[32;1m\u001b[1;3mAction:\n",
"```\n", "```\n",
"{\n", "{\n",
" \"action\": \"Calculator\",\n", " \"action\": \"Calculator\",\n",
" \"action_input\": \"26 ** 0.43\"\n", " \"action_input\": \"28 ** 0.43\"\n",
"}\n", "}\n",
"```\n", "```\n",
"\n",
"\u001b[0m\n", "\u001b[0m\n",
"\n", "\n",
"\u001b[1m> Entering new LLMMathChain chain...\u001b[0m\n", "\u001b[1m> Entering new LLMMathChain chain...\u001b[0m\n",
"26 ** 0.43\u001b[32;1m\u001b[1;3m\n", "28 ** 0.43\u001b[32;1m\u001b[1;3m\n",
"```text\n", "```text\n",
"26 ** 0.43\n", "28 ** 0.43\n",
"```\n", "```\n",
"...numexpr.evaluate(\"26 ** 0.43\")...\n", "...numexpr.evaluate(\"28 ** 0.43\")...\n",
"\u001b[0m\n", "\u001b[0m\n",
"Answer: \u001b[33;1m\u001b[1;3m4.059182145592686\u001b[0m\n", "Answer: \u001b[33;1m\u001b[1;3m4.1906168361987195\u001b[0m\n",
"\u001b[1m> Finished chain.\u001b[0m\n", "\u001b[1m> Finished chain.\u001b[0m\n",
"\n", "\n",
"Observation: \u001b[33;1m\u001b[1;3mAnswer: 4.059182145592686\u001b[0m\n", "Observation: \u001b[33;1m\u001b[1;3mAnswer: 4.1906168361987195\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mThe current objective is to raise Gigi Hadid's age to the 0.43 power. \n", "Thought:\u001b[32;1m\u001b[1;3mThe next step is to provide the answer to the user's question.\n",
"\n",
"Action:\n",
"```\n",
"{\n",
" \"action\": \"Calculator\",\n",
" \"action_input\": \"26 ** 0.43\"\n",
"}\n",
"```\n",
"\n",
"\u001b[0m\n",
"\n",
"\u001b[1m> Entering new LLMMathChain chain...\u001b[0m\n",
"26 ** 0.43\u001b[32;1m\u001b[1;3m\n",
"```text\n",
"26 ** 0.43\n",
"```\n",
"...numexpr.evaluate(\"26 ** 0.43\")...\n",
"\u001b[0m\n",
"Answer: \u001b[33;1m\u001b[1;3m4.059182145592686\u001b[0m\n",
"\u001b[1m> Finished chain.\u001b[0m\n",
"\n",
"Observation: \u001b[33;1m\u001b[1;3mAnswer: 4.059182145592686\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mThe answer to the current objective is 4.059182145592686.\n",
"\n", "\n",
"Action:\n", "Action:\n",
"```\n", "```\n",
"{\n", "{\n",
" \"action\": \"Final Answer\",\n", " \"action\": \"Final Answer\",\n",
" \"action_input\": \"Gigi Hadid's age raised to the 0.43 power is approximately 4.059 years.\"\n", " \"action_input\": \"Gigi Hadid's current age raised to the 0.43 power is approximately 4.19.\"\n",
"}\n", "}\n",
"```\n", "```\n",
"\n", "\n",
"\n",
"\u001b[0m\n", "\u001b[0m\n",
"\n", "\n",
"\u001b[1m> Finished chain.\u001b[0m\n", "\u001b[1m> Finished chain.\u001b[0m\n",
@ -274,14 +253,14 @@
"\n", "\n",
"Step: Raise her current age to the 0.43 power using a calculator or programming language.\n", "Step: Raise her current age to the 0.43 power using a calculator or programming language.\n",
"\n", "\n",
"Response: Gigi Hadid's age raised to the 0.43 power is approximately 4.059 years.\n", "Response: Gigi Hadid's current age raised to the 0.43 power is approximately 4.19.\n",
"\n", "\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction:\n", "\u001b[32;1m\u001b[1;3mAction:\n",
"```\n", "```\n",
"{\n", "{\n",
" \"action\": \"Final Answer\",\n", " \"action\": \"Final Answer\",\n",
" \"action_input\": \"Gigi Hadid's age raised to the 0.43 power is approximately 4.059 years.\"\n", " \"action_input\": \"The result is approximately 4.19.\"\n",
"}\n", "}\n",
"```\n", "```\n",
"\u001b[0m\n", "\u001b[0m\n",
@ -291,14 +270,14 @@
"\n", "\n",
"Step: Output the result.\n", "Step: Output the result.\n",
"\n", "\n",
"Response: Gigi Hadid's age raised to the 0.43 power is approximately 4.059 years.\n", "Response: The result is approximately 4.19.\n",
"\n", "\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction:\n", "\u001b[32;1m\u001b[1;3mAction:\n",
"```\n", "```\n",
"{\n", "{\n",
" \"action\": \"Final Answer\",\n", " \"action\": \"Final Answer\",\n",
" \"action_input\": \"Gigi Hadid's age raised to the 0.43 power is approximately 4.059 years.\"\n", " \"action_input\": \"Gigi Hadid's current age raised to the 0.43 power is approximately 4.19.\"\n",
"}\n", "}\n",
"```\n", "```\n",
"\u001b[0m\n", "\u001b[0m\n",
@ -310,14 +289,14 @@
"\n", "\n",
"\n", "\n",
"\n", "\n",
"Response: Gigi Hadid's age raised to the 0.43 power is approximately 4.059 years.\n", "Response: Gigi Hadid's current age raised to the 0.43 power is approximately 4.19.\n",
"\u001b[1m> Finished chain.\u001b[0m\n" "\u001b[1m> Finished chain.\u001b[0m\n"
] ]
}, },
{ {
"data": { "data": {
"text/plain": [ "text/plain": [
"\"Gigi Hadid's age raised to the 0.43 power is approximately 4.059 years.\"" "\"Gigi Hadid's current age raised to the 0.43 power is approximately 4.19.\""
] ]
}, },
"execution_count": 10, "execution_count": 10,