forked from Archives/langchain
Add HumanInputLLM (#4160)
Related: #4028, I opened a new PR because (1) I was unable to unstage mistakenly committed files (I'm not familiar with git enough to resolve this issue), (2) I felt closing the original PR and opening a new PR would be more appropriate if I changed the class name. This PR creates HumanInputLLM(HumanLLM in #4028), a simple LLM wrapper class that returns user input as the response. I also added a simple Jupyter notebook regarding how and why to use this LLM wrapper. In the notebook, I went over how to use this LLM wrapper and showed example of testing `WikipediaQueryRun` using HumanInputLLM. I believe this LLM wrapper will be useful especially for debugging, educational or testing purpose.parallel_dir_loader
Myeongseop Kim
1 year ago
committed by
GitHub
parent
6cd51ef3d0
commit
747b5f87c2
@ -0,0 +1,231 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# How (and why) to use the the human input LLM\n",
|
||||
"\n",
|
||||
"Similar to the fake LLM, LangChain provides a pseudo LLM class that can be used for testing, debugging, or educational purposes. This allows you to mock out calls to the LLM and simulate how a human would respond if they received the prompts.\n",
|
||||
"\n",
|
||||
"In this notebook, we go over how to use this.\n",
|
||||
"\n",
|
||||
"We start this with using the HumanInputLLM in an agent."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langchain.llms.human import HumanInputLLM"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langchain.agents import load_tools\n",
|
||||
"from langchain.agents import initialize_agent\n",
|
||||
"from langchain.agents import AgentType"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"tools = load_tools([\"wikipedia\"])\n",
|
||||
"llm = HumanInputLLM(prompt_func=lambda prompt: print(f\"\\n===PROMPT====\\n{prompt}\\n=====END OF PROMPT======\"))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"\n",
|
||||
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
|
||||
"\n",
|
||||
"===PROMPT====\n",
|
||||
"Answer the following questions as best you can. You have access to the following tools:\n",
|
||||
"\n",
|
||||
"Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, historical events, or other subjects. Input should be a search query.\n",
|
||||
"\n",
|
||||
"Use the following format:\n",
|
||||
"\n",
|
||||
"Question: the input question you must answer\n",
|
||||
"Thought: you should always think about what to do\n",
|
||||
"Action: the action to take, should be one of [Wikipedia]\n",
|
||||
"Action Input: the input to the action\n",
|
||||
"Observation: the result of the action\n",
|
||||
"... (this Thought/Action/Action Input/Observation can repeat N times)\n",
|
||||
"Thought: I now know the final answer\n",
|
||||
"Final Answer: the final answer to the original input question\n",
|
||||
"\n",
|
||||
"Begin!\n",
|
||||
"\n",
|
||||
"Question: What is 'Bocchi the Rock!'?\n",
|
||||
"Thought:\n",
|
||||
"=====END OF PROMPT======\n",
|
||||
"\u001b[32;1m\u001b[1;3mI need to use a tool.\n",
|
||||
"Action: Wikipedia\n",
|
||||
"Action Input: Bocchi the Rock!, Japanese four-panel manga and anime series.\u001b[0m\n",
|
||||
"Observation: \u001b[36;1m\u001b[1;3mPage: Bocchi the Rock!\n",
|
||||
"Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.\n",
|
||||
"An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.\n",
|
||||
"\n",
|
||||
"Page: Manga Time Kirara\n",
|
||||
"Summary: Manga Time Kirara (まんがタイムきらら, Manga Taimu Kirara) is a Japanese seinen manga magazine published by Houbunsha which mainly serializes four-panel manga. The magazine is sold on the ninth of each month and was first published as a special edition of Manga Time, another Houbunsha magazine, on May 17, 2002. Characters from this magazine have appeared in a crossover role-playing game called Kirara Fantasia.\n",
|
||||
"\n",
|
||||
"Page: Manga Time Kirara Max\n",
|
||||
"Summary: Manga Time Kirara Max (まんがタイムきららMAX) is a Japanese four-panel seinen manga magazine published by Houbunsha. It is the third magazine of the \"Kirara\" series, after \"Manga Time Kirara\" and \"Manga Time Kirara Carat\". The first issue was released on September 29, 2004. Currently the magazine is released on the 19th of each month.\u001b[0m\n",
|
||||
"Thought:\n",
|
||||
"===PROMPT====\n",
|
||||
"Answer the following questions as best you can. You have access to the following tools:\n",
|
||||
"\n",
|
||||
"Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, historical events, or other subjects. Input should be a search query.\n",
|
||||
"\n",
|
||||
"Use the following format:\n",
|
||||
"\n",
|
||||
"Question: the input question you must answer\n",
|
||||
"Thought: you should always think about what to do\n",
|
||||
"Action: the action to take, should be one of [Wikipedia]\n",
|
||||
"Action Input: the input to the action\n",
|
||||
"Observation: the result of the action\n",
|
||||
"... (this Thought/Action/Action Input/Observation can repeat N times)\n",
|
||||
"Thought: I now know the final answer\n",
|
||||
"Final Answer: the final answer to the original input question\n",
|
||||
"\n",
|
||||
"Begin!\n",
|
||||
"\n",
|
||||
"Question: What is 'Bocchi the Rock!'?\n",
|
||||
"Thought:I need to use a tool.\n",
|
||||
"Action: Wikipedia\n",
|
||||
"Action Input: Bocchi the Rock!, Japanese four-panel manga and anime series.\n",
|
||||
"Observation: Page: Bocchi the Rock!\n",
|
||||
"Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.\n",
|
||||
"An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.\n",
|
||||
"\n",
|
||||
"Page: Manga Time Kirara\n",
|
||||
"Summary: Manga Time Kirara (まんがタイムきらら, Manga Taimu Kirara) is a Japanese seinen manga magazine published by Houbunsha which mainly serializes four-panel manga. The magazine is sold on the ninth of each month and was first published as a special edition of Manga Time, another Houbunsha magazine, on May 17, 2002. Characters from this magazine have appeared in a crossover role-playing game called Kirara Fantasia.\n",
|
||||
"\n",
|
||||
"Page: Manga Time Kirara Max\n",
|
||||
"Summary: Manga Time Kirara Max (まんがタイムきららMAX) is a Japanese four-panel seinen manga magazine published by Houbunsha. It is the third magazine of the \"Kirara\" series, after \"Manga Time Kirara\" and \"Manga Time Kirara Carat\". The first issue was released on September 29, 2004. Currently the magazine is released on the 19th of each month.\n",
|
||||
"Thought:\n",
|
||||
"=====END OF PROMPT======\n",
|
||||
"\u001b[32;1m\u001b[1;3mThese are not relevant articles.\n",
|
||||
"Action: Wikipedia\n",
|
||||
"Action Input: Bocchi the Rock!, Japanese four-panel manga series written and illustrated by Aki Hamaji.\u001b[0m\n",
|
||||
"Observation: \u001b[36;1m\u001b[1;3mPage: Bocchi the Rock!\n",
|
||||
"Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.\n",
|
||||
"An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.\u001b[0m\n",
|
||||
"Thought:\n",
|
||||
"===PROMPT====\n",
|
||||
"Answer the following questions as best you can. You have access to the following tools:\n",
|
||||
"\n",
|
||||
"Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, historical events, or other subjects. Input should be a search query.\n",
|
||||
"\n",
|
||||
"Use the following format:\n",
|
||||
"\n",
|
||||
"Question: the input question you must answer\n",
|
||||
"Thought: you should always think about what to do\n",
|
||||
"Action: the action to take, should be one of [Wikipedia]\n",
|
||||
"Action Input: the input to the action\n",
|
||||
"Observation: the result of the action\n",
|
||||
"... (this Thought/Action/Action Input/Observation can repeat N times)\n",
|
||||
"Thought: I now know the final answer\n",
|
||||
"Final Answer: the final answer to the original input question\n",
|
||||
"\n",
|
||||
"Begin!\n",
|
||||
"\n",
|
||||
"Question: What is 'Bocchi the Rock!'?\n",
|
||||
"Thought:I need to use a tool.\n",
|
||||
"Action: Wikipedia\n",
|
||||
"Action Input: Bocchi the Rock!, Japanese four-panel manga and anime series.\n",
|
||||
"Observation: Page: Bocchi the Rock!\n",
|
||||
"Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.\n",
|
||||
"An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.\n",
|
||||
"\n",
|
||||
"Page: Manga Time Kirara\n",
|
||||
"Summary: Manga Time Kirara (まんがタイムきらら, Manga Taimu Kirara) is a Japanese seinen manga magazine published by Houbunsha which mainly serializes four-panel manga. The magazine is sold on the ninth of each month and was first published as a special edition of Manga Time, another Houbunsha magazine, on May 17, 2002. Characters from this magazine have appeared in a crossover role-playing game called Kirara Fantasia.\n",
|
||||
"\n",
|
||||
"Page: Manga Time Kirara Max\n",
|
||||
"Summary: Manga Time Kirara Max (まんがタイムきららMAX) is a Japanese four-panel seinen manga magazine published by Houbunsha. It is the third magazine of the \"Kirara\" series, after \"Manga Time Kirara\" and \"Manga Time Kirara Carat\". The first issue was released on September 29, 2004. Currently the magazine is released on the 19th of each month.\n",
|
||||
"Thought:These are not relevant articles.\n",
|
||||
"Action: Wikipedia\n",
|
||||
"Action Input: Bocchi the Rock!, Japanese four-panel manga series written and illustrated by Aki Hamaji.\n",
|
||||
"Observation: Page: Bocchi the Rock!\n",
|
||||
"Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.\n",
|
||||
"An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.\n",
|
||||
"Thought:\n",
|
||||
"=====END OF PROMPT======\n",
|
||||
"\u001b[32;1m\u001b[1;3mIt worked.\n",
|
||||
"Final Answer: Bocchi the Rock! is a four-panel manga series and anime television series. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.\u001b[0m\n",
|
||||
"\n",
|
||||
"\u001b[1m> Finished chain.\u001b[0m\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"\"Bocchi the Rock! is a four-panel manga series and anime television series. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.\""
|
||||
]
|
||||
},
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"agent.run(\"What is 'Bocchi the Rock!'?\")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": ".venv",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.3"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
"hash": "ab4db1680e5f8d10489fb83454f4ec01729e3bd5bdb28eaf0a13b95ddb6ae5ea"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@ -0,0 +1,84 @@
|
||||
from typing import Any, Callable, List, Mapping, Optional
|
||||
|
||||
from pydantic import Field
|
||||
|
||||
from langchain.callbacks.manager import CallbackManagerForLLMRun
|
||||
from langchain.llms.base import LLM
|
||||
from langchain.llms.utils import enforce_stop_tokens
|
||||
|
||||
|
||||
def _display_prompt(prompt: str) -> None:
|
||||
"""Displays the given prompt to the user."""
|
||||
print(f"\n{prompt}")
|
||||
|
||||
|
||||
def _collect_user_input(
|
||||
separator: Optional[str] = None, stop: Optional[List[str]] = None
|
||||
) -> str:
|
||||
"""Collects and returns user input as a single string."""
|
||||
separator = separator or "\n"
|
||||
lines = []
|
||||
|
||||
while True:
|
||||
line = input()
|
||||
if not line:
|
||||
break
|
||||
lines.append(line)
|
||||
|
||||
if stop and any(seq in line for seq in stop):
|
||||
break
|
||||
# Combine all lines into a single string
|
||||
multi_line_input = separator.join(lines)
|
||||
return multi_line_input
|
||||
|
||||
|
||||
class HumanInputLLM(LLM):
|
||||
"""
|
||||
A LLM wrapper which returns user input as the response.
|
||||
"""
|
||||
|
||||
input_func: Callable = Field(default_factory=lambda: _collect_user_input)
|
||||
prompt_func: Callable[[str], None] = Field(default_factory=lambda: _display_prompt)
|
||||
separator: str = "\n"
|
||||
input_kwargs: Mapping[str, Any] = {}
|
||||
prompt_kwargs: Mapping[str, Any] = {}
|
||||
|
||||
@property
|
||||
def _identifying_params(self) -> Mapping[str, Any]:
|
||||
"""
|
||||
Returns an empty dictionary as there are no identifying parameters.
|
||||
"""
|
||||
return {}
|
||||
|
||||
@property
|
||||
def _llm_type(self) -> str:
|
||||
"""Returns the type of LLM."""
|
||||
return "human-input"
|
||||
|
||||
def _call(
|
||||
self,
|
||||
prompt: str,
|
||||
stop: Optional[List[str]] = None,
|
||||
run_manager: Optional[CallbackManagerForLLMRun] = None,
|
||||
) -> str:
|
||||
"""
|
||||
Displays the prompt to the user and returns their input as a response.
|
||||
|
||||
Args:
|
||||
prompt (str): The prompt to be displayed to the user.
|
||||
stop (Optional[List[str]]): A list of stop strings.
|
||||
run_manager (Optional[CallbackManagerForLLMRun]): Currently not used.
|
||||
|
||||
Returns:
|
||||
str: The user's input as a response.
|
||||
"""
|
||||
self.prompt_func(prompt, **self.prompt_kwargs)
|
||||
user_input = self.input_func(
|
||||
separator=self.separator, stop=stop, **self.input_kwargs
|
||||
)
|
||||
|
||||
if stop is not None:
|
||||
# I believe this is required since the stop tokens
|
||||
# are not enforced by the human themselves
|
||||
user_input = enforce_stop_tokens(user_input, stop)
|
||||
return user_input
|
||||
Loading…
Reference in New Issue