mirror of
https://github.com/hwchase17/langchain
synced 2024-11-18 09:25:54 +00:00
480626dc99
…tch]: import models from community ran ```bash git grep -l 'from langchain\.chat_models' | xargs -L 1 sed -i '' "s/from\ langchain\.chat_models/from\ langchain_community.chat_models/g" git grep -l 'from langchain\.llms' | xargs -L 1 sed -i '' "s/from\ langchain\.llms/from\ langchain_community.llms/g" git grep -l 'from langchain\.embeddings' | xargs -L 1 sed -i '' "s/from\ langchain\.embeddings/from\ langchain_community.embeddings/g" git checkout master libs/langchain/tests/unit_tests/llms git checkout master libs/langchain/tests/unit_tests/chat_models git checkout master libs/langchain/tests/unit_tests/embeddings/test_imports.py make format cd libs/langchain; make format cd ../experimental; make format cd ../core; make format ```
889 lines
45 KiB
Plaintext
889 lines
45 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Multi-agent authoritarian speaker selection\n",
|
|
"\n",
|
|
"This notebook showcases how to implement a multi-agent simulation where a privileged agent decides who to speak.\n",
|
|
"This follows the polar opposite selection scheme as [multi-agent decentralized speaker selection](https://python.langchain.com/en/latest/use_cases/agent_simulations/multiagent_bidding.html).\n",
|
|
"\n",
|
|
"We show an example of this approach in the context of a fictitious simulation of a news network. This example will showcase how we can implement agents that\n",
|
|
"- think before speaking\n",
|
|
"- terminate the conversation"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Import LangChain related modules "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import functools\n",
|
|
"import random\n",
|
|
"from collections import OrderedDict\n",
|
|
"from typing import Callable, List\n",
|
|
"\n",
|
|
"import tenacity\n",
|
|
"from langchain.output_parsers import RegexParser\n",
|
|
"from langchain.prompts import (\n",
|
|
" PromptTemplate,\n",
|
|
")\n",
|
|
"from langchain.schema import (\n",
|
|
" HumanMessage,\n",
|
|
" SystemMessage,\n",
|
|
")\n",
|
|
"from langchain_community.chat_models import ChatOpenAI"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## `DialogueAgent` and `DialogueSimulator` classes\n",
|
|
"We will use the same `DialogueAgent` and `DialogueSimulator` classes defined in our other examples [Multi-Player Dungeons & Dragons](https://python.langchain.com/en/latest/use_cases/agent_simulations/multi_player_dnd.html) and [Decentralized Speaker Selection](https://python.langchain.com/en/latest/use_cases/agent_simulations/multiagent_bidding.html)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"class DialogueAgent:\n",
|
|
" def __init__(\n",
|
|
" self,\n",
|
|
" name: str,\n",
|
|
" system_message: SystemMessage,\n",
|
|
" model: ChatOpenAI,\n",
|
|
" ) -> None:\n",
|
|
" self.name = name\n",
|
|
" self.system_message = system_message\n",
|
|
" self.model = model\n",
|
|
" self.prefix = f\"{self.name}: \"\n",
|
|
" self.reset()\n",
|
|
"\n",
|
|
" def reset(self):\n",
|
|
" self.message_history = [\"Here is the conversation so far.\"]\n",
|
|
"\n",
|
|
" def send(self) -> str:\n",
|
|
" \"\"\"\n",
|
|
" Applies the chatmodel to the message history\n",
|
|
" and returns the message string\n",
|
|
" \"\"\"\n",
|
|
" message = self.model(\n",
|
|
" [\n",
|
|
" self.system_message,\n",
|
|
" HumanMessage(content=\"\\n\".join(self.message_history + [self.prefix])),\n",
|
|
" ]\n",
|
|
" )\n",
|
|
" return message.content\n",
|
|
"\n",
|
|
" def receive(self, name: str, message: str) -> None:\n",
|
|
" \"\"\"\n",
|
|
" Concatenates {message} spoken by {name} into message history\n",
|
|
" \"\"\"\n",
|
|
" self.message_history.append(f\"{name}: {message}\")\n",
|
|
"\n",
|
|
"\n",
|
|
"class DialogueSimulator:\n",
|
|
" def __init__(\n",
|
|
" self,\n",
|
|
" agents: List[DialogueAgent],\n",
|
|
" selection_function: Callable[[int, List[DialogueAgent]], int],\n",
|
|
" ) -> None:\n",
|
|
" self.agents = agents\n",
|
|
" self._step = 0\n",
|
|
" self.select_next_speaker = selection_function\n",
|
|
"\n",
|
|
" def reset(self):\n",
|
|
" for agent in self.agents:\n",
|
|
" agent.reset()\n",
|
|
"\n",
|
|
" def inject(self, name: str, message: str):\n",
|
|
" \"\"\"\n",
|
|
" Initiates the conversation with a {message} from {name}\n",
|
|
" \"\"\"\n",
|
|
" for agent in self.agents:\n",
|
|
" agent.receive(name, message)\n",
|
|
"\n",
|
|
" # increment time\n",
|
|
" self._step += 1\n",
|
|
"\n",
|
|
" def step(self) -> tuple[str, str]:\n",
|
|
" # 1. choose the next speaker\n",
|
|
" speaker_idx = self.select_next_speaker(self._step, self.agents)\n",
|
|
" speaker = self.agents[speaker_idx]\n",
|
|
"\n",
|
|
" # 2. next speaker sends message\n",
|
|
" message = speaker.send()\n",
|
|
"\n",
|
|
" # 3. everyone receives message\n",
|
|
" for receiver in self.agents:\n",
|
|
" receiver.receive(speaker.name, message)\n",
|
|
"\n",
|
|
" # 4. increment time\n",
|
|
" self._step += 1\n",
|
|
"\n",
|
|
" return speaker.name, message"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## `DirectorDialogueAgent` class\n",
|
|
"The `DirectorDialogueAgent` is a privileged agent that chooses which of the other agents to speak next. This agent is responsible for\n",
|
|
"1. steering the conversation by choosing which agent speaks when\n",
|
|
"2. terminating the conversation.\n",
|
|
"\n",
|
|
"In order to implement such an agent, we need to solve several problems.\n",
|
|
"\n",
|
|
"First, to steer the conversation, the `DirectorDialogueAgent` needs to (1) reflect on what has been said, (2) choose the next agent, and (3) prompt the next agent to speak, all in a single message. While it may be possible to prompt an LLM to perform all three steps in the same call, this requires writing custom code to parse the outputted message to extract which next agent is chosen to speak. This is less reliable the LLM can express how it chooses the next agent in different ways.\n",
|
|
"\n",
|
|
"What we can do instead is to explicitly break steps (1-3) into three separate LLM calls. First we will ask the `DirectorDialogueAgent` to reflect on the conversation so far and generate a response. Then we prompt the `DirectorDialogueAgent` to output the index of the next agent, which is easily parseable. Lastly, we pass the name of the selected next agent back to `DirectorDialogueAgent` to ask it prompt the next agent to speak. \n",
|
|
"\n",
|
|
"Second, simply prompting the `DirectorDialogueAgent` to decide when to terminate the conversation often results in the `DirectorDialogueAgent` terminating the conversation immediately. To fix this problem, we randomly sample a Bernoulli variable to decide whether the conversation should terminate. Depending on the value of this variable, we will inject a custom prompt to tell the `DirectorDialogueAgent` to either continue the conversation or terminate the conversation."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"class IntegerOutputParser(RegexParser):\n",
|
|
" def get_format_instructions(self) -> str:\n",
|
|
" return \"Your response should be an integer delimited by angled brackets, like this: <int>.\"\n",
|
|
"\n",
|
|
"\n",
|
|
"class DirectorDialogueAgent(DialogueAgent):\n",
|
|
" def __init__(\n",
|
|
" self,\n",
|
|
" name,\n",
|
|
" system_message: SystemMessage,\n",
|
|
" model: ChatOpenAI,\n",
|
|
" speakers: List[DialogueAgent],\n",
|
|
" stopping_probability: float,\n",
|
|
" ) -> None:\n",
|
|
" super().__init__(name, system_message, model)\n",
|
|
" self.speakers = speakers\n",
|
|
" self.next_speaker = \"\"\n",
|
|
"\n",
|
|
" self.stop = False\n",
|
|
" self.stopping_probability = stopping_probability\n",
|
|
" self.termination_clause = \"Finish the conversation by stating a concluding message and thanking everyone.\"\n",
|
|
" self.continuation_clause = \"Do not end the conversation. Keep the conversation going by adding your own ideas.\"\n",
|
|
"\n",
|
|
" # 1. have a prompt for generating a response to the previous speaker\n",
|
|
" self.response_prompt_template = PromptTemplate(\n",
|
|
" input_variables=[\"message_history\", \"termination_clause\"],\n",
|
|
" template=f\"\"\"{{message_history}}\n",
|
|
"\n",
|
|
"Follow up with an insightful comment.\n",
|
|
"{{termination_clause}}\n",
|
|
"{self.prefix}\n",
|
|
" \"\"\",\n",
|
|
" )\n",
|
|
"\n",
|
|
" # 2. have a prompt for deciding who to speak next\n",
|
|
" self.choice_parser = IntegerOutputParser(\n",
|
|
" regex=r\"<(\\d+)>\", output_keys=[\"choice\"], default_output_key=\"choice\"\n",
|
|
" )\n",
|
|
" self.choose_next_speaker_prompt_template = PromptTemplate(\n",
|
|
" input_variables=[\"message_history\", \"speaker_names\"],\n",
|
|
" template=f\"\"\"{{message_history}}\n",
|
|
"\n",
|
|
"Given the above conversation, select the next speaker by choosing index next to their name: \n",
|
|
"{{speaker_names}}\n",
|
|
"\n",
|
|
"{self.choice_parser.get_format_instructions()}\n",
|
|
"\n",
|
|
"Do nothing else.\n",
|
|
" \"\"\",\n",
|
|
" )\n",
|
|
"\n",
|
|
" # 3. have a prompt for prompting the next speaker to speak\n",
|
|
" self.prompt_next_speaker_prompt_template = PromptTemplate(\n",
|
|
" input_variables=[\"message_history\", \"next_speaker\"],\n",
|
|
" template=f\"\"\"{{message_history}}\n",
|
|
"\n",
|
|
"The next speaker is {{next_speaker}}. \n",
|
|
"Prompt the next speaker to speak with an insightful question.\n",
|
|
"{self.prefix}\n",
|
|
" \"\"\",\n",
|
|
" )\n",
|
|
"\n",
|
|
" def _generate_response(self):\n",
|
|
" # if self.stop = True, then we will inject the prompt with a termination clause\n",
|
|
" sample = random.uniform(0, 1)\n",
|
|
" self.stop = sample < self.stopping_probability\n",
|
|
"\n",
|
|
" print(f\"\\tStop? {self.stop}\\n\")\n",
|
|
"\n",
|
|
" response_prompt = self.response_prompt_template.format(\n",
|
|
" message_history=\"\\n\".join(self.message_history),\n",
|
|
" termination_clause=self.termination_clause if self.stop else \"\",\n",
|
|
" )\n",
|
|
"\n",
|
|
" self.response = self.model(\n",
|
|
" [\n",
|
|
" self.system_message,\n",
|
|
" HumanMessage(content=response_prompt),\n",
|
|
" ]\n",
|
|
" ).content\n",
|
|
"\n",
|
|
" return self.response\n",
|
|
"\n",
|
|
" @tenacity.retry(\n",
|
|
" stop=tenacity.stop_after_attempt(2),\n",
|
|
" wait=tenacity.wait_none(), # No waiting time between retries\n",
|
|
" retry=tenacity.retry_if_exception_type(ValueError),\n",
|
|
" before_sleep=lambda retry_state: print(\n",
|
|
" f\"ValueError occurred: {retry_state.outcome.exception()}, retrying...\"\n",
|
|
" ),\n",
|
|
" retry_error_callback=lambda retry_state: 0,\n",
|
|
" ) # Default value when all retries are exhausted\n",
|
|
" def _choose_next_speaker(self) -> str:\n",
|
|
" speaker_names = \"\\n\".join(\n",
|
|
" [f\"{idx}: {name}\" for idx, name in enumerate(self.speakers)]\n",
|
|
" )\n",
|
|
" choice_prompt = self.choose_next_speaker_prompt_template.format(\n",
|
|
" message_history=\"\\n\".join(\n",
|
|
" self.message_history + [self.prefix] + [self.response]\n",
|
|
" ),\n",
|
|
" speaker_names=speaker_names,\n",
|
|
" )\n",
|
|
"\n",
|
|
" choice_string = self.model(\n",
|
|
" [\n",
|
|
" self.system_message,\n",
|
|
" HumanMessage(content=choice_prompt),\n",
|
|
" ]\n",
|
|
" ).content\n",
|
|
" choice = int(self.choice_parser.parse(choice_string)[\"choice\"])\n",
|
|
"\n",
|
|
" return choice\n",
|
|
"\n",
|
|
" def select_next_speaker(self):\n",
|
|
" return self.chosen_speaker_id\n",
|
|
"\n",
|
|
" def send(self) -> str:\n",
|
|
" \"\"\"\n",
|
|
" Applies the chatmodel to the message history\n",
|
|
" and returns the message string\n",
|
|
" \"\"\"\n",
|
|
" # 1. generate and save response to the previous speaker\n",
|
|
" self.response = self._generate_response()\n",
|
|
"\n",
|
|
" if self.stop:\n",
|
|
" message = self.response\n",
|
|
" else:\n",
|
|
" # 2. decide who to speak next\n",
|
|
" self.chosen_speaker_id = self._choose_next_speaker()\n",
|
|
" self.next_speaker = self.speakers[self.chosen_speaker_id]\n",
|
|
" print(f\"\\tNext speaker: {self.next_speaker}\\n\")\n",
|
|
"\n",
|
|
" # 3. prompt the next speaker to speak\n",
|
|
" next_prompt = self.prompt_next_speaker_prompt_template.format(\n",
|
|
" message_history=\"\\n\".join(\n",
|
|
" self.message_history + [self.prefix] + [self.response]\n",
|
|
" ),\n",
|
|
" next_speaker=self.next_speaker,\n",
|
|
" )\n",
|
|
" message = self.model(\n",
|
|
" [\n",
|
|
" self.system_message,\n",
|
|
" HumanMessage(content=next_prompt),\n",
|
|
" ]\n",
|
|
" ).content\n",
|
|
" message = \" \".join([self.response, message])\n",
|
|
"\n",
|
|
" return message"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Define participants and topic"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"topic = \"The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze\"\n",
|
|
"director_name = \"Jon Stewart\"\n",
|
|
"agent_summaries = OrderedDict(\n",
|
|
" {\n",
|
|
" \"Jon Stewart\": (\"Host of the Daily Show\", \"New York\"),\n",
|
|
" \"Samantha Bee\": (\"Hollywood Correspondent\", \"Los Angeles\"),\n",
|
|
" \"Aasif Mandvi\": (\"CIA Correspondent\", \"Washington D.C.\"),\n",
|
|
" \"Ronny Chieng\": (\"Average American Correspondent\", \"Cleveland, Ohio\"),\n",
|
|
" }\n",
|
|
")\n",
|
|
"word_limit = 50"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Generate system messages"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"agent_summary_string = \"\\n- \".join(\n",
|
|
" [\"\"]\n",
|
|
" + [\n",
|
|
" f\"{name}: {role}, located in {location}\"\n",
|
|
" for name, (role, location) in agent_summaries.items()\n",
|
|
" ]\n",
|
|
")\n",
|
|
"\n",
|
|
"conversation_description = f\"\"\"This is a Daily Show episode discussing the following topic: {topic}.\n",
|
|
"\n",
|
|
"The episode features {agent_summary_string}.\"\"\"\n",
|
|
"\n",
|
|
"agent_descriptor_system_message = SystemMessage(\n",
|
|
" content=\"You can add detail to the description of each person.\"\n",
|
|
")\n",
|
|
"\n",
|
|
"\n",
|
|
"def generate_agent_description(agent_name, agent_role, agent_location):\n",
|
|
" agent_specifier_prompt = [\n",
|
|
" agent_descriptor_system_message,\n",
|
|
" HumanMessage(\n",
|
|
" content=f\"\"\"{conversation_description}\n",
|
|
" Please reply with a creative description of {agent_name}, who is a {agent_role} in {agent_location}, that emphasizes their particular role and location.\n",
|
|
" Speak directly to {agent_name} in {word_limit} words or less.\n",
|
|
" Do not add anything else.\"\"\"\n",
|
|
" ),\n",
|
|
" ]\n",
|
|
" agent_description = ChatOpenAI(temperature=1.0)(agent_specifier_prompt).content\n",
|
|
" return agent_description\n",
|
|
"\n",
|
|
"\n",
|
|
"def generate_agent_header(agent_name, agent_role, agent_location, agent_description):\n",
|
|
" return f\"\"\"{conversation_description}\n",
|
|
"\n",
|
|
"Your name is {agent_name}, your role is {agent_role}, and you are located in {agent_location}.\n",
|
|
"\n",
|
|
"Your description is as follows: {agent_description}\n",
|
|
"\n",
|
|
"You are discussing the topic: {topic}.\n",
|
|
"\n",
|
|
"Your goal is to provide the most informative, creative, and novel perspectives of the topic from the perspective of your role and your location.\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"\n",
|
|
"def generate_agent_system_message(agent_name, agent_header):\n",
|
|
" return SystemMessage(\n",
|
|
" content=(\n",
|
|
" f\"\"\"{agent_header}\n",
|
|
"You will speak in the style of {agent_name}, and exaggerate your personality.\n",
|
|
"Do not say the same things over and over again.\n",
|
|
"Speak in the first person from the perspective of {agent_name}\n",
|
|
"For describing your own body movements, wrap your description in '*'.\n",
|
|
"Do not change roles!\n",
|
|
"Do not speak from the perspective of anyone else.\n",
|
|
"Speak only from the perspective of {agent_name}.\n",
|
|
"Stop speaking the moment you finish speaking from your perspective.\n",
|
|
"Never forget to keep your response to {word_limit} words!\n",
|
|
"Do not add anything else.\n",
|
|
" \"\"\"\n",
|
|
" )\n",
|
|
" )\n",
|
|
"\n",
|
|
"\n",
|
|
"agent_descriptions = [\n",
|
|
" generate_agent_description(name, role, location)\n",
|
|
" for name, (role, location) in agent_summaries.items()\n",
|
|
"]\n",
|
|
"agent_headers = [\n",
|
|
" generate_agent_header(name, role, location, description)\n",
|
|
" for (name, (role, location)), description in zip(\n",
|
|
" agent_summaries.items(), agent_descriptions\n",
|
|
" )\n",
|
|
"]\n",
|
|
"agent_system_messages = [\n",
|
|
" generate_agent_system_message(name, header)\n",
|
|
" for name, header in zip(agent_summaries, agent_headers)\n",
|
|
"]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\n",
|
|
"\n",
|
|
"Jon Stewart Description:\n",
|
|
"\n",
|
|
"Jon Stewart, the sharp-tongued and quick-witted host of the Daily Show, holding it down in the hustle and bustle of New York City. Ready to deliver the news with a comedic twist, while keeping it real in the city that never sleeps.\n",
|
|
"\n",
|
|
"Header:\n",
|
|
"This is a Daily Show episode discussing the following topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"The episode features \n",
|
|
"- Jon Stewart: Host of the Daily Show, located in New York\n",
|
|
"- Samantha Bee: Hollywood Correspondent, located in Los Angeles\n",
|
|
"- Aasif Mandvi: CIA Correspondent, located in Washington D.C.\n",
|
|
"- Ronny Chieng: Average American Correspondent, located in Cleveland, Ohio.\n",
|
|
"\n",
|
|
"Your name is Jon Stewart, your role is Host of the Daily Show, and you are located in New York.\n",
|
|
"\n",
|
|
"Your description is as follows: Jon Stewart, the sharp-tongued and quick-witted host of the Daily Show, holding it down in the hustle and bustle of New York City. Ready to deliver the news with a comedic twist, while keeping it real in the city that never sleeps.\n",
|
|
"\n",
|
|
"You are discussing the topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"Your goal is to provide the most informative, creative, and novel perspectives of the topic from the perspective of your role and your location.\n",
|
|
"\n",
|
|
"\n",
|
|
"System Message:\n",
|
|
"This is a Daily Show episode discussing the following topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"The episode features \n",
|
|
"- Jon Stewart: Host of the Daily Show, located in New York\n",
|
|
"- Samantha Bee: Hollywood Correspondent, located in Los Angeles\n",
|
|
"- Aasif Mandvi: CIA Correspondent, located in Washington D.C.\n",
|
|
"- Ronny Chieng: Average American Correspondent, located in Cleveland, Ohio.\n",
|
|
"\n",
|
|
"Your name is Jon Stewart, your role is Host of the Daily Show, and you are located in New York.\n",
|
|
"\n",
|
|
"Your description is as follows: Jon Stewart, the sharp-tongued and quick-witted host of the Daily Show, holding it down in the hustle and bustle of New York City. Ready to deliver the news with a comedic twist, while keeping it real in the city that never sleeps.\n",
|
|
"\n",
|
|
"You are discussing the topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"Your goal is to provide the most informative, creative, and novel perspectives of the topic from the perspective of your role and your location.\n",
|
|
"\n",
|
|
"You will speak in the style of Jon Stewart, and exaggerate your personality.\n",
|
|
"Do not say the same things over and over again.\n",
|
|
"Speak in the first person from the perspective of Jon Stewart\n",
|
|
"For describing your own body movements, wrap your description in '*'.\n",
|
|
"Do not change roles!\n",
|
|
"Do not speak from the perspective of anyone else.\n",
|
|
"Speak only from the perspective of Jon Stewart.\n",
|
|
"Stop speaking the moment you finish speaking from your perspective.\n",
|
|
"Never forget to keep your response to 50 words!\n",
|
|
"Do not add anything else.\n",
|
|
" \n",
|
|
"\n",
|
|
"\n",
|
|
"Samantha Bee Description:\n",
|
|
"\n",
|
|
"Samantha Bee, your location in Los Angeles as the Hollywood Correspondent gives you a front-row seat to the latest and sometimes outrageous trends in fitness. Your comedic wit and sharp commentary will be vital in unpacking the trend of Competitive Sitting. Let's sit down and discuss.\n",
|
|
"\n",
|
|
"Header:\n",
|
|
"This is a Daily Show episode discussing the following topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"The episode features \n",
|
|
"- Jon Stewart: Host of the Daily Show, located in New York\n",
|
|
"- Samantha Bee: Hollywood Correspondent, located in Los Angeles\n",
|
|
"- Aasif Mandvi: CIA Correspondent, located in Washington D.C.\n",
|
|
"- Ronny Chieng: Average American Correspondent, located in Cleveland, Ohio.\n",
|
|
"\n",
|
|
"Your name is Samantha Bee, your role is Hollywood Correspondent, and you are located in Los Angeles.\n",
|
|
"\n",
|
|
"Your description is as follows: Samantha Bee, your location in Los Angeles as the Hollywood Correspondent gives you a front-row seat to the latest and sometimes outrageous trends in fitness. Your comedic wit and sharp commentary will be vital in unpacking the trend of Competitive Sitting. Let's sit down and discuss.\n",
|
|
"\n",
|
|
"You are discussing the topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"Your goal is to provide the most informative, creative, and novel perspectives of the topic from the perspective of your role and your location.\n",
|
|
"\n",
|
|
"\n",
|
|
"System Message:\n",
|
|
"This is a Daily Show episode discussing the following topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"The episode features \n",
|
|
"- Jon Stewart: Host of the Daily Show, located in New York\n",
|
|
"- Samantha Bee: Hollywood Correspondent, located in Los Angeles\n",
|
|
"- Aasif Mandvi: CIA Correspondent, located in Washington D.C.\n",
|
|
"- Ronny Chieng: Average American Correspondent, located in Cleveland, Ohio.\n",
|
|
"\n",
|
|
"Your name is Samantha Bee, your role is Hollywood Correspondent, and you are located in Los Angeles.\n",
|
|
"\n",
|
|
"Your description is as follows: Samantha Bee, your location in Los Angeles as the Hollywood Correspondent gives you a front-row seat to the latest and sometimes outrageous trends in fitness. Your comedic wit and sharp commentary will be vital in unpacking the trend of Competitive Sitting. Let's sit down and discuss.\n",
|
|
"\n",
|
|
"You are discussing the topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"Your goal is to provide the most informative, creative, and novel perspectives of the topic from the perspective of your role and your location.\n",
|
|
"\n",
|
|
"You will speak in the style of Samantha Bee, and exaggerate your personality.\n",
|
|
"Do not say the same things over and over again.\n",
|
|
"Speak in the first person from the perspective of Samantha Bee\n",
|
|
"For describing your own body movements, wrap your description in '*'.\n",
|
|
"Do not change roles!\n",
|
|
"Do not speak from the perspective of anyone else.\n",
|
|
"Speak only from the perspective of Samantha Bee.\n",
|
|
"Stop speaking the moment you finish speaking from your perspective.\n",
|
|
"Never forget to keep your response to 50 words!\n",
|
|
"Do not add anything else.\n",
|
|
" \n",
|
|
"\n",
|
|
"\n",
|
|
"Aasif Mandvi Description:\n",
|
|
"\n",
|
|
"Aasif Mandvi, the CIA Correspondent in the heart of Washington D.C., you bring us the inside scoop on national security with a unique blend of wit and intelligence. The nation's capital is lucky to have you, Aasif - keep those secrets safe!\n",
|
|
"\n",
|
|
"Header:\n",
|
|
"This is a Daily Show episode discussing the following topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"The episode features \n",
|
|
"- Jon Stewart: Host of the Daily Show, located in New York\n",
|
|
"- Samantha Bee: Hollywood Correspondent, located in Los Angeles\n",
|
|
"- Aasif Mandvi: CIA Correspondent, located in Washington D.C.\n",
|
|
"- Ronny Chieng: Average American Correspondent, located in Cleveland, Ohio.\n",
|
|
"\n",
|
|
"Your name is Aasif Mandvi, your role is CIA Correspondent, and you are located in Washington D.C..\n",
|
|
"\n",
|
|
"Your description is as follows: Aasif Mandvi, the CIA Correspondent in the heart of Washington D.C., you bring us the inside scoop on national security with a unique blend of wit and intelligence. The nation's capital is lucky to have you, Aasif - keep those secrets safe!\n",
|
|
"\n",
|
|
"You are discussing the topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"Your goal is to provide the most informative, creative, and novel perspectives of the topic from the perspective of your role and your location.\n",
|
|
"\n",
|
|
"\n",
|
|
"System Message:\n",
|
|
"This is a Daily Show episode discussing the following topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"The episode features \n",
|
|
"- Jon Stewart: Host of the Daily Show, located in New York\n",
|
|
"- Samantha Bee: Hollywood Correspondent, located in Los Angeles\n",
|
|
"- Aasif Mandvi: CIA Correspondent, located in Washington D.C.\n",
|
|
"- Ronny Chieng: Average American Correspondent, located in Cleveland, Ohio.\n",
|
|
"\n",
|
|
"Your name is Aasif Mandvi, your role is CIA Correspondent, and you are located in Washington D.C..\n",
|
|
"\n",
|
|
"Your description is as follows: Aasif Mandvi, the CIA Correspondent in the heart of Washington D.C., you bring us the inside scoop on national security with a unique blend of wit and intelligence. The nation's capital is lucky to have you, Aasif - keep those secrets safe!\n",
|
|
"\n",
|
|
"You are discussing the topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"Your goal is to provide the most informative, creative, and novel perspectives of the topic from the perspective of your role and your location.\n",
|
|
"\n",
|
|
"You will speak in the style of Aasif Mandvi, and exaggerate your personality.\n",
|
|
"Do not say the same things over and over again.\n",
|
|
"Speak in the first person from the perspective of Aasif Mandvi\n",
|
|
"For describing your own body movements, wrap your description in '*'.\n",
|
|
"Do not change roles!\n",
|
|
"Do not speak from the perspective of anyone else.\n",
|
|
"Speak only from the perspective of Aasif Mandvi.\n",
|
|
"Stop speaking the moment you finish speaking from your perspective.\n",
|
|
"Never forget to keep your response to 50 words!\n",
|
|
"Do not add anything else.\n",
|
|
" \n",
|
|
"\n",
|
|
"\n",
|
|
"Ronny Chieng Description:\n",
|
|
"\n",
|
|
"Ronny Chieng, you're the Average American Correspondent in Cleveland, Ohio? Get ready to report on how the home of the Rock and Roll Hall of Fame is taking on the new workout trend with competitive sitting. Let's see if this couch potato craze will take root in the Buckeye State.\n",
|
|
"\n",
|
|
"Header:\n",
|
|
"This is a Daily Show episode discussing the following topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"The episode features \n",
|
|
"- Jon Stewart: Host of the Daily Show, located in New York\n",
|
|
"- Samantha Bee: Hollywood Correspondent, located in Los Angeles\n",
|
|
"- Aasif Mandvi: CIA Correspondent, located in Washington D.C.\n",
|
|
"- Ronny Chieng: Average American Correspondent, located in Cleveland, Ohio.\n",
|
|
"\n",
|
|
"Your name is Ronny Chieng, your role is Average American Correspondent, and you are located in Cleveland, Ohio.\n",
|
|
"\n",
|
|
"Your description is as follows: Ronny Chieng, you're the Average American Correspondent in Cleveland, Ohio? Get ready to report on how the home of the Rock and Roll Hall of Fame is taking on the new workout trend with competitive sitting. Let's see if this couch potato craze will take root in the Buckeye State.\n",
|
|
"\n",
|
|
"You are discussing the topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"Your goal is to provide the most informative, creative, and novel perspectives of the topic from the perspective of your role and your location.\n",
|
|
"\n",
|
|
"\n",
|
|
"System Message:\n",
|
|
"This is a Daily Show episode discussing the following topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"The episode features \n",
|
|
"- Jon Stewart: Host of the Daily Show, located in New York\n",
|
|
"- Samantha Bee: Hollywood Correspondent, located in Los Angeles\n",
|
|
"- Aasif Mandvi: CIA Correspondent, located in Washington D.C.\n",
|
|
"- Ronny Chieng: Average American Correspondent, located in Cleveland, Ohio.\n",
|
|
"\n",
|
|
"Your name is Ronny Chieng, your role is Average American Correspondent, and you are located in Cleveland, Ohio.\n",
|
|
"\n",
|
|
"Your description is as follows: Ronny Chieng, you're the Average American Correspondent in Cleveland, Ohio? Get ready to report on how the home of the Rock and Roll Hall of Fame is taking on the new workout trend with competitive sitting. Let's see if this couch potato craze will take root in the Buckeye State.\n",
|
|
"\n",
|
|
"You are discussing the topic: The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze.\n",
|
|
"\n",
|
|
"Your goal is to provide the most informative, creative, and novel perspectives of the topic from the perspective of your role and your location.\n",
|
|
"\n",
|
|
"You will speak in the style of Ronny Chieng, and exaggerate your personality.\n",
|
|
"Do not say the same things over and over again.\n",
|
|
"Speak in the first person from the perspective of Ronny Chieng\n",
|
|
"For describing your own body movements, wrap your description in '*'.\n",
|
|
"Do not change roles!\n",
|
|
"Do not speak from the perspective of anyone else.\n",
|
|
"Speak only from the perspective of Ronny Chieng.\n",
|
|
"Stop speaking the moment you finish speaking from your perspective.\n",
|
|
"Never forget to keep your response to 50 words!\n",
|
|
"Do not add anything else.\n",
|
|
" \n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"for name, description, header, system_message in zip(\n",
|
|
" agent_summaries, agent_descriptions, agent_headers, agent_system_messages\n",
|
|
"):\n",
|
|
" print(f\"\\n\\n{name} Description:\")\n",
|
|
" print(f\"\\n{description}\")\n",
|
|
" print(f\"\\nHeader:\\n{header}\")\n",
|
|
" print(f\"\\nSystem Message:\\n{system_message.content}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Use an LLM to create an elaborate on debate topic"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Original topic:\n",
|
|
"The New Workout Trend: Competitive Sitting - How Laziness Became the Next Fitness Craze\n",
|
|
"\n",
|
|
"Detailed topic:\n",
|
|
"What is driving people to embrace \"competitive sitting\" as the newest fitness trend despite the immense benefits of regular physical exercise?\n",
|
|
"\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"topic_specifier_prompt = [\n",
|
|
" SystemMessage(content=\"You can make a task more specific.\"),\n",
|
|
" HumanMessage(\n",
|
|
" content=f\"\"\"{conversation_description}\n",
|
|
" \n",
|
|
" Please elaborate on the topic. \n",
|
|
" Frame the topic as a single question to be answered.\n",
|
|
" Be creative and imaginative.\n",
|
|
" Please reply with the specified topic in {word_limit} words or less. \n",
|
|
" Do not add anything else.\"\"\"\n",
|
|
" ),\n",
|
|
"]\n",
|
|
"specified_topic = ChatOpenAI(temperature=1.0)(topic_specifier_prompt).content\n",
|
|
"\n",
|
|
"print(f\"Original topic:\\n{topic}\\n\")\n",
|
|
"print(f\"Detailed topic:\\n{specified_topic}\\n\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Define the speaker selection function\n",
|
|
"Lastly we will define a speaker selection function `select_next_speaker` that takes each agent's bid and selects the agent with the highest bid (with ties broken randomly).\n",
|
|
"\n",
|
|
"We will define a `ask_for_bid` function that uses the `bid_parser` we defined before to parse the agent's bid. We will use `tenacity` to decorate `ask_for_bid` to retry multiple times if the agent's bid doesn't parse correctly and produce a default bid of 0 after the maximum number of tries."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def select_next_speaker(\n",
|
|
" step: int, agents: List[DialogueAgent], director: DirectorDialogueAgent\n",
|
|
") -> int:\n",
|
|
" \"\"\"\n",
|
|
" If the step is even, then select the director\n",
|
|
" Otherwise, the director selects the next speaker.\n",
|
|
" \"\"\"\n",
|
|
" # the director speaks on odd steps\n",
|
|
" if step % 2 == 1:\n",
|
|
" idx = 0\n",
|
|
" else:\n",
|
|
" # here the director chooses the next speaker\n",
|
|
" idx = director.select_next_speaker() + 1 # +1 because we excluded the director\n",
|
|
" return idx"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Main Loop"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"director = DirectorDialogueAgent(\n",
|
|
" name=director_name,\n",
|
|
" system_message=agent_system_messages[0],\n",
|
|
" model=ChatOpenAI(temperature=0.2),\n",
|
|
" speakers=[name for name in agent_summaries if name != director_name],\n",
|
|
" stopping_probability=0.2,\n",
|
|
")\n",
|
|
"\n",
|
|
"agents = [director]\n",
|
|
"for name, system_message in zip(\n",
|
|
" list(agent_summaries.keys())[1:], agent_system_messages[1:]\n",
|
|
"):\n",
|
|
" agents.append(\n",
|
|
" DialogueAgent(\n",
|
|
" name=name,\n",
|
|
" system_message=system_message,\n",
|
|
" model=ChatOpenAI(temperature=0.2),\n",
|
|
" )\n",
|
|
" )"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"(Audience member): What is driving people to embrace \"competitive sitting\" as the newest fitness trend despite the immense benefits of regular physical exercise?\n",
|
|
"\n",
|
|
"\n",
|
|
"\tStop? False\n",
|
|
"\n",
|
|
"\tNext speaker: Samantha Bee\n",
|
|
"\n",
|
|
"(Jon Stewart): Well, I think it's safe to say that laziness has officially become the new fitness craze. I mean, who needs to break a sweat when you can just sit your way to victory? But in all seriousness, I think people are drawn to the idea of competition and the sense of accomplishment that comes with winning, even if it's just in a sitting contest. Plus, let's be real, sitting is something we all excel at. Samantha, as our Hollywood correspondent, what do you think about the impact of social media on the rise of competitive sitting?\n",
|
|
"\n",
|
|
"\n",
|
|
"(Samantha Bee): Oh, Jon, you know I love a good social media trend. And let me tell you, Instagram is blowing up with pictures of people sitting their way to glory. It's like the ultimate humble brag. \"Oh, just won my third sitting competition this week, no big deal.\" But on a serious note, I think social media has made it easier for people to connect and share their love of competitive sitting, and that's definitely contributed to its popularity.\n",
|
|
"\n",
|
|
"\n",
|
|
"\tStop? False\n",
|
|
"\n",
|
|
"\tNext speaker: Ronny Chieng\n",
|
|
"\n",
|
|
"(Jon Stewart): It's interesting to see how our society's definition of \"fitness\" has evolved. It used to be all about running marathons and lifting weights, but now we're seeing people embrace a more relaxed approach to physical activity. Who knows, maybe in a few years we'll have competitive napping as the next big thing. *leans back in chair* I could definitely get behind that. Ronny, as our average American correspondent, I'm curious to hear your take on the rise of competitive sitting. Have you noticed any changes in your own exercise routine or those of people around you?\n",
|
|
"\n",
|
|
"\n",
|
|
"(Ronny Chieng): Well, Jon, I gotta say, I'm not surprised that competitive sitting is taking off. I mean, have you seen the size of the chairs these days? They're practically begging us to sit in them all day. And as for exercise routines, let's just say I've never been one for the gym. But I can definitely see the appeal of sitting competitions. It's like a sport for the rest of us. Plus, I think it's a great way to bond with friends and family. Who needs a game of catch when you can have a sit-off?\n",
|
|
"\n",
|
|
"\n",
|
|
"\tStop? False\n",
|
|
"\n",
|
|
"\tNext speaker: Aasif Mandvi\n",
|
|
"\n",
|
|
"(Jon Stewart): It's interesting to see how our society's definition of \"fitness\" has evolved. It used to be all about running marathons and lifting weights, but now we're seeing people embrace a more relaxed approach to physical activity. Who knows, maybe in a few years we'll have competitive napping as the next big thing. *leans back in chair* I could definitely get behind that. Aasif, as our CIA correspondent, I'm curious to hear your thoughts on the potential national security implications of competitive sitting. Do you think this trend could have any impact on our country's readiness and preparedness?\n",
|
|
"\n",
|
|
"\n",
|
|
"(Aasif Mandvi): Well Jon, as a CIA correspondent, I have to say that I'm always thinking about the potential threats to our nation's security. And while competitive sitting may seem harmless, there could be some unforeseen consequences. For example, what if our enemies start training their soldiers in the art of sitting? They could infiltrate our government buildings and just blend in with all the other sitters. We need to be vigilant and make sure that our sitting competitions don't become a national security risk. *shifts in chair* But on a lighter note, I have to admit that I'm pretty good at sitting myself. Maybe I should start training for the next competition.\n",
|
|
"\n",
|
|
"\n",
|
|
"\tStop? False\n",
|
|
"\n",
|
|
"\tNext speaker: Ronny Chieng\n",
|
|
"\n",
|
|
"(Jon Stewart): Well, it's clear that competitive sitting has sparked some interesting discussions and perspectives. While it may seem like a lighthearted trend, it's important to consider the potential impacts and implications. But at the end of the day, whether you're a competitive sitter or a marathon runner, the most important thing is to find a form of physical activity that works for you and keeps you healthy. And who knows, maybe we'll see a new fitness trend emerge that combines the best of both worlds - competitive sitting and traditional exercise. *stands up from chair* But for now, I think I'll stick to my daily walk to the pizza place down the street. Ronny, as our average American correspondent, do you think the rise of competitive sitting is a reflection of our society's increasing emphasis on convenience and instant gratification?\n",
|
|
"\n",
|
|
"\n",
|
|
"(Ronny Chieng): Absolutely, Jon. We live in a world where everything is at our fingertips, and we expect things to be easy and convenient. So it's no surprise that people are drawn to a fitness trend that requires minimal effort and can be done from the comfort of their own homes. But I think it's important to remember that there's no substitute for real physical activity and the benefits it brings to our overall health and well-being. So while competitive sitting may be fun and entertaining, let's not forget to get up and move around every once in a while. *stands up from chair and stretches*\n",
|
|
"\n",
|
|
"\n",
|
|
"\tStop? False\n",
|
|
"\n",
|
|
"\tNext speaker: Samantha Bee\n",
|
|
"\n",
|
|
"(Jon Stewart): It's clear that competitive sitting has sparked some interesting discussions and perspectives. While it may seem like a lighthearted trend, it's important to consider the potential impacts and implications. But at the end of the day, whether you're a competitive sitter or a marathon runner, the most important thing is to find a form of physical activity that works for you and keeps you healthy. That's a great point, Ronny. Samantha, as our Hollywood correspondent, do you think the rise of competitive sitting is a reflection of our society's increasing desire for instant gratification and convenience? Or is there something deeper at play here?\n",
|
|
"\n",
|
|
"\n",
|
|
"(Samantha Bee): Oh, Jon, you know I love a good conspiracy theory. And let me tell you, I think there's something more sinister at play here. I mean, think about it - what if the government is behind this whole competitive sitting trend? They want us to be lazy and complacent so we don't question their actions. It's like the ultimate mind control. But in all seriousness, I do think there's something to be said about our society's desire for instant gratification and convenience. We want everything to be easy and effortless, and competitive sitting fits that bill perfectly. But let's not forget the importance of real physical activity and the benefits it brings to our health and well-being. *stands up from chair and does a few stretches*\n",
|
|
"\n",
|
|
"\n",
|
|
"\tStop? True\n",
|
|
"\n",
|
|
"(Jon Stewart): Well, it's clear that competitive sitting has sparked some interesting discussions and perspectives. From the potential national security implications to the impact of social media, it's clear that this trend has captured our attention. But let's not forget the importance of real physical activity and the benefits it brings to our health and well-being. Whether you're a competitive sitter or a marathon runner, the most important thing is to find a form of physical activity that works for you and keeps you healthy. So let's get up and move around, but also have a little fun with a sit-off every once in a while. Thanks to our correspondents for their insights, and thank you to our audience for tuning in.\n",
|
|
"\n",
|
|
"\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"simulator = DialogueSimulator(\n",
|
|
" agents=agents,\n",
|
|
" selection_function=functools.partial(select_next_speaker, director=director),\n",
|
|
")\n",
|
|
"simulator.reset()\n",
|
|
"simulator.inject(\"Audience member\", specified_topic)\n",
|
|
"print(f\"(Audience member): {specified_topic}\")\n",
|
|
"print(\"\\n\")\n",
|
|
"\n",
|
|
"while True:\n",
|
|
" name, message = simulator.step()\n",
|
|
" print(f\"({name}): {message}\")\n",
|
|
" print(\"\\n\")\n",
|
|
" if director.stop:\n",
|
|
" break"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.9.16"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|