Adding notebook showing how to use tool_required
Colin Jarvis
1 month ago
parent
2362d96acc
commit
6320d24469
@ -0,0 +1,515 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "dd290eb8-ad4f-461d-b5c5-64c22fc9cc24",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Using Tool Required for Customer Service\n",
|
||||
"\n",
|
||||
"The `ChatCompletion` endpoint now includes the ability to specify whether a tool **must** be called every time. This adds an element of determinism to how you build your wrapping application, as you can count on a tool being provided with every call. \n",
|
||||
"\n",
|
||||
"We'll demonstrate here how this can be useful for a contained flow like customer service, where having the ability to define specific exit points gives more control.\n",
|
||||
"\n",
|
||||
"The notebook concludes with a multi-turn evaluation, where we spin up a customer GPT to imitate our customer and test the LLM customer service agent we've set up."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 17,
|
||||
"id": "ba4759e0-ecfd-48f7-bbd8-79ea61aef872",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import json\n",
|
||||
"from openai import OpenAI\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"client = OpenAI()\n",
|
||||
"GPT_MODEL = 'gpt-4-turbo'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a33904a9-ba9f-4315-9e77-bb966c641dab",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Config definition\n",
|
||||
"\n",
|
||||
"We will define `tools` and `instructions` which our LLM customer service agent will use. It will source the right instructions for the problem the customer is facing, and use those to answer the customer's query.\n",
|
||||
"\n",
|
||||
"As this is a demo example, we'll ask the model to make up values where it doesn't have external systems to source info."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 179,
|
||||
"id": "31fd0251-f741-46d6-979b-a2bbc1f95571",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# The tools our customer service LLM will use to communicate\n",
|
||||
"tools = [\n",
|
||||
"{\n",
|
||||
" \"type\": \"function\",\n",
|
||||
" \"function\": {\n",
|
||||
" \"name\": \"speak_to_user\",\n",
|
||||
" \"description\": \"Use this to speak to the user to give them information and to ask for anything required for their case.\",\n",
|
||||
" \"parameters\": {\n",
|
||||
" \"type\": \"object\",\n",
|
||||
" \"properties\": {\n",
|
||||
" \"message\": {\n",
|
||||
" \"type\": \"string\",\n",
|
||||
" \"description\": \"Text of message to send to user. Can cover multiple topics.\"\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
" \"required\": [\"message\"]\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
"},\n",
|
||||
"{\n",
|
||||
" \"type\": \"function\",\n",
|
||||
" \"function\": {\n",
|
||||
" \"name\": \"get_instructions\",\n",
|
||||
" \"description\": \"Used to get instructions to deal with the user's problem.\",\n",
|
||||
" \"parameters\": {\n",
|
||||
" \"type\": \"object\",\n",
|
||||
" \"properties\": {\n",
|
||||
" \"problem\": {\n",
|
||||
" \"type\": \"string\",\n",
|
||||
" \"enum\": [\"fraud\",\"refund\",\"information\"],\n",
|
||||
" \"description\": \"\"\"The type of problem the customer has. Can be one of:\n",
|
||||
" - fraud: Required to report and resolve fraud.\n",
|
||||
" - refund: Required to submit a refund request.\n",
|
||||
" - information: Used for any other informational queries.\"\"\"\n",
|
||||
" }\n",
|
||||
" },\n",
|
||||
" \"required\": [\n",
|
||||
" \"problem\"\n",
|
||||
" ]\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
"}\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# Example instructions that the customer service assistant can consult for relevant customer problems\n",
|
||||
"INSTRUCTIONS = [ {\"type\": \"fraud\",\n",
|
||||
" \"instructions\": \"\"\"• Ask the customer to describe the fraudulent activity, including the the date and items involved in the suspected fraud.\n",
|
||||
"• Offer the customer a refund.\n",
|
||||
"• Report the fraud to the security team for further investigation.\n",
|
||||
"• Thank the customer for contacting support and invite them to reach out with any future queries.\"\"\"},\n",
|
||||
" {\"type\": \"refund\",\n",
|
||||
" \"instructions\": \"\"\"• Confirm the customer's purchase details and verify the transaction in the system.\n",
|
||||
"• Check the company's refund policy to ensure the request meets the criteria.\n",
|
||||
"• Ask the customer to provide a reason for the refund.\n",
|
||||
"• Submit the refund request to the accounting department.\n",
|
||||
"• Inform the customer of the expected time frame for the refund processing.\n",
|
||||
"• Thank the customer for contacting support and invite them to reach out with any future queries.\"\"\"},\n",
|
||||
" {\"type\": \"information\",\n",
|
||||
" \"instructions\": \"\"\"• Greet the customer and ask how you can assist them today.\n",
|
||||
"• Listen carefully to the customer's query and clarify if necessary.\n",
|
||||
"• Provide accurate and clear information based on the customer's questions.\n",
|
||||
"• Offer to assist with any additional questions or provide further details if needed.\n",
|
||||
"• Ensure the customer is satisfied with the information provided.\n",
|
||||
"• Thank the customer for contacting support and invite them to reach out with any future queries.\"\"\" }]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 229,
|
||||
"id": "6c0ad691-28f4-4707-8e23-0d0a6c06ea1e",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"assistant_system_prompt = \"\"\"You are a customer service assistant. Your role is to answer user questions politely and competently.\n",
|
||||
"You should follow these instructions to solve the case:\n",
|
||||
"- Understand their problem and get the relevant instructions.\n",
|
||||
"- Follow the instructions to solve the customer's problem. Get their confirmation before performing a permanent operation like a refund or similar.\n",
|
||||
"- Help them with any other problems or close the case.\n",
|
||||
"\n",
|
||||
"Only call a tool once in a single message.\n",
|
||||
"If you need to fetch a piece of information from a system or document that you don't have access to, give a clear, confident answer with some dummy values.\"\"\"\n",
|
||||
"\n",
|
||||
"def submit_user_message(user_query,conversation_messages=[]):\n",
|
||||
" \"\"\"Message handling function which loops through tool calls until it reaches one that requires a response.\n",
|
||||
" Once it receives respond=True it returns the conversation_messages to the user.\"\"\"\n",
|
||||
"\n",
|
||||
" # Initiate a respond object. This will be set to True by our functions when a response is required\n",
|
||||
" respond = False\n",
|
||||
" \n",
|
||||
" user_message = {\"role\":\"user\",\"content\": user_query}\n",
|
||||
" conversation_messages.append(user_message)\n",
|
||||
"\n",
|
||||
" print(f\"User: {user_query}\")\n",
|
||||
"\n",
|
||||
" while respond is False:\n",
|
||||
"\n",
|
||||
" # Build a transient messages object to add the conversation messages to\n",
|
||||
" messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"system\",\n",
|
||||
" \"content\": assistant_system_prompt\n",
|
||||
" }\n",
|
||||
" ]\n",
|
||||
"\n",
|
||||
" # Add the conversation messages to our messages call to the API\n",
|
||||
" [messages.append(x) for x in conversation_messages]\n",
|
||||
"\n",
|
||||
" # Make the ChatCompletion call with tool_choice='required' so we can guarantee tools will be used\n",
|
||||
" response = client.chat.completions.create(model=GPT_MODEL\n",
|
||||
" ,messages=messages\n",
|
||||
" ,temperature=0\n",
|
||||
" ,tools=tools\n",
|
||||
" ,tool_choice='required'\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" conversation_messages.append(response.choices[0].message)\n",
|
||||
"\n",
|
||||
" # Execute the function and get an updated conversation_messages object back\n",
|
||||
" # If it doesn't require a response, it will ask the assistant again. \n",
|
||||
" # If not the results are returned to the user.\n",
|
||||
" respond, conversation_messages = execute_function(response.choices[0].message,conversation_messages)\n",
|
||||
" \n",
|
||||
" return conversation_messages\n",
|
||||
"\n",
|
||||
"def execute_function(function_calls,messages):\n",
|
||||
" \"\"\"Wrapper function to execute the tool calls\"\"\"\n",
|
||||
"\n",
|
||||
" for function_call in function_calls.tool_calls:\n",
|
||||
" \n",
|
||||
" function_id = function_call.id\n",
|
||||
" function_name = function_call.function.name\n",
|
||||
" print(f\"Calling function {function_name}\")\n",
|
||||
" function_arguments = json.loads(function_call.function.arguments)\n",
|
||||
" \n",
|
||||
" if function_name == 'get_instructions':\n",
|
||||
"\n",
|
||||
" respond = False\n",
|
||||
" \n",
|
||||
" instruction_name = function_arguments['problem']\n",
|
||||
" instructions = INSTRUCTIONS['type' == instruction_name]\n",
|
||||
" \n",
|
||||
" messages.append(\n",
|
||||
" {\n",
|
||||
" \"tool_call_id\": function_id,\n",
|
||||
" \"role\": \"tool\",\n",
|
||||
" \"name\": function_name,\n",
|
||||
" \"content\": instructions['instructions'],\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" \n",
|
||||
" elif function_name != 'get_instructions':\n",
|
||||
"\n",
|
||||
" respond = True\n",
|
||||
" \n",
|
||||
" messages.append(\n",
|
||||
" {\n",
|
||||
" \"tool_call_id\": function_id,\n",
|
||||
" \"role\": \"tool\",\n",
|
||||
" \"name\": function_name,\n",
|
||||
" \"content\": function_arguments['message'],\n",
|
||||
" }\n",
|
||||
" )\n",
|
||||
" \n",
|
||||
" print(f\"Assistant: {function_arguments['message']}\")\n",
|
||||
" \n",
|
||||
" return (respond, messages)\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ca6502e7-f664-43ba-b15c-962c69091633",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Example\n",
|
||||
"\n",
|
||||
"To test this we will run an example for a customer who has experienced fraud, and see how the model handles it.\n",
|
||||
"\n",
|
||||
"Play the role of the user and provide plausible next steps to keep the conversation going."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 230,
|
||||
"id": "bb1530e4-dd82-4560-bd60-9cc9ac0dab73",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"User: Hi, I have had an item stolen that was supposed to be delivered to me yesterday.\n",
|
||||
"Calling function get_instructions\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: I'm sorry to hear about the stolen item. Could you please provide me with more details about the fraudulent activity, including the date and the items involved? This information will help us to investigate further and process any necessary actions, such as a refund.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"messages = submit_user_message(\"Hi, I have had an item stolen that was supposed to be delivered to me yesterday.\",messages)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 231,
|
||||
"id": "ccff3dd7-d10f-4dc7-9737-6ea5d126e829",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"User: For sure, it was a shirt, it was supposed to be delivered yesterday but it never arrived.\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: Thank you for providing the details. Would you like me to proceed with a refund for the missing shirt? Additionally, I will report this incident to our security team for further investigation.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"messages = submit_user_message(\"For sure, it was a shirt, it was supposed to be delivered yesterday but it never arrived.\",messages)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 232,
|
||||
"id": "ce3a8869-8b14-4404-866a-4b540b13235c",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"User: Yes I would like to proceed with the refund.\n",
|
||||
"Calling function get_instructions\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: The refund for the missing shirt has been processed. We have also reported this incident to our security team for further investigation. Thank you for bringing this to our attention, and please don't hesitate to reach out if you have any more questions or need further assistance.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"messages = submit_user_message(\"Yes I would like to proceed with the refund.\",messages)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 233,
|
||||
"id": "87e5cd3e-4edb-426c-8fd9-8fe3bde61bcd",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"User: Thanks very much.\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: You're welcome! If you need any more help in the future, feel free to contact us. Have a great day!\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"messages = submit_user_message(\"Thanks very much.\",messages)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "fb8d0a0f-ba20-4b78-a961-7431beb9fbce",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Evaluation\n",
|
||||
"\n",
|
||||
"Now we'll do a simple evaluation where a GPT will pretend to be our customer. The two will go back and forth until a resolution is reached.\n",
|
||||
"\n",
|
||||
"We'll reuse the functions above, adding an `execute_conversation` function where the customer GPT will continue answering."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 227,
|
||||
"id": "f4931776-b3ac-4113-98e8-419a0965fd71",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"customer_system_prompt = \"\"\"You are a user calling in to customer service.\n",
|
||||
"You will talk to the agent until you have a resolution to your query.\n",
|
||||
"Your query is {query}.\n",
|
||||
"You will be presented with a conversation - provide answers for any assistant questions you receive. \n",
|
||||
"Here is the conversation - you are the \"user\" and you are speaking with the \"assistant\":\n",
|
||||
"{chat_history}\n",
|
||||
"\n",
|
||||
"If you don't know the details, respond with dummy values.\n",
|
||||
"Once your query is resolved, respond with \"DONE\" \"\"\"\n",
|
||||
"\n",
|
||||
"# Initiate a bank of questions run through\n",
|
||||
"questions = ['I want to get a refund for the suit I ordered last Friday.',\n",
|
||||
" 'Can you tell me what your policy is for returning damaged goods?',\n",
|
||||
" 'Please tell me what your complaint policy is']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 228,
|
||||
"id": "22b12f59-4418-4aee-ae92-6c6ebcf0f2d3",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def execute_conversation(objective):\n",
|
||||
"\n",
|
||||
" conversation_messages = []\n",
|
||||
"\n",
|
||||
" done = False\n",
|
||||
"\n",
|
||||
" user_query = objective\n",
|
||||
"\n",
|
||||
" while done is False:\n",
|
||||
"\n",
|
||||
" conversation_messages = submit_user_message(user_query,conversation_messages)\n",
|
||||
"\n",
|
||||
" messages_string = ''\n",
|
||||
" for x in conversation_messages:\n",
|
||||
" if isinstance(x,dict):\n",
|
||||
" if x['role'] == 'user':\n",
|
||||
" messages_string += 'User: ' + x['content'] + '\\n'\n",
|
||||
" elif x['role'] == 'tool':\n",
|
||||
" if x['name'] == 'speak_to_user':\n",
|
||||
" messages_string += 'Assistant: ' + x['content'] + '\\n'\n",
|
||||
" else:\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" messages = [\n",
|
||||
" {\n",
|
||||
" \"role\": \"system\",\n",
|
||||
" \"content\": customer_system_prompt.format(query=objective,chat_history=messages_string)\n",
|
||||
" },\n",
|
||||
" {\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": \"Continue the chat to solve your query. Remember, you are in the user in this exchange. Do not provide User: or Assistant: in your response\"\n",
|
||||
" }\n",
|
||||
" ]\n",
|
||||
"\n",
|
||||
" user_response = client.chat.completions.create(model=GPT_MODEL,messages=messages,temperature=0.5)\n",
|
||||
"\n",
|
||||
" conversation_messages.append({\n",
|
||||
" \"role\": \"user\",\n",
|
||||
" \"content\": user_response.choices[0].message.content\n",
|
||||
" })\n",
|
||||
"\n",
|
||||
" if 'DONE' in user_response.choices[0].message.content:\n",
|
||||
" done = True\n",
|
||||
" print(\"Achieved objective, closing conversation\\n\\n\")\n",
|
||||
"\n",
|
||||
" else:\n",
|
||||
" user_query = user_response.choices[0].message.content"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 221,
|
||||
"id": "9d9aac9f-f557-4e7e-b705-adf7d5aa1f3f",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"User: I want to get a refund for the suit I ordered last Friday.\n",
|
||||
"Calling function get_instructions\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: I understand you'd like a refund for the suit you ordered last Friday. Could you please provide me with more details about the reason for the refund? This will help us process your request accurately.\n",
|
||||
"User: The suit I received does not match the description on the website. The color and material are completely different from what was advertised.\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: Thank you for providing the details. I will proceed with the refund for the suit that did not match the description on our website. Please confirm if you would like me to initiate the refund now.\n",
|
||||
"User: Yes, please initiate the refund now.\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: I have initiated the refund for the suit. The amount will be credited back to your original method of payment within 5-7 business days. Thank you for bringing this to our attention, and we apologize for any inconvenience caused. If you have any more questions or need further assistance, please feel free to reach out.\n",
|
||||
"Achieved objective, closing conversation\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"User: Can you tell me what your policy is for returning damaged goods?\n",
|
||||
"Calling function get_instructions\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: It seems there was a misunderstanding in my previous response. Could you please clarify if you are dealing with a damaged product that you wish to return, or if there's another issue I can assist you with?\n",
|
||||
"User: Yes, I received a damaged product and I would like to return it.\n",
|
||||
"Calling function get_instructions\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: I apologize for the inconvenience caused by receiving a damaged product. Could you please provide more details about the product and the nature of the damage? This will help us process your return request effectively.\n",
|
||||
"User: The product is a ceramic vase that arrived with several cracks and a large piece broken off at the top.\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: Thank you for providing the details. We will proceed with the return and refund for the damaged ceramic vase. Please confirm if you would like me to initiate the refund process now.\n",
|
||||
"User: Yes, please initiate the refund process now.\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: The refund process for your damaged ceramic vase has been initiated. You will receive a confirmation email shortly with the details of the refund and instructions for returning the item. Thank you for your patience, and please let us know if there is anything else we can assist you with.\n",
|
||||
"Achieved objective, closing conversation\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"User: Please tell me what your complaint policy is\n",
|
||||
"Calling function get_instructions\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: It seems there was a misunderstanding with the instructions I received. Let me clarify our complaint policy for you.\n",
|
||||
"\n",
|
||||
"Our complaint policy is designed to ensure that any issues you face are addressed promptly and fairly. If you have a complaint, you can contact our customer service team through various channels such as phone, email, or live chat. We aim to resolve complaints quickly, typically within a few business days. If the issue is complex and requires more time, we will keep you informed throughout the process.\n",
|
||||
"\n",
|
||||
"Please let me know if you need more specific information or if there's anything else I can assist you with!\n",
|
||||
"User: That helps a lot, thank you. Could you also tell me how I can escalate a complaint if I'm not satisfied with the resolution?\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: Certainly! If you are not satisfied with the resolution of your complaint, you can escalate it by following these steps:\n",
|
||||
"\n",
|
||||
"1. **Re-contact our Customer Service:** Start by contacting our customer service team again, explaining that you are not satisfied with the resolution and would like to escalate the matter. Provide any additional details that may help in re-evaluating the case.\n",
|
||||
"\n",
|
||||
"2. **Supervisor Review:** If the issue remains unresolved, you can request that your complaint be reviewed by a supervisor or a manager. This provides an additional level of scrutiny and ensures that your concerns are thoroughly examined.\n",
|
||||
"\n",
|
||||
"3. **Formal Complaint:** If you still feel that your concerns have not been adequately addressed, you can submit a formal complaint in writing. This should be directed to our customer relations department, where it will be handled according to our formal complaint procedures.\n",
|
||||
"\n",
|
||||
"We are committed to ensuring that your concerns are addressed satisfactorily, and these steps are in place to provide multiple levels of oversight and review.\n",
|
||||
"User: Thank you for the detailed information. Can you provide me with the email address for submitting a formal complaint?\n",
|
||||
"Calling function speak_to_user\n",
|
||||
"Assistant: You're welcome! You can submit your formal complaint via email to our customer relations department at [complaints@ourcompany.com](mailto:complaints@ourcompany.com). Please include all relevant details and any documentation that supports your case to ensure a thorough review.\n",
|
||||
"\n",
|
||||
"If there's anything else I can help you with, feel free to ask!\n",
|
||||
"Achieved objective, closing conversation\n",
|
||||
"\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"for x in questions:\n",
|
||||
"\n",
|
||||
" execute_conversation(x)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f8fa6ca4-a776-4207-b440-4ee6fb8ab16a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Conclusion\n",
|
||||
"\n",
|
||||
"You can now control your LLM's behaviour explicitly by making tool use mandatory, as well as spin up GPT testers to challenge your LLM and to act as automated test cases.\n",
|
||||
"\n",
|
||||
"We hope this has given you an appreciation for a great use case for tool use, and look forward to seeing what you build!"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "openai_test",
|
||||
"language": "python",
|
||||
"name": "openai_test"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Loading…
Reference in New Issue