"The `ChatCompletion` endpoint now includes the ability to specify whether a tool **must** be called every time. This adds an element of determinism to how you build your wrapping application, as you can count on a tool being provided with every call. \n",
"The `ChatCompletion` endpoint now includes the ability to specify whether a tool **must** be called every time, by adding `tool_choice='required'` as a parameter. \n",
"\n",
"We'll demonstrate here how this can be useful for a contained flow like customer service, where having the ability to define specific exit points gives more control.\n",
"This adds an element of determinism to how you build your wrapping application, as you can count on a tool being provided with every call. We'll demonstrate here how this can be useful for a contained flow like customer service, where having the ability to define specific exit points gives more control.\n",
"\n",
"The notebook concludes with a multi-turn evaluation, where we spin up a customer GPT to imitate our customer and test the LLM customer service agent we've set up."
]
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": 1,
"id": "ba4759e0-ecfd-48f7-bbd8-79ea61aef872",
"metadata": {},
"outputs": [],
@ -43,7 +43,7 @@
},
{
"cell_type": "code",
"execution_count": 179,
"execution_count": 2,
"id": "31fd0251-f741-46d6-979b-a2bbc1f95571",
"metadata": {},
"outputs": [],
@ -116,7 +116,7 @@
},
{
"cell_type": "code",
"execution_count": 229,
"execution_count": 3,
"id": "6c0ad691-28f4-4707-8e23-0d0a6c06ea1e",
"metadata": {},
"outputs": [],
@ -231,7 +231,7 @@
},
{
"cell_type": "code",
"execution_count": 230,
"execution_count": 4,
"id": "bb1530e4-dd82-4560-bd60-9cc9ac0dab73",
"metadata": {},
"outputs": [
@ -242,17 +242,17 @@
"User: Hi, I have had an item stolen that was supposed to be delivered to me yesterday.\n",
"Calling function get_instructions\n",
"Calling function speak_to_user\n",
"Assistant: I'm sorry to hear about the stolen item. Could you please provide me with more details about the fraudulent activity, including the date and the items involved? This information will help us to investigate further and process any necessary actions, such as a refund.\n"
"Assistant: I'm sorry to hear about the stolen item. Could you please provide me with more details about the fraudulent activity, including the date and the items involved? This information will help us to investigate the issue further and proceed with the necessary actions, including offering you a refund.\n"
]
}
],
"source": [
"messages = submit_user_message(\"Hi, I have had an item stolen that was supposed to be delivered to me yesterday.\",messages)"
"messages = submit_user_message(\"Hi, I have had an item stolen that was supposed to be delivered to me yesterday.\")"
]
},
{
"cell_type": "code",
"execution_count": 231,
"execution_count": 5,
"id": "ccff3dd7-d10f-4dc7-9737-6ea5d126e829",
"metadata": {},
"outputs": [
@ -262,7 +262,9 @@
"text": [
"User: For sure, it was a shirt, it was supposed to be delivered yesterday but it never arrived.\n",
"Calling function speak_to_user\n",
"Assistant: Thank you for providing the details. Would you like me to proceed with a refund for the missing shirt? Additionally, I will report this incident to our security team for further investigation.\n"
"Assistant: Thank you for providing the details. I will now proceed to report this incident to our security team for further investigation and arrange a refund for the stolen shirt. Please confirm if you would like me to go ahead with the refund.\n",
"Calling function speak_to_user\n",
"Assistant: Thank you for contacting us about this issue. Please don't hesitate to reach out if you have any more questions or need further assistance in the future.\n"
]
}
],
@ -272,7 +274,7 @@
},
{
"cell_type": "code",
"execution_count": 232,
"execution_count": 6,
"id": "ce3a8869-8b14-4404-866a-4b540b13235c",
"metadata": {},
"outputs": [
@ -283,7 +285,7 @@
"User: Yes I would like to proceed with the refund.\n",
"Calling function get_instructions\n",
"Calling function speak_to_user\n",
"Assistant: The refund for the missing shirt has been processed. We have also reported this incident to our security team for further investigation. Thank you for bringing this to our attention, and please don't hesitate to reach out if you have any more questions or need further assistance.\n"
"Assistant: Thank you for confirming. I have processed the refund for the stolen shirt. The amount should be reflected in your account within 5-7 business days. If you have any more questions or need further assistance, please feel free to contact us.\n"
]
}
],
@ -293,7 +295,7 @@
},
{
"cell_type": "code",
"execution_count": 233,
"execution_count": 7,
"id": "87e5cd3e-4edb-426c-8fd9-8fe3bde61bcd",
"metadata": {},
"outputs": [
@ -303,7 +305,7 @@
"text": [
"User: Thanks very much.\n",
"Calling function speak_to_user\n",
"Assistant: You're welcome! If you need any more help in the future, feel free to contact us. Have a great day!\n"
"Assistant: You're welcome! If you need any more help in the future, don't hesitate to reach out. Have a great day!\n"
]
}
],
@ -325,7 +327,7 @@
},
{
"cell_type": "code",
"execution_count": 227,
"execution_count": 8,
"id": "f4931776-b3ac-4113-98e8-419a0965fd71",
"metadata": {},
"outputs": [],
@ -348,7 +350,7 @@
},
{
"cell_type": "code",
"execution_count": 228,
"execution_count": 9,
"id": "22b12f59-4418-4aee-ae92-6c6ebcf0f2d3",
"metadata": {},
"outputs": [],
@ -404,7 +406,7 @@
},
{
"cell_type": "code",
"execution_count": 221,
"execution_count": 10,
"id": "9d9aac9f-f557-4e7e-b705-adf7d5aa1f3f",
"metadata": {},
"outputs": [
@ -415,57 +417,72 @@
"User: I want to get a refund for the suit I ordered last Friday.\n",
"Calling function get_instructions\n",
"Calling function speak_to_user\n",
"Assistant: I understand you'd like a refund for the suit you ordered last Friday. Could you please provide me with more details about the reason for the refund? This will help us process your request accurately.\n",
"User: The suit I received does not match the description on the website. The color and material are completely different from what was advertised.\n",
"Assistant: I understand you'd like a refund for the suit you ordered last Friday. Could you please provide more details about the issue with the suit? This will help us process your refund request accurately.\n",
"User: The suit I received is not the color I ordered. I ordered a navy blue suit, but the one I received is black.\n",
"Calling function speak_to_user\n",
"Assistant: Thank you for providing the details. I will proceed with the refund for the suit that did not match the description on our website. Please confirm if you would like me to initiate the refund now.\n",
"User: Yes, please initiate the refund now.\n",
"Assistant: Thank you for providing the details. I will proceed with the refund for the navy blue suit that was incorrectly sent as black. Please confirm if you would like me to go ahead with the refund.\n",
"User: Yes, please go ahead with the refund.\n",
"Calling function speak_to_user\n",
"Assistant: I have initiated the refund for the suit. The amount will be credited back to your original method of payment within 5-7 business days. Thank you for bringing this to our attention, and we apologize for any inconvenience caused. If you have any more questions or need further assistance, please feel free to reach out.\n",
"Assistant: The refund for the incorrectly colored suit has been processed. You should see the amount credited back to your original payment method within 5-7 business days. Thank you for contacting us, and if you have any more questions or need further assistance, please feel free to reach out.\n",
"Achieved objective, closing conversation\n",
"\n",
"\n",
"User: Can you tell me what your policy is for returning damaged goods?\n",
"Calling function get_instructions\n",
"Calling function speak_to_user\n",
"Assistant: It seems there was a misunderstanding in my previous response. Could you please clarify if you are dealing with a damaged product that you wish to return, or if there's another issue I can assist you with?\n",
"User: Yes, I received a damaged product and I would like to return it.\n",
"Assistant: It seems there was a misunderstanding in my previous request. I'm looking for information on our policy for returning damaged goods. Could you please provide me with the details on how to handle returns for damaged items?\n",
"User: Yes, I'd appreciate that. Could you please tell me the steps I need to follow to return a damaged item?\n",
"Calling function get_instructions\n",
"Calling function speak_to_user\n",
"Assistant: I apologize for the inconvenience caused by receiving a damaged product. Could you please provide more details about the product and the nature of the damage? This will help us process your return request effectively.\n",
"User: The product is a ceramic vase that arrived with several cracks and a large piece broken off at the top.\n",
"Assistant: I apologize for the confusion earlier. Here's the correct information regarding our policy for returning damaged goods:\n",
"\n",
"1. Please provide a description of the damage and the item involved.\n",
"2. Include the date of purchase and your order number if available.\n",
"3. You can choose to return the item by mail or in person at one of our stores. Please let us know which method you prefer, and we will provide the necessary details for the return process.\n",
"4. Once we receive the item, we will inspect it and process a refund or exchange based on your preference and our return policy guidelines.\n",
"\n",
"Please let me know if you need further assistance with this process or if there's anything else I can help you with!\n",
"User: I would like to return the item by mail. Could you please provide me with the details on how to do that?\n",
"Calling function speak_to_user\n",
"Assistant: Certainly! To return the damaged item by mail, please follow these steps:\n",
"\n",
"1. Pack the item securely in its original packaging, if possible.\n",
"2. Include all original materials such as manuals, accessories, and receipts.\n",
"3. Address the package to our returns department at the following address:\n",
" [Your Company's Return Address]\n",
"4. Use a trackable shipping service and keep the tracking number for your records.\n",
"5. Once we receive and inspect the item, we will process your refund or exchange as per your request.\n",
"\n",
"Please let me know if you need a prepaid shipping label or any other assistance with your return!\n",
"User: Yes, please, I would need a prepaid shipping label.\n",
"Calling function speak_to_user\n",
"Assistant: Thank you for providing the details. We will proceed with the return and refund for the damaged ceramic vase. Please confirm if you would like me to initiate the refund process now.\n",
"User: Yes, please initiate the refund process now.\n",
"Assistant: I will arrange for a prepaid shipping label to be sent to your email address. Please ensure that the email address we have on file is correct. You should receive the label shortly, which you can print and attach to your return package.\n",
"\n",
"Once you've shipped the item, please provide us with the tracking number so we can monitor the return's progress. If there's anything else you need help with, feel free to ask!\n",
"User: That sounds great, thank you! My email address is correct on file. I'll look out for the prepaid shipping label and send the package back as soon as I receive it.\n",
"Calling function speak_to_user\n",
"Assistant: The refund process for your damaged ceramic vase has been initiated. You will receive a confirmation email shortly with the details of the refund and instructions for returning the item. Thank you for your patience, and please let us know if there is anything else we can assist you with.\n",
"Assistant: You're welcome! I'm glad I could assist you. Once you receive the prepaid shipping label, please proceed with the return as discussed. If you have any more questions or need further assistance in the future, don't hesitate to reach out. Have a great day!\n",
"Achieved objective, closing conversation\n",
"\n",
"\n",
"User: Please tell me what your complaint policy is\n",
"Calling function get_instructions\n",
"Calling function speak_to_user\n",
"Assistant: It seems there was a misunderstanding with the instructions I received. Let me clarify our complaint policy for you.\n",
"\n",
"Our complaint policy is designed to ensure that any issues you face are addressed promptly and fairly. If you have a complaint, you can contact our customer service team through various channels such as phone, email, or live chat. We aim to resolve complaints quickly, typically within a few business days. If the issue is complex and requires more time, we will keep you informed throughout the process.\n",
"\n",
"Please let me know if you need more specific information or if there's anything else I can assist you with!\n",
"User: That helps a lot, thank you. Could you also tell me how I can escalate a complaint if I'm not satisfied with the resolution?\n",
"Assistant: It seems there was a misunderstanding. I will provide you with information about our complaint policy. Please give me a moment to gather the correct details for you.\n",
"User: Sure, I'll wait. Thank you.\n",
"Calling function speak_to_user\n",
"Assistant: Certainly! If you are not satisfied with the resolution of your complaint, you can escalate it by following these steps:\n",
"Assistant: Our complaint policy is designed to ensure that any issues you encounter are addressed promptly and fairly. Here’s a brief overview:\n",
"\n",
"1. **Re-contact our Customer Service:** Start by contacting our customer service team again, explaining that you are not satisfied with the resolution and would like to escalate the matter. Provide any additional details that may help in re-evaluating the case.\n",
"1. **Submission**: You can submit your complaint through our website, by email, or by calling our customer service hotline.\n",
"2. **Acknowledgment**: We will acknowledge receipt of your complaint within 24 hours.\n",
"3. **Investigation**: We will investigate your complaint thoroughly. This process typically takes 3-5 business days.\n",
"4. **Resolution**: After the investigation, we will contact you with the outcome and any steps we will take to resolve the issue.\n",
"5. **Follow-up**: If you are not satisfied with the resolution, you can request a review of the decision.\n",
"\n",
"2. **Supervisor Review:** If the issue remains unresolved, you can request that your complaint be reviewed by a supervisor or a manager. This provides an additional level of scrutiny and ensures that your concerns are thoroughly examined.\n",
"\n",
"3. **Formal Complaint:** If you still feel that your concerns have not been adequately addressed, you can submit a formal complaint in writing. This should be directed to our customer relations department, where it will be handled according to our formal complaint procedures.\n",
"\n",
"We are committed to ensuring that your concerns are addressed satisfactorily, and these steps are in place to provide multiple levels of oversight and review.\n",
"User: Thank you for the detailed information. Can you provide me with the email address for submitting a formal complaint?\n",
"Please let me know if you need more detailed information or if there's anything else I can assist you with!\n",
"User: That covers everything I needed to know, thank you!\n",
"Calling function speak_to_user\n",
"Assistant: You're welcome! You can submit your formal complaint via email to our customer relations department at [complaints@ourcompany.com](mailto:complaints@ourcompany.com). Please include all relevant details and any documentation that supports your case to ensure a thorough review.\n",
"\n",
"If there's anything else I can help you with, feel free to ask!\n",
"Assistant: You're welcome! I'm glad I could help. If you have any more questions in the future or need further assistance, feel free to reach out. Have a great day!\n",
Colin Jarvis
1 month ago