"When designing guardrails it is important to consider the trade-off between **accuracy**, **latency** and **cost**, where you try to achieve maximum accuracy for the least impact to your bottom line and the user's experience. \n",
"\n",
"We'll begin with a simple **topical guardrail** which aims to detect off-topic questions and prevent the LLM from answering if triggered. This guardrail consists of a simple prompt and uses `gpt-3.5-turbo`, maximising latency/cost over accuracy, but if we wanted to optimize further we could consider:\n",
"- **Accuracy:** You could consider using a fine-tuned model or few-shot examples to increase the accuracy. RAG can also be effective if you have a corpus of information that can help determine whether a piece of content is allowed or not.\n",
"We'll begin with a simple **topical guardrail** which aims to detect off-topic questions and prevent the LLM from answering if triggered. This guardrail consists of a simple prompt and uses `gpt-4o-mini`, maximising latency/cost holding a good enough accuracy, but if we wanted to optimize further we could consider:\n",
"- **Accuracy:** You could consider fine-tuning `gpt-4o-mini` or few-shot examples to increase the accuracy. RAG can also be effective if you have a corpus of information that can help determine whether a piece of content is allowed or not.\n",
"- **Latency/Cost:** You could try fine-tuning smaller models, such as `babbage-002` or open-source offerings like Llama, which can perform quite well when given enough training examples. When using open-source offerings you can also tune the machines you are using for inference to maximize either cost or latency reduction.\n",
"\n",
"This simple guardrail aims to ensure the LLM only answers to a predefined set of topics, and responds to out-of-bounds queries with a canned message.\n",
@ -78,7 +78,7 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 21,
"id": "e95efc89",
"metadata": {},
"outputs": [],
@ -91,7 +91,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 22,
"id": "fee948e2",
"metadata": {},
"outputs": [],
@ -153,7 +153,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 23,
"id": "eba51754",
"metadata": {},
"outputs": [
@ -165,19 +165,29 @@
"Got guardrail response\n",
"Getting LLM response\n",
"Got LLM response\n",
"If you're a cat lover considering getting a dog, it's important to choose a breed that typically has a more cat-like temperament. Here are some dog breeds that are known to be more cat-friendly:\n",
"If you like cats and are considering getting a dog, there are several breeds known for their compatibility with feline friends. Here are some of the best dog breeds that tend to get along well with cats:\n",
"\n",
"1. Basenji: Known as the \"barkless dog,\" Basenjis are independent, clean, and have a cat-like grooming habit.\n",
"1. **Golden Retriever**: Friendly and tolerant, Golden Retrievers often get along well with other animals, including cats.\n",
"\n",
"2. Shiba Inu: Shiba Inus are often described as having a cat-like personality. They are independent, clean, and tend to be reserved with strangers.\n",
"2. **Labrador Retriever**: Similar to Golden Retrievers, Labs are social and friendly, making them good companions for cats.\n",
"\n",
"3. Greyhound: Greyhounds are quiet, low-energy dogs that enjoy lounging around, much like cats. They are also known for their gentle and calm nature.\n",
"3. **Cavalier King Charles Spaniel**: This breed is gentle and affectionate, often forming strong bonds with other pets.\n",
"\n",
"4. Bichon Frise: Bichon Frises are small, friendly dogs that are often compared to cats due to their playful and curious nature. They are also hypoallergenic, making them a good choice for those with allergies.\n",
"4. **Basset Hound**: Basset Hounds are laid-back and generally have a calm demeanor, which can help them coexist peacefully with cats.\n",
"\n",
"5. Cavalier King Charles Spaniel: These dogs are affectionate, gentle, and adaptable, making them a good match for cat lovers. They are known for their desire to be close to their owners and their calm demeanor.\n",
"5. **Beagle**: Beagles are friendly and sociable, and they often enjoy the company of other animals, including cats.\n",
"\n",
"Remember, individual dogs can have different personalities, so it's important to spend time with the specific dog you're considering to see if their temperament aligns with your preferences.\n"
"6. **Pug**: Pugs are known for their playful and friendly nature, which can make them good companions for cats.\n",
"\n",
"7. **Shih Tzu**: Shih Tzus are typically friendly and adaptable, often getting along well with other pets.\n",
"\n",
"8. **Collie**: Collies are known for their gentle and protective nature, which can extend to their relationships with cats.\n",
"\n",
"9. **Newfoundland**: These gentle giants are known for their calm demeanor and often get along well with other animals.\n",
"\n",
"10. **Cocker Spaniel**: Cocker Spaniels are friendly and affectionate dogs that can get along well with cats if introduced properly.\n",
"\n",
"When introducing a dog to a cat, it's important to do so gradually and supervise their interactions to ensure a positive relationship. Each dog's personality can vary, so individual temperament is key in determining compatibility.\n"
]
}
],
@ -189,7 +199,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 24,
"id": "c7d88b57",
"metadata": {},
"outputs": [
@ -254,7 +264,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 16,
"id": "6c7b459f",
"metadata": {},
"outputs": [],
@ -294,7 +304,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 17,
"id": "43e3fd36",
"metadata": {},
"outputs": [],
@ -347,7 +357,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 18,
"id": "beea1305",
"metadata": {},
"outputs": [],
@ -358,7 +368,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 19,
"id": "1c582b4d",
"metadata": {},
"outputs": [
@ -392,30 +402,8 @@
"Got LLM response\n",
"Checking moderation guardrail\n",
"Got moderation response\n",
"Passed moderation\n",
"As a new dog owner, here are some helpful tips:\n",
"\n",
"1. Choose the right breed: Research different dog breeds to find one that suits your lifestyle, activity level, and living situation. Some breeds require more exercise and attention than others.\n",
"\n",
"2. Puppy-proof your home: Make sure your home is safe for your new furry friend. Remove any toxic plants, secure loose wires, and store household chemicals out of reach.\n",
"\n",
"3. Establish a routine: Dogs thrive on routine, so establish a consistent schedule for feeding, exercise, and bathroom breaks. This will help your dog feel secure and reduce any anxiety.\n",
"\n",
"4. Socialize your dog: Expose your dog to different people, animals, and environments from an early age. This will help them become well-adjusted and comfortable in various situations.\n",
"\n",
"5. Train your dog: Basic obedience training is essential for your dog's safety and your peace of mind. Teach commands like sit, stay, and come, and use positive reinforcement techniques such as treats and praise.\n",
"\n",
"6. Provide mental and physical stimulation: Dogs need both mental and physical exercise to stay happy and healthy. Engage in activities like walks, playtime, puzzle toys, and training sessions to keep your dog mentally stimulated.\n",
"\n",
"7. Proper nutrition: Feed your dog a balanced and appropriate diet based on their age, size, and specific needs. Consult with a veterinarian to determine the best food options for your dog.\n",
"\n",
"8. Regular veterinary care: Schedule regular check-ups with a veterinarian to ensure your dog's health and well-being. Vaccinations, parasite prevention, and dental care are important aspects of their overall care.\n",
"\n",
"9. Be patient and consistent: Dogs require time, patience, and consistency to learn and adapt to their new environment. Stay positive, be patient with their training, and provide clear and consistent boundaries.\n",
"\n",
"10. Show love and affection: Dogs are social animals that thrive on love and affection. Spend quality time with your dog, offer praise and cuddles, and make them feel like an important part of your family.\n",
"\n",
"Remember, being a responsible dog owner involves commitment, time, and effort. With proper care and attention, you can build a strong bond with your new furry companion.\n",
"Moderation guardrail flagged with a score of 3\n",
"Sorry, we're not permitted to give animal breed advice. I can help you with any general queries you might have.\n",
"\n",
"\n",
"\n"
@ -447,13 +435,19 @@
"\n",
"We look forward to seeing how you take this forward, and how thinking on guardrails evolves as the ecosystem matures. "
"Custom moderation provides a tailored approach to content filtering, allowing for precise adherence to specific community standards or topics, making it an ideal choice for niche platforms or specialized content. Unlike the general categories covered by the moderation API, custom moderation offers a higher degree of control and requires more effort to set up and maintain. It shares the common goal of ensuring content safety with guardrails, focusing on setting criteria to filter and manage content, whether it's user input or output from language models. Custom moderation can be an alternative or an extension to the moderation API, that uses our completions endpoint. For those interested in exploring guardrails further, a cookbook with examples and guidance is available [here](https://cookbook.openai.com/examples/how_to_use_guardrails).\n",
"\n",
"#### Workflow\n",
"We will implement a small example to showcase custom moderation. We will use an extra call to gpt-4 to assess messages and decide if they should be moderated or not."
"We will implement a small example to showcase custom moderation. We will use an extra call to gpt-4o mini to assess messages and decide if they should be moderated or not."
]
},
{
@ -349,7 +349,7 @@
"outputs": [],
"source": [
"def custom_moderation(content, parameters):\n",
" # Define the prompt for GPT-4\n",
" # Define the prompt \n",
" prompt = f\"\"\"Please assess the following content for any inappropriate material. You should base your assessment on the given parameters.\n",
" Your answer should be in json format with the following fields: \n",
" - flagged: a boolean indicating whether the content is flagged for any of the categories in the parameters\n",
@ -357,9 +357,9 @@
" - parameters: a dictionary of the parameters used for the assessment and their values\n",
"This cookbook will walk you through how to use the Batch API with a couple of practical examples.\n",
"\n",
"We will start with an example to categorize movies using `gpt-3.5-turbo`, and then cover how we can use the vision capabilities of `gpt-4-turbo` to caption images.\n",
"We will start with an example to categorize movies using `gpt-4o-mini`, and then cover how we can use the vision capabilities of this model to caption images.\n",
"\n",
"Please note that multiple models are available through the Batch API, and that you can use the same parameters in your Batch API calls as with the Chat Completions endpoint."
]
@ -77,7 +77,7 @@
"source": [
"## First example: Categorizing movies\n",
"\n",
"In this example, we will use `gpt-3.5-turbo` to extract movie categories from a description of the movie. We will also extract a 1-sentence summary from this description. \n",
"In this example, we will use `gpt-4o-mini` to extract movie categories from a description of the movie. We will also extract a 1-sentence summary from this description. \n",
"\n",
"We will use [JSON mode](https://platform.openai.com/docs/guides/text-generation/json-mode) to extract categories as an array of strings and the 1-sentence summary in a structured format. \n",
"\n",
@ -338,7 +338,7 @@
"\n",
"def get_categories(description):\n",
" response = client.chat.completions.create(\n",
" model=\"gpt-3.5-turbo\",\n",
" model=\"gpt-4o-mini\",\n",
" temperature=0.1,\n",
" # This is to enable JSON mode, making sure responses are valid json objects\n",
" response_format={ \n",
@ -374,7 +374,7 @@
"\n",
"RESULT: {\n",
" \"categories\": [\"drama\"],\n",
" \"summary\": \"Two imprisoned men bond over the years and find redemption through acts of common decency.\"\n",
" \"summary\": \"Two imprisoned men develop a deep bond over the years, ultimately finding redemption through their shared acts of kindness.\"\n",
"}\n",
"\n",
"\n",
@ -386,7 +386,7 @@
"\n",
"RESULT: {\n",
" \"categories\": [\"crime\", \"drama\"],\n",
" \"summary\": \"A crime drama about an aging patriarch passing on his empire to his son.\"\n",
" \"summary\": \"An aging crime lord hands over his empire to his hesitant son.\"\n",
"}\n",
"\n",
"\n",
@ -397,8 +397,8 @@
"OVERVIEW: When the menace known as the Joker wreaks havoc and chaos on the people of Gotham, Batman must accept one of the greatest psychological and physical tests of his ability to fight injustice.\n",
"\n",
"RESULT: {\n",
" \"categories\": [\"action\", \"thriller\"],\n",
" \"summary\": \"A thrilling action movie where Batman faces the chaotic Joker in a battle of justice.\"\n",
" \"summary\": \"Batman faces a formidable challenge as the Joker unleashes chaos on Gotham City.\"\n",
"}\n",
"\n",
"\n",
@ -410,7 +410,7 @@
"\n",
"RESULT: {\n",
" \"categories\": [\"crime\", \"drama\"],\n",
" \"summary\": \"A portrayal of Vito Corleone's early life and career in 1920s New York City, as his son Michael expands the family crime syndicate.\"\n",
" \"summary\": \"The film depicts the early life of Vito Corleone and the rise of his son Michael within the family crime syndicate in 1920s New York City.\"\n",
"}\n",
"\n",
"\n",
@ -421,8 +421,8 @@
"OVERVIEW: A jury holdout attempts to prevent a miscarriage of justice by forcing his colleagues to reconsider the evidence.\n",
"\n",
"RESULT: {\n",
" \"categories\": [\"drama\"],\n",
" \"summary\": \"A gripping drama about a jury holdout trying to prevent a miscarriage of justice by challenging his colleagues to reconsider the evidence.\"\n",
" \"categories\": [\"drama\", \"thriller\"],\n",
" \"summary\": \"A jury holdout fights to ensure justice is served by challenging his fellow jurors to reevaluate the evidence.\"\n",
"}\n",
"\n",
"\n",
@ -489,7 +489,7 @@
" \"url\": \"/v1/chat/completions\",\n",
" \"body\": {\n",
" # This is what you would have in your Chat Completions API call\n",
"CAPTION: White multi-layer metal shoe rack featuring eight double hooks for hanging accessories, ideal for organizing footwear and small items in living spaces.\n",
"CAPTION: A stylish white free-standing shoe rack featuring multiple layers and eight double hooks, perfect for organizing shoes and accessories in living rooms, bathrooms, or hallways.\n",
"\n",
"\n"
]
@ -1161,7 +1153,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"CAPTION: A set of two elegant black leather dining chairs with a sleek design and vertical stitching detail on the backrest.\n",
"CAPTION: Set of 2 black leather dining chairs featuring a sleek design with vertical stitching and sturdy wooden legs.\n",
"\n",
"\n"
]
@ -1182,7 +1174,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"CAPTION: A green, waterproof, square, foldable repotting mat designed for indoor gardening, featuring raised edges and displayed with gardening tools and small potted plants.\n",
"CAPTION: The MUYETOL Plant Repotting Mat is a waterproof, portable, and foldable gardening work mat measuring 26.8\" x 26.8\", designed for easy soil changing and indoor transplanting.\n",
"\n",
"\n"
]
@ -1203,7 +1195,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"CAPTION: A brown, absorbent non-slip doormat featuring the phrase \"It's a good day to play PICKLEBALL\" with a pickleball paddle graphic, ideal for sports enthusiasts.\n",
"CAPTION: Absorbent non-slip doormat featuring the phrase \"It's a good day to play PICKLEBALL\" with paddle graphics, measuring 16x24 inches.\n",
"\n",
"\n"
]
@ -1224,7 +1216,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"CAPTION: Set of four foldable grey TV trays with a stand, featuring a sleek, space-saving design suitable for small areas.\n",
"CAPTION: Set of 4 foldable TV trays in grey, featuring a compact design with a stand for easy storage, perfect for small spaces.\n",
"\n",
"\n"
]
@ -1252,7 +1244,7 @@
},
{
"cell_type": "code",
"execution_count": 35,
"execution_count": 16,
"id": "48e59bb1",
"metadata": {},
"outputs": [],
@ -1272,7 +1264,7 @@
" \"url\": \"/v1/chat/completions\",\n",
" \"body\": {\n",
" # This is what you would have in your Chat Completions API call\n",
"In this cookbook, we have seen two examples of how to use the new Batch API, but keep in mind that the Batch API works the same way as the Chat Completions endpoint, supporting the same parameters and most of the recent models (gpt-3.5-turbo, gpt-4, gpt-4-vision-preview, gpt-4-turbo...).\n",
"In this cookbook, we have seen two examples of how to use the new Batch API, but keep in mind that the Batch API works the same way as the Chat Completions endpoint, supporting the same parameters and most of the recent models (gpt-4o, gpt-4o-mini, gpt-4-turbo, gpt-3.5-turbo...).\n",
"\n",
"By using this API, you can significantly reduce costs, so we recommend switching every workload that can happen async to a batch job with this new API."
arnaud-openai
2 months ago
committed by
GitHub