You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
341 lines
11 KiB
Plaintext
341 lines
11 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Azure chat completions example\n",
|
|
"\n",
|
|
"This example will cover chat completions using the Azure OpenAI service. It also includes information on content filtering."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Setup\n",
|
|
"\n",
|
|
"First, we install the necessary dependencies and import the libraries we will be using."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"! pip install \"openai>=1.0.0,<2.0.0\"\n",
|
|
"! pip install python-dotenv"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"import openai\n",
|
|
"import dotenv\n",
|
|
"\n",
|
|
"dotenv.load_dotenv()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Authentication\n",
|
|
"\n",
|
|
"The Azure OpenAI service supports multiple authentication mechanisms that include API keys and Azure Active Directory token credentials."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"use_azure_active_directory = False # Set this flag to True if you are using Azure Active Directory"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Authentication using API key\n",
|
|
"\n",
|
|
"To set up the OpenAI SDK to use an *Azure API Key*, we need to set `api_key` to a key associated with your endpoint (you can find this key in *\"Keys and Endpoints\"* under *\"Resource Management\"* in the [Azure Portal](https://portal.azure.com)). You'll also find the endpoint for your resource here."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"if not use_azure_active_directory:\n",
|
|
" endpoint = os.environ[\"AZURE_OPENAI_ENDPOINT\"]\n",
|
|
" api_key = os.environ[\"AZURE_OPENAI_API_KEY\"]\n",
|
|
"\n",
|
|
" client = openai.AzureOpenAI(\n",
|
|
" azure_endpoint=endpoint,\n",
|
|
" api_key=api_key,\n",
|
|
" api_version=\"2023-09-01-preview\"\n",
|
|
" )"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Authentication using Azure Active Directory\n",
|
|
"Let's now see how we can autheticate via Azure Active Directory. We'll start by installing the `azure-identity` library. This library will provide the token credentials we need to authenticate and help us build a token credential provider through the `get_bearer_token_provider` helper function. It's recommended to use `get_bearer_token_provider` over providing a static token to `AzureOpenAI` because this API will automatically cache and refresh tokens for you. \n",
|
|
"\n",
|
|
"For more information on how to set up Azure Active Directory authentication with Azure OpenAI, see the [documentation](https://learn.microsoft.com/azure/ai-services/openai/how-to/managed-identity)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"! pip install \"azure-identity>=1.15.0\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azure.identity import DefaultAzureCredential, get_bearer_token_provider\n",
|
|
"\n",
|
|
"if use_azure_active_directory:\n",
|
|
" endpoint = os.environ[\"AZURE_OPENAI_ENDPOINT\"]\n",
|
|
"\n",
|
|
" client = openai.AzureOpenAI(\n",
|
|
" azure_endpoint=endpoint,\n",
|
|
" azure_ad_token_provider=get_bearer_token_provider(DefaultAzureCredential(), \"https://cognitiveservices.azure.com/.default\"),\n",
|
|
" api_version=\"2023-09-01-preview\"\n",
|
|
" )"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"> Note: the AzureOpenAI infers the following arguments from their corresponding environment variables if they are not provided:\n",
|
|
"\n",
|
|
"- `api_key` from `AZURE_OPENAI_API_KEY`\n",
|
|
"- `azure_ad_token` from `AZURE_OPENAI_AD_TOKEN`\n",
|
|
"- `api_version` from `OPENAI_API_VERSION`\n",
|
|
"- `azure_endpoint` from `AZURE_OPENAI_ENDPOINT`\n"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Deployments\n",
|
|
"\n",
|
|
"In this section we are going to create a deployment of a GPT model that we can use to create chat completions."
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Deployments: Create in the Azure OpenAI Studio\n",
|
|
"Let's deploy a model to use with chat completions. Go to https://portal.azure.com, find your Azure OpenAI resource, and then navigate to the Azure OpenAI Studio. Click on the \"Deployments\" tab and then create a deployment for the model you want to use for chat completions. The deployment name that you give the model will be used in the code below."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"deployment = \"\" # Fill in the deployment name from the portal here"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Create chat completions\n",
|
|
"\n",
|
|
"Now let's create a chat completion using the client we built."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# For all possible arguments see https://platform.openai.com/docs/api-reference/chat-completions/create\n",
|
|
"response = client.chat.completions.create(\n",
|
|
" model=deployment,\n",
|
|
" messages=[\n",
|
|
" {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
|
|
" {\"role\": \"user\", \"content\": \"Knock knock.\"},\n",
|
|
" {\"role\": \"assistant\", \"content\": \"Who's there?\"},\n",
|
|
" {\"role\": \"user\", \"content\": \"Orange.\"},\n",
|
|
" ],\n",
|
|
" temperature=0,\n",
|
|
")\n",
|
|
"\n",
|
|
"print(f\"{response.choices[0].message.role}: {response.choices[0].message.content}\")"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Create a streaming chat completion\n",
|
|
"\n",
|
|
"We can also stream the response."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.chat.completions.create(\n",
|
|
" model=deployment,\n",
|
|
" messages=[\n",
|
|
" {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
|
|
" {\"role\": \"user\", \"content\": \"Knock knock.\"},\n",
|
|
" {\"role\": \"assistant\", \"content\": \"Who's there?\"},\n",
|
|
" {\"role\": \"user\", \"content\": \"Orange.\"},\n",
|
|
" ],\n",
|
|
" temperature=0,\n",
|
|
" stream=True\n",
|
|
")\n",
|
|
"\n",
|
|
"for chunk in response:\n",
|
|
" if len(chunk.choices) > 0:\n",
|
|
" delta = chunk.choices[0].delta\n",
|
|
"\n",
|
|
" if delta.role:\n",
|
|
" print(delta.role + \": \", end=\"\", flush=True)\n",
|
|
" if delta.content:\n",
|
|
" print(delta.content, end=\"\", flush=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Content filtering\n",
|
|
"\n",
|
|
"Azure OpenAI service includes content filtering of prompts and completion responses. You can learn more about content filtering and how to configure it [here](https://learn.microsoft.com/azure/ai-services/openai/concepts/content-filter).\n",
|
|
"\n",
|
|
"If the prompt is flagged by the content filter, the library will raise a `BadRequestError` exception with a `content_filter` error code. Otherwise, you can access the `prompt_filter_results` and `content_filter_results` on the response to see the results of the content filtering and what categories were flagged."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Prompt flagged by content filter"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import json\n",
|
|
"\n",
|
|
"messages = [\n",
|
|
" {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
|
|
" {\"role\": \"user\", \"content\": \"<text violating the content policy>\"}\n",
|
|
"]\n",
|
|
"\n",
|
|
"try:\n",
|
|
" completion = client.chat.completions.create(\n",
|
|
" messages=messages,\n",
|
|
" model=deployment,\n",
|
|
" )\n",
|
|
"except openai.BadRequestError as e:\n",
|
|
" err = json.loads(e.response.text)\n",
|
|
" if err[\"error\"][\"code\"] == \"content_filter\":\n",
|
|
" print(\"Content filter triggered!\")\n",
|
|
" content_filter_result = err[\"error\"][\"innererror\"][\"content_filter_result\"]\n",
|
|
" for category, details in content_filter_result.items():\n",
|
|
" print(f\"{category}:\\n filtered={details['filtered']}\\n severity={details['severity']}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Checking the result of the content filter"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"messages = [\n",
|
|
" {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
|
|
" {\"role\": \"user\", \"content\": \"What's the biggest city in Washington?\"}\n",
|
|
"]\n",
|
|
"\n",
|
|
"completion = client.chat.completions.create(\n",
|
|
" messages=messages,\n",
|
|
" model=deployment,\n",
|
|
")\n",
|
|
"print(f\"Answer: {completion.choices[0].message.content}\")\n",
|
|
"\n",
|
|
"# prompt content filter result in \"model_extra\" for azure\n",
|
|
"prompt_filter_result = completion.model_extra[\"prompt_filter_results\"][0][\"content_filter_results\"]\n",
|
|
"print(\"\\nPrompt content filter results:\")\n",
|
|
"for category, details in prompt_filter_result.items():\n",
|
|
" print(f\"{category}:\\n filtered={details['filtered']}\\n severity={details['severity']}\")\n",
|
|
"\n",
|
|
"# completion content filter result\n",
|
|
"print(\"\\nCompletion content filter results:\")\n",
|
|
"completion_filter_result = completion.choices[0].model_extra[\"content_filter_results\"]\n",
|
|
"for category, details in completion_filter_result.items():\n",
|
|
" print(f\"{category}:\\n filtered={details['filtered']}\\n severity={details['severity']}\")"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.0"
|
|
},
|
|
"orig_nbformat": 4
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|