mirror of
https://github.com/openai/openai-cookbook
synced 2024-11-11 13:11:02 +00:00
318 lines
13 KiB
Plaintext
318 lines
13 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Azure chat completion models with your own data (preview)\n",
|
|
"This example shows how to use Azure OpenAI service models with your own data. The feature is currently in preview. \n",
|
|
"\n",
|
|
"Azure OpenAI on your data enables you to run supported chat models such as GPT-3.5-Turbo and GPT-4 on your data without needing to train or fine-tune models. Running models on your data enables you to chat on top of, and analyze your data with greater accuracy and speed. One of the key benefits of Azure OpenAI on your data is its ability to tailor the content of conversational AI. Because the model has access to, and can reference specific sources to support its responses, answers are not only based on its pretrained knowledge but also on the latest information available in the designated data source. This grounding data also helps the model avoid generating responses based on outdated or incorrect information.\n",
|
|
"\n",
|
|
"Azure OpenAI on your own data with Azure AI Search (f.k.a. Azure Cognitive Search) provides a customizable, pre-built solution for knowledge retrieval, from which a conversational AI application can be built. To see alternative methods for knowledge retrieval and semantic search, check out the cookbook examples for [vector databases](https://github.com/openai/openai-cookbook/tree/main/examples/vector_databases)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## How it works\n",
|
|
"\n",
|
|
"[Azure OpenAI on your own data](https://learn.microsoft.com/azure/ai-services/openai/concepts/use-your-data) connects the model with your data, giving it the ability to retrieve and utilize data in a way that enhances the model's output. Together with Azure AI Search, data is retrieved from designated data sources based on the user input and provided conversation history. The data is then augmented and resubmitted as a prompt to the model, giving the model contextual information it can use to generate a response.\n",
|
|
"\n",
|
|
"See the [Data, privacy, and security for Azure OpenAI Service](https://learn.microsoft.com/legal/cognitive-services/openai/data-privacy?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext) for more information."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Prerequisites\n",
|
|
"To get started, we'll cover a few prequisites. \n",
|
|
"\n",
|
|
"To properly access the Azure OpenAI Service, we need to create the proper resources at the [Azure Portal](https://portal.azure.com) (you can check a detailed guide on how to do this in the [Microsoft Docs](https://learn.microsoft.com/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal))\n",
|
|
"\n",
|
|
"To use your own data with Azure OpenAI models, you will need:\n",
|
|
"\n",
|
|
"1. Azure OpenAI access and a resource with a chat model deployed (for example, GPT-3 or GPT-4)\n",
|
|
"2. Azure AI Search (f.k.a. Azure Cognitive Search) resource\n",
|
|
"3. Azure Blob Storage resource\n",
|
|
"4. Your documents to be used as data (See [data source options](https://learn.microsoft.com/azure/ai-services/openai/concepts/use-your-data#data-source-options))\n",
|
|
"\n",
|
|
"\n",
|
|
"For a full walk-through on how to upload your documents to blob storage and create an index using the Azure AI Studio, see this [Quickstart](https://learn.microsoft.com/azure/ai-services/openai/use-your-data-quickstart?pivots=programming-language-studio&tabs=command-line)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Setup\n",
|
|
"\n",
|
|
"First, we install the necessary dependencies."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"! pip install \"openai>=1.0.0,<2.0.0\"\n",
|
|
"! pip install python-dotenv"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In this example, we'll use `dotenv` to load our environment variables. To connect with Azure OpenAI and the Search index, the following variables should be added to a `.env` file in `KEY=VALUE` format:\n",
|
|
"\n",
|
|
"* `AZURE_OPENAI_ENDPOINT` - the Azure OpenAI endpoint. This can be found under \"Keys and Endpoints\" for your Azure OpenAI resource in the Azure Portal.\n",
|
|
"* `AZURE_OPENAI_API_KEY` - the Azure OpenAI API key. This can be found under \"Keys and Endpoints\" for your Azure OpenAI resource in the Azure Portal. Omit if using Azure Active Directory authentication (see below `Authentication using Microsoft Active Directory`)\n",
|
|
"* `SEARCH_ENDPOINT` - the AI Search endpoint. This URL be found on the \"Overview\" of your Search resource on the Azure Portal.\n",
|
|
"* `SEARCH_KEY` - the AI Search API key. Found under \"Keys\" for your Search resource in the Azure Portal.\n",
|
|
"* `SEARCH_INDEX_NAME` - the name of the index you created with your own data."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"import openai\n",
|
|
"import dotenv\n",
|
|
"\n",
|
|
"dotenv.load_dotenv()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Authentication\n",
|
|
"\n",
|
|
"The Azure OpenAI service supports multiple authentication mechanisms that include API keys and Azure Active Directory token credentials."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"use_azure_active_directory = False # Set this flag to True if you are using Azure Active Directory"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Authentication using API key\n",
|
|
"\n",
|
|
"To set up the OpenAI SDK to use an *Azure API Key*, we need to set `api_key` to a key associated with your endpoint (you can find this key in *\"Keys and Endpoints\"* under *\"Resource Management\"* in the [Azure Portal](https://portal.azure.com)). You'll also find the endpoint for your resource here."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"if not use_azure_active_directory:\n",
|
|
" endpoint = os.environ[\"AZURE_OPENAI_ENDPOINT\"]\n",
|
|
" api_key = os.environ[\"AZURE_OPENAI_API_KEY\"]\n",
|
|
" # set the deployment name for the model we want to use\n",
|
|
" deployment = \"<deployment-id-of-the-model-to-use>\"\n",
|
|
"\n",
|
|
" client = openai.AzureOpenAI(\n",
|
|
" base_url=f\"{endpoint}/openai/deployments/{deployment}/extensions\",\n",
|
|
" api_key=api_key,\n",
|
|
" api_version=\"2023-09-01-preview\"\n",
|
|
" )"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Authentication using Azure Active Directory\n",
|
|
"Let's now see how we can autheticate via Azure Active Directory. We'll start by installing the `azure-identity` library. This library will provide the token credentials we need to authenticate and help us build a token credential provider through the `get_bearer_token_provider` helper function. It's recommended to use `get_bearer_token_provider` over providing a static token to `AzureOpenAI` because this API will automatically cache and refresh tokens for you. \n",
|
|
"\n",
|
|
"For more information on how to set up Azure Active Directory authentication with Azure OpenAI, see the [documentation](https://learn.microsoft.com/azure/ai-services/openai/how-to/managed-identity)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"! pip install \"azure-identity>=1.15.0\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azure.identity import DefaultAzureCredential, get_bearer_token_provider\n",
|
|
"\n",
|
|
"if use_azure_active_directory:\n",
|
|
" endpoint = os.environ[\"AZURE_OPENAI_ENDPOINT\"]\n",
|
|
" api_key = os.environ[\"AZURE_OPENAI_API_KEY\"]\n",
|
|
" # set the deployment name for the model we want to use\n",
|
|
" deployment = \"<deployment-id-of-the-model-to-use>\"\n",
|
|
"\n",
|
|
" client = openai.AzureOpenAI(\n",
|
|
" base_url=f\"{endpoint}/openai/deployments/{deployment}/extensions\",\n",
|
|
" azure_ad_token_provider=get_bearer_token_provider(DefaultAzureCredential(), \"https://cognitiveservices.azure.com/.default\"),\n",
|
|
" api_version=\"2023-09-01-preview\"\n",
|
|
" )"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"> Note: the AzureOpenAI infers the following arguments from their corresponding environment variables if they are not provided:\n",
|
|
"\n",
|
|
"- `api_key` from `AZURE_OPENAI_API_KEY`\n",
|
|
"- `azure_ad_token` from `AZURE_OPENAI_AD_TOKEN`\n",
|
|
"- `api_version` from `OPENAI_API_VERSION`\n",
|
|
"- `azure_endpoint` from `AZURE_OPENAI_ENDPOINT`\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Chat completion model with your own data"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Setting the context"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In this example, we want our model to base its responses on Azure AI services documentation data. Following the [Quickstart](https://learn.microsoft.com/azure/ai-services/openai/use-your-data-quickstart?tabs=command-line&pivots=programming-language-studio) shared previously, we have added the [markdown](https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/ai-services/cognitive-services-and-machine-learning.md) file for the [Azure AI services and machine learning](https://learn.microsoft.com/azure/ai-services/cognitive-services-and-machine-learning) documentation page to our search index. The model is now ready to answer questions about Azure AI services and machine learning."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Code"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now we can use Azure on your own data with Chat Completions. Providing our search endpoint, key, and index name in `dataSources`, any questions posed to the model will now be grounded in our own data. An additional property, `context`, will be provided in the response to show the data the model referenced to answer the question."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"completion = client.chat.completions.create(\n",
|
|
" messages=[{\"role\": \"user\", \"content\": \"What are the differences between Azure Machine Learning and Azure AI services?\"}],\n",
|
|
" model=deployment,\n",
|
|
" extra_body={\n",
|
|
" \"dataSources\": [\n",
|
|
" {\n",
|
|
" \"type\": \"AzureCognitiveSearch\",\n",
|
|
" \"parameters\": {\n",
|
|
" \"endpoint\": os.environ[\"SEARCH_ENDPOINT\"],\n",
|
|
" \"key\": os.environ[\"SEARCH_KEY\"],\n",
|
|
" \"indexName\": os.environ[\"SEARCH_INDEX_NAME\"],\n",
|
|
" }\n",
|
|
" }\n",
|
|
" ]\n",
|
|
" }\n",
|
|
")\n",
|
|
"print(f\"{completion.choices[0].message.role}: {completion.choices[0].message.content}\")\n",
|
|
"\n",
|
|
"# `context` is in the model_extra for Azure\n",
|
|
"print(f\"\\nContext: {completion.choices[0].message.model_extra['context']['messages'][0]['content']}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"If you would prefer to stream the response from the model, you can pass the `stream=True` keyword argument:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"response = client.chat.completions.create(\n",
|
|
" messages=[{\"role\": \"user\", \"content\": \"What are the differences between Azure Machine Learning and Azure AI services?\"}],\n",
|
|
" model=deployment,\n",
|
|
" extra_body={\n",
|
|
" \"dataSources\": [\n",
|
|
" {\n",
|
|
" \"type\": \"AzureCognitiveSearch\",\n",
|
|
" \"parameters\": {\n",
|
|
" \"endpoint\": os.environ[\"SEARCH_ENDPOINT\"],\n",
|
|
" \"key\": os.environ[\"SEARCH_KEY\"],\n",
|
|
" \"indexName\": os.environ[\"SEARCH_INDEX_NAME\"],\n",
|
|
" }\n",
|
|
" }\n",
|
|
" ]\n",
|
|
" },\n",
|
|
" stream=True,\n",
|
|
")\n",
|
|
"\n",
|
|
"for chunk in response:\n",
|
|
" delta = chunk.choices[0].delta\n",
|
|
"\n",
|
|
" if delta.role:\n",
|
|
" print(\"\\n\"+ delta.role + \": \", end=\"\", flush=True)\n",
|
|
" if delta.content:\n",
|
|
" print(delta.content, end=\"\", flush=True)\n",
|
|
" if delta.model_extra.get(\"context\"):\n",
|
|
" print(f\"Context: {delta.model_extra['context']}\", end=\"\", flush=True)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": ".venv",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.0"
|
|
},
|
|
"orig_nbformat": 4
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|