openai-cookbook/examples/Zero-shot_classification_with_embeddings.ipynb

240 lines
62 KiB
Plaintext

{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Zero-shot classification with embeddings\n",
"\n",
"In this notebook we will classify the sentiment of reviews using embeddings and zero labeled data! The dataset is created in the [Get_embeddings_from_dataset Notebook](Get_embeddings_from_dataset.ipynb).\n",
"\n",
"We'll define positive sentiment to be 4- and 5-star reviews, and negative sentiment to be 1- and 2-star reviews. 3-star reviews are considered neutral and we won't use them for this example.\n",
"\n",
"We will perform zero-shot classification by embedding descriptions of each class and then comparing new samples to those class embeddings."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"# imports\n",
"import pandas as pd\n",
"import numpy as np\n",
"from ast import literal_eval\n",
"\n",
"from sklearn.metrics import classification_report\n",
"\n",
"# parameters\n",
"EMBEDDING_MODEL = \"text-embedding-ada-002\"\n",
"\n",
"# load data\n",
"datafile_path = \"data/fine_food_reviews_with_embeddings_1k.csv\"\n",
"\n",
"df = pd.read_csv(datafile_path)\n",
"df[\"embedding\"] = df.embedding.apply(literal_eval).apply(np.array)\n",
"\n",
"# convert 5-star rating to binary sentiment\n",
"df = df[df.Score != 3]\n",
"df[\"sentiment\"] = df.Score.replace({1: \"negative\", 2: \"negative\", 4: \"positive\", 5: \"positive\"})\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Zero-Shot Classification\n",
"To perform zero shot classification, we want to predict labels for our samples without any training. To do this, we can simply embed short descriptions of each label, such as positive and negative, and then compare the cosine distance between embeddings of samples and label descriptions. \n",
"\n",
"The highest similarity label to the sample input is the predicted label. We can also define a prediction score to be the difference between the cosine distance to the positive and to the negative label. This score can be used for plotting a precision-recall curve, which can be used to select a different tradeoff between precision and recall, by selecting a different threshold."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" precision recall f1-score support\n",
"\n",
" negative 0.61 0.88 0.72 136\n",
" positive 0.98 0.90 0.94 789\n",
"\n",
" accuracy 0.90 925\n",
" macro avg 0.79 0.89 0.83 925\n",
"weighted avg 0.92 0.90 0.91 925\n",
"\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA5S0lEQVR4nO3dd3xW5f3/8dc7A8IGAREZgogioKBGXFWQKg7EXUddWBy0dfy0ddW60FZarf1qtSoWKo6CSq1SRREVXLWVIRsExAgBlT0DIePz++OcxJt4JzkhuXNnfJ6PRx65z3XW57oD9+c+13XOdcnMcM4550pKSXYAzjnnaiZPEM455+LyBOGccy4uTxDOOefi8gThnHMuLk8Qzjnn4vIE4aqUpKGSPk52HFVJ0iWS3omw3VOS7qqOmKqDpCxJJ4Wv75X0QrJjctXLE4RDUkNJoyV9LWmrpNmSTkt2XFGEH2I7JG2T9J2kZyU1rcpzmNmLZjYownbDzez+qjx3EUkmaXtYz1WSHpGUmohzOVfEE4QDSANWAv2BFsBvgZcldUlmUBUwxMyaAocDmQTx70ZSWrVHVfX6hPXsD1wI/CzJ8VSpOvI3qlM8QTjMbLuZ3WtmWWZWaGZvAF8BR5S2j6ROkl6VtFbSekmPl7Ldo5JWStoiaaak42PW9ZM0I1z3naRHwvIMSS+Ex90kabqkdhHqsQp4C+gdHsck/VLSUmBpWHZGeIW0SdJ/JB1aXp1im80U+LOkNWHc8yQVne9ZSQ/EHO9qScskbZA0UdK+MetM0nBJS8NYnpCk8uoY1nMZ8AnQN+Z4e1KvbpLeD8vWSXpRUssoMZQk6azw/FskfSnp1LC8uJkqXC5uqpLUJXwfhklaAbwv6S1J15U49hxJ54ave0iaEr6nX0i6YE/iddF4gnA/EH4YHwgsKGV9KvAG8DXQBegAjC/lcNMJPsj2Av4BvCIpI1z3KPComTUHugEvh+VXEFzJdAJaA8OBHRHi7gScDnweU3w2cBTQU9JhwBjg2vC4TwMTwya2qHUaBJxA8P60AC4A1seJZSDwYLi+fXjcksc7AzgSODTc7pTy6hgeuwdwPLAsXN7TeimMcV/gYIL3+94oMZSIpx/wHHAL0JLg/cmqwCH6h+c/BRgHXBxz7J7AfsCbkpoAUwj+He0NXAT8NdzGJYAnCLcbSenAi8BYM1tcymb9CD5UbgmvPnaaWdyOaTN7wczWm1m+mf0JaAgcFK7OAw6Q1MbMtpnZf2PKWwMHmFmBmc00sy1lhP2apE3Ax8AHwO9j1j1oZhvMbAdwDfC0mf0vPO5YIBc4ugJ1ygOaAT0AmdkiM/smznaXAGPMbJaZ5QJ3AMeUaLYbaWabzGwFMJWYK4JSzJK0HVgETAP+GpbvUb3MbJmZTTGzXDNbCzxC8GFdUcPCuk4Jr0BXlfFvJ557w9h2AP8C+kraL1x3CfBq+B6eAWSZ2d/Df0+fA/8EfrIHMbsIPEG4YpJSgOeBXcB1MeVvKegc3SbpEoJvml+bWX6EY/5a0iJJm8MP8RZAm3D1MIJv4ovDZqQzwvLngcnAeEmrJf0xTFylOdvMWprZfmb2i/CDpsjKmNf7Ab8Km2E2hfF0IvgAjVQnM3sfeBx4AlgjaZSk5nE23ZfgW3vRftsIrjQ6xGzzbczrHKApgKQFMe/38THbHB5ucyHBVVGTytRLUjtJ4xV0em8BXuD7v01FdAK+3IP9ihT/jcxsK/AmwdUBBFcTL4av9wOOKlHPS4B9KnFuVwZPEA4I2taB0UA74DwzyytaZ2anmVnT8OdFgv/QnVVOp2L44XYrQfNJKzNrCWwmaNrAzJaa2cUEzQV/ACZIamJmeWZ2n5n1BI4l+OZ4+R5WLXa44pXA78JkUvTT2MzGRa1TGPdjZnYE0JMgwd0SZ7PVBB9oAITNI62BVRGO3yvm/f6oxDozs5eBT4G7K1mv3xO8P4eEzXyXEv5tKmglQRNhPNuBxjHL8T7MSw4pPQ64WNIxQAbB1VXReT4oUc+mZvbzPYjZReAJwhV5kqAdeEiJb+DxfAZ8A4yU1ERBp/JxcbZrBuQDa4E0SXcDxd+2JV0qqa2ZFQKbwuJCSSdKOiRsP99C0KxTWJnKhZ4Bhks6SoEmkgZLaha1TpKODPdPJ/jw21lKbOOAKyX1ldSQ4MP4f2aWVQX1ABgJXC1pn0rUqxmwDdgsqQPxE10Uownq+mNJKZI6hP0kALOBiySlS8oEzo9wvEkEyXUE8FL47wOCvpQDJV0WHi89/HscvIdxu3J4gnCE7b3XErSBf1uiOekHzKwAGAIcAKwAsgmaPUqaDLwNLCFobtnJ7k0+pwILJG0j6LC+KExO+wATCJLDIoJ+hecrWU3MbAZwNUET0UaCTt6hFaxTc4IP5I1hndYDD8U517vAXQRt5N8QfMO+qOR2lajLPOBDgr6FPa3XfQTNVpsJmnVe3cNYPgOuBP4cHusDvr96uoug7hvD8/0jwvFyw1hOit0+bH4aRPA+riZoovsDQb+WSwD5hEHOOefi8SsI55xzcXmCcM45F5cnCOecc3F5gnDOORdXnRkcq02bNtalS5dkh+Gcc7XKzJkz15lZ23jr6kyC6NKlCzNmzEh2GM45V6tI+rq0dd7E5JxzLi5PEM455+LyBOGccy4uTxDOOefi8gThnHMuroQlCEljFEzLOL+U9ZL0mIIpGedKOjxm3RUKpmJcKumKRMXonHOudIm8gniWYLTO0pwGdA9/riEYbhpJewH3EEyI0g+4R1KrBMbpnHMujoQ9B2FmH5aYXrGks4DnLBhO9r+SWkpqDwwAppjZBgBJUwgSzbhExJmzK5+nplVmMixXV3Ru3YTzj+iY7DCcqzGS+aBcB3afGyA7LCut/AckXUNw9UHnzp33KIgduwr4y9Rle7SvqzuKRr0f0qc9DdNSkxuMczVErX6S2sxGAaMAMjMz92hii9ZNG/LVg4OrNC5X+/x12jL++PYX+PQozn0vmQliFcFk50U6hmWrCJqZYsunVVtUzlWCmVFQaOQXGrl5heQVFpJfYOQVFJJXUEjOrgIA8goKyS8Myjfn5NEgLYW8AiO/sJCCQmNXfiFrt+XSrGEa+YVGfkFwzILCQlZt2sleTdKD5eLyYN8v12ynbbOGFBQaBWbF8RRYENvX63No17wh+YVGYbhNfoFRGG5XaNCtbVP+dkVmkt9JVxMkM0FMBK6TNJ6gQ3qzmX0jaTLw+5iO6UHAHckK0tUP32zaCcB1//ic1k0akF9o7CooZP22XHJ2FZCRnkJ+QVCWtW47bZo2JK+wkLx849stO0lLERLkF1q1XYU0TEshLUWkpoi01BRSBAWFxqJvt9C+RQYpCtalSKSkiFTBXk0asH77Ljq0bESKRFpK0TqRmiqWfreVdxd9Vz0VcDVewhKEpHEEVwJtJGUT3JmUDmBmTxFMTH46wfy5OQRz2mJmGyTdD0wPDzWiqMPauUSRgt8fL1vLzrxCOrRsRIO0FNJTxbad+bRp1pBmGWk0aZhGy84NyM0roEPLRqSnppCWKjbm7KLTXo1pkJpSXLZlRz77tswgLSVYbpCaQs6uAto2a0haikhLFWkpKUiQnppCRnq4b0pQnpICDdNSSU8NPujTU1NIDT/MU1KUkPfhkSlLWPLd0oQc29U+ibyL6eJy1hvwy1LWjQHGJCIu5+IZcVZvRpzVO9lhOFej+JPUzjnn4vIE4ZxzLi5PEM455+LyBOGccy4uTxDOOefi8gThnHMuLk8Qzjnn4vIE4ZxzLi5PEM455+Kq1aO5Oudqh5xd+UzP2oiFA1Vt2L6LLTvyyCswvlq/nVSJL77bSqP0VLbl5nNyz3YM798tyVE7TxDOuQrbsauAtVtz2Zizi6VrtpFXUMiKDTl8t3knkli2dhvrtuaSniqy1udEOmaHlo3YkVcQjExbUOgJogaIlCDCkVX3BXYAWWZWmNConHNJkVcQ/Nd+ffYqlq3ZxvJ12yksNFZv2sG23Hy+Wredwgij1XZo2YimDdNomJ7CuYd3oGFaKp32asTR+7cGoLDQaNk4nZaNG9CyUTppqd+3dne9403mZG/mijGf0bxROn++oM9u6131KTVBSGpBMJjexUADYC2QAbST9F/gr2Y2tVqidM5Vi8kLvgXgxvGzdys/tGMLUlPEoJ77kJISJIBOezWmUXpq8evmGek0b5SGVLmRZls1bsCG7bv4YMlaAH47+GDaNc+o1DHdninrCmIC8BxwvJltil0h6QjgMkn7m9noBMbnnKtGE4Yfy7/nrKZvp5a0b5lBmyYNEza0eGlm3XUyAP/43wp+86951Xput7tSE4SZnVzGupnAzIRE5JxLmr2aNOCKY7skOwxXQ5TbB6HgevESYH8zGyGpM7CPmX2W8Oicc/XWttw8AG6ZMJdWjdNZuHoLDdNTaJyexnPD+pGRnprkCOu+KJ3UfwUKgYHACGAr8E/gyATG5Zyr59Zt2wXAh2FfRI99mpG1Lodtufms376LfVtksG7bLtZs3cmqjTvYviufxd9u5au129m+K5+CQuOwzq247dQeFTpvfkEhG3PyWLN1J9kbd7B2ay5bd+azbM02tu7MI2dXAcvWbKNhegqpEuOvPZq9m9XNPpIoCeIoMztc0ucAZrZRUoMEx+Wcq+fuOK0HN/64O40bpBZ3fP9+0iJGfbic40a+X+7+bZs1ZMl327jt1B6YGWu25rJ60w627MznyzXbyFq/nfXbd5GTm8+KDTms3ZqLJDbvyCvzuM0z0jhon2as3rSTrzflsHLDjnqdIPIkpQIGIKktwRVFuSSdCjwKpAJ/M7ORJdbvRzC1aFtgA3CpmWWH6/4IDCZ42nsKcKNZdU0H75xLNkk0abj7R9Tx3dsw6sPlHNa5Jfu2bMS+LTLo1rYp7Vpk0LZpQzq0bETLxulI4ujfv8fa7bkc8JtJ5Jdxb+5B7ZqRmiK6t2tGk4ZpHLh3U5pmpNG6aUO6tG5M6yYN2bt5Q/Zq3GC3DvsPlqzlijF1u6U9SoJ4DPgXsLek3wHnA78tb6cwqTwBnAxkA9MlTTSzhTGbPQw8Z2ZjJQ0EHiS4O+pY4Djg0HC7j4H+wLRItXLO1UnHd29L1sjBkbYd0qc9z3z0FUfs14rOezVmnxYZdNqrMa2bNKBjq8a0b5lBs4aVvy23Lis3QZjZi5JmAj8GBJxtZosiHLsfsMzMlgNIGg+cBcQmiJ7AzeHrqcBrRacleOaiQXjOdOC7COd0zjkA7hzckzsH90x2GLVauY8nSnoM2MvMnjCzxyMmB4AOwMqY5eywLNYc4Nzw9TlAM0mtzexTgoTxTfgzOd55JV0jaYakGWvXro0YlnPOuSiiPL8+E/itpC8lPSwpswrP/2ugf9gB3h9YBRRIOgA4GOhIkFQGSjq+5M5mNsrMMs0ss23btlUYlnPOVUxhobF1Zx4rN+Twny/XMXXxGrbsLLvDu6aL0sQ0FhgraS/gPOAPkjqbWfdydl0FdIpZ7hiWxR57NeEVhKSmwHlmtknS1cB/zWxbuO4t4Bjgo2jVcs65xNq2Mx+A8578D3s1CYYHKen6gQfwq0EHAWBmFBqkVvOT6ZVRkdFcDwB6APsBUZqZpgPdJXUlSAwXAT+N3UBSG2BDOPjfHQR3NAGsAK6W9CBBH0R/4P8qEKtzziVUm6bB3f6pKaJt04Yc3L4ZHVo2Yr/WTejYqhE3jp/NX95fxkvTV7IpJ49dBYU0aZDKtFtOpG2zhkmOPpooT1L/kaB/4EvgJeD+kmMzxWNm+ZKuAyYT3OY6xswWSBoBzDCzicAA4EFJBnxIMDggBONADQTmEXRYv21m/65g3ZxzLmGO2r91mXdUFQ142K1tU3LyCti5q4AvvtvKum25dSdBECSGY8xsXUUPbmaTgEklyu6OeT2BIBmU3K8AuLai53POuZqiZPJ4e/43DH9hVpKi2TNlDffdw8wWEzQVdQ7HYCpmZrWrps455yqkrCuIm4FrgD/FWWcETUDOOefqqLKG+74mfHmame2MXSepbg484pxzCbJ83XYATnv0IybdcDw9922e5IjKF+U5iP9ELHPOOVeKU3rtU/z6X59nJzGS6EpNEJL2CWeOayTpMEmHhz8DgMbVFaBzztUF3do25a0bg+d9n/noq7jPTdQ0ZfVBnAIMJXjA7ZGY8q3AbxIYk3PO1UkHt2/OEfu1YubXG8nNL0h2OOUqqw+i6Anq88zsn9UYk3PO1Vk/OaIjM7/eSBkjkNcYZd3meqmZvQB0kXRzyfVm9kic3ZxzzpXh8xWbADhu5Pu8e3N/Dti7aXIDKkNZndRNwt9NgWZxfpxzzlXQ0OO6FL+e9fXG5AUSgerKJG2ZmZk2Y8aMZIfhnHPl+mjpWi4b/Rkpgsn/7wRy8wtZuzWXLm2a0LVNk/IPUIUkzTSzuKN0Rx2L6QFgB/A2wSxvN4XNT8455yro+O7B9ASFBif/+cPi8o6tGvHxbTXnGeQoz0EMMrMtwBlAFsGorrckMijnnKvrGqQGH7+P//Qwnrr0cACyN+6gy+1vcvj9U/huy86ydq8WUQbrK9pmMPCKmW32OVydc65ylvzutN2WB/VsxzsLg5mVN2zfxTebd9KueXIHrYiSIN6QtJigiennktoCyU9tzjlXh4y6POgGmLp4DVc+Oz3J0QTKbWIys9uBY4FMM8sDtgNnJTow55xzyRWlkzoduBQ4IWxa+gB4KsFxOeecS7IoTUxPAunAX8Ply8KyqxIVlHPOueSLkiCONLM+McvvS5qTqICcc87VDFFucy2Q1K1oQdL+QKRRpiSdKukLScsk3R5n/X6S3pM0V9I0SR1j1nWW9I6kRZIWSuoS5ZzOOeeqRpQEcQswNfwA/wB4H/hVeTtJSgWeAE4DegIXS+pZYrOHgefM7FBgBPBgzLrngIfM7GCgH7AmQqzOOVerZa0PJhY6+4lP+HxFcofiiHIX03tAd+AG4HrgIDObGuHY/YBlZrbczHYB4/nh3U89CRIOwNSi9WEiSTOzKWEM28wsJ8I5nXOuVhtw0N7Fr5ev3Z7ESCIkiHB60V8C9wL3EDwLEeXpjQ7Aypjl7LAs1hzg3PD1OUAzSa2BA4FNkl6V9Lmkh8IrkpKxXSNphqQZa9eujRCSc87VbF3bNOHDW05MdhhAtCam54BewF+Ax8PXz1fR+X8N9Jf0OdAfWEXQv5EGHB+uPxLYn2Dyot2Y2SgzyzSzzLZt21ZRSM455yDaXUy9zSy272CqpIUR9lsFdIpZ7hiWFTOz1YRXEJKaAueZ2SZJ2cBsM1sernsNOBoYHeG8zjnnqkCUK4hZko4uWpB0FBBlXO3pQHdJXSU1AC4CJsZuIKmNpKIY7gDGxOzbMhzWA2AgECUpOeecqyJREsQRwH8kZUnKAj4FjpQ0T9Lc0nYys3zgOmAysAh42cwWSBoh6cxwswHAF5KWAO2A34X7FhA0L70naR4g4Jk9qaBzztU2RjBPz69emcPc7E1JiyNKE9Ope3pwM5sETCpRdnfM6wnAhFL2nUIw94RzztUrjdK/vyfnsfeW8szlmSRjFO0ot7l+XdZPdQTpnHP1yd7NM8gaORiAdxetYXpWcp6HiNLE5JxzLglOPCjohn1lxspytkyMKE1MzjnnkuD+s3vzoz9M5ZWZ2WRv3AHA4EPbc+nR+1XL+T1BOOdcDdWxVePi158uX0+zjDTyCgqrLUFEamKSNKqsZeecc4kx+opM7j+rF1/+/nQO7diiWs8d9Qri6XKWnXPOJcCPD26XtHNHuoIws5llLTvnnKt7Sr2CkPRvCJ/WiMPMzixtnXPOudqvrCamh6stCuecczVOqQnCzD4oei2pEdDZzL6olqicc84lXZT5IIYAs4G3w+W+kiaWuZNzzrlaL0on9b0Es8NtAjCz2UDXhEXknHOuRoiSIPLMbHOJslI7r51zztUNUZ6DWCDpp0CqpKK5qf+T2LCcc84lW5QriOsJphnNBcYBW4D/l8CYnHPOxZG1LocZX2/kntfnsylnV8LPF2W47xwzuxP4MXCimd1pZjsTHplzzrndrNoUDNg39tOv+eyrDQk/X5S7mI4MZ3WbC8yTNEfSEQmPzDnn3G4+vWMgf/pJn2o7X5QmptHAL8ysi5l1AX4J/D2hUTnnnPuB9i0a0aVNEwDeW7Qm4eeLkiAKzOyjogUz+xjIj3JwSadK+kLSMkm3x1m/n6T3JM2VNE1SxxLrm0vKlvR4lPM551xd17ZpQwCmfpH4BFHWWEyHhy8/kPQ0QQe1ARcC08o7sKRU4AngZCAbmC5popktjNnsYeA5MxsraSDwIHBZzPr7gQ+jV8c55+q2zq0b02vf5rRvkZHwc5V1m+ufSizfE/M6ynMQ/YBlZrYcQNJ44CwgNkH0BG4OX08FXitaEfZztCN4gjszwvmcc85VobLGYjqxksfuAMROpJoNHFVimznAucCjwDlAM0mtgY0ECepS4KTSTiDpGuAagM6dO1cyXOecc7EiTRgkaTDBsxDF1zRmNqIKzv9r4HFJQwmaklYBBcAvgElmli2p1J3NbBQwCiAzM9Of7nbOuSpUboKQ9BTQGDgR+BtwPvBZhGOvAjrFLHcMy4qZ2WqCKwgkNQXOM7NNko4Bjpf0C6Ap0EDSNjP7QUe3c865xIhyF9OxZnY5sNHM7gOOAQ6MsN90oLukrpIaABcBu40CK6mNpKIY7gDGAJjZJWbWObyt9tcEHdmeHJxzrhpFSRA7wt85kvYF8oD25e1kZvnAdcBkYBHwspktkDRCUtFsdAOALyQtIeiQ/l0F43fOOZcgUfog3pDUEngImEVwB9PfohzczCYBk0qU3R3zegIwoZxjPAs8G+V8zjlXHyxYvYUFq7dgZpTVT1tZ5SYIM7s/fPlPSW8AGXGG/3bOOVfNNmzfRevwwblEKOtBuXPLWIeZvZqYkJxzzpVlxFm9uPv1BQk/T1lXEEPKWGeAJwjnnKvDynpQ7srqDMQ551w0u/ILg98FhQk9T5S7mJxzztUgnyxbB8ArM7ITeh5PEM45V8s8FM4JkZaauDuYwBOEc87VOk0bRholqdKizCjXWNJdkp4Jl7tLOiPxoTnnnCtLXn5ih6CLcgXxdyCXYIgNCMZTeiBhETnnnCuThXnhz+8uIWvd9oSdJ0qC6GZmfyQYYgMzywES2/DlnHOuVI0apBa/XrM1N2HniZIgdklqRDhJkKRuBFcUzjnnkuTFq0pOr1P1ovR03Eswq1snSS8CxwFDExiTc865GiDKWEzvSJoJHE3QtHSjma1LeGTOOeeSKsqEQf8G/gFMNLPE9YY455yrUaL0QTwMHA8slDRB0vmSMsrbyTnnXO1WboIwsw/M7BfA/sDTwAXAmkQH5pxzrnTfbdkJwOuzV5Wz5Z6L9DheeBfTEOBC4HBgbMIics45V66+nVoC8FUCn4OI0gfxMtCP4E6mx4EPzCyxQwg655wr0/5tm7JviwzaNU9ci3+UPojRBA/LDTezqRVJDpJOlfSFpGWSbo+zfj9J70maK2mapI5heV9Jn0paEK67MHqVnHOufli9eSf/+nwVZokZcqOsGeUGmtn7QBPgrJLznpY3o5ykVOAJ4GQgG5guaaKZLYzZ7GHgOTMbK2kg8CBwGZADXG5mSyXtC8yUNNnMNlW4hs45V8cVGiRiYNeyriD6h7+HxPmJMlhfP2CZmS03s13AeOCsEtv0BN4PX08tWm9mS8xsafh6NUGneNsI53TOuXrj5pMPTOjxy5pR7p7w5Qgz+yp2naSuEY7dAVgZs5wNlHw2fA5wLvAocA7QTFJrM1sfc65+QAPgy5InkHQNcA1A586dI4TknHMuqih9EP+MUzahis7/a6C/pM8JrlhWAQVFKyW1B54HrozX92Fmo8ws08wy27b1CwznnKtKZfVB9AB6AS0knRuzqjkQpdt8FdApZrljWFYsbD46NzxfU+C8on4GSc2BN4E7zey/Ec7nnHOuCpV1m+tBBH0NLQn6HYpsBa6OcOzpQPewOWoVcBHw09gNJLUBNoRXB3cAY8LyBsC/CDqwq+pqxTnnXAWU1QfxOvC6pGPM7NOKHtjM8iVdB0wGUoExZrZA0ghghplNBAYAD0oy4EPgl+HuFwAnAK0lDQ3LhprZ7IrG4ZxzddWked8A8NlXGzimW+sqP75Ku39W0q1m9kdJfyGcCyKWmd1Q5dFUQmZmps2YMSPZYTjnXLUZ/fFX3P/GQh489xAu7rdnN+pImmlmmfHWldXEtCj87Z+6zjlXAw0+pD33v7GQTTl5CTl+WU1M/w5/F4+7JCkFaGpmWxISjXPOucgKwhagP7y9mJ8P6Fblxy/3NldJ/5DUXFITYD7BsN+3VHkkzjnnKqRDy0YAdG3TJCHHj/IcRM/wiuFs4C2gK8FwGM4555LshAPb0qJRekKOHSVBpEtKJ0gQE80sjzid1s455+qWKAniaSCLYNC+DyXtB3gfhHPO1XHlzgdhZo8Bj8UUfS3pxMSF5JxzriaI0kndQtIjkmaEP38iuJpwzjlXh0VpYhpDMLzGBeHPFuDviQzKOedc8kWZk7qbmZ0Xs3yfpNkJisc551wNEeUKYoekHxUtSDoO2JG4kJxzztUEUa4ghgPPSWoRLm8ErkhcSM4552qCMhOEpL7AAQRDda8C8GE2nHOufii1iUnS3cDLwHkEE/dc6MnBOefqj7KuIC4E+ppZjqTWwNvAM9UTlnPOuWQrq5M618xyAMxsfTnbOuecq2PKuoLYX9LE8LWAbjHLmNmZCY3MOedcUpWVIM4qsfxwIgNxzjlXs5Q1YdAHlT24pFOBRwnmpP6bmY0ssX4/gie12wIbgEvNLDtcdwXw23DTB2InLnLOOZd4Zd3F9G9JQ8Khvkuu21/SCEk/K2P/VOAJ4DSgJ3CxpJ4lNnsYeM7MDgVGAA+G++4F3AMcBfQD7pHUqmJVc845VxlldTxfDRwPLJY0XdIkSe9LWk4wBPhMMxtTxv79gGVmttzMdgHj+WGzVU/g/fD11Jj1pwBTzGyDmW0EpgCnVqhmzjnnKqWsJqZvgVuBWyV1AdoTDLGxpOjupnJ0AFbGLGcTXBHEmgOcS9AMdQ7QLLylNt6+HUqeQNI1wDUAnTt3jhCSc865qCLdumpmWWb2qZnNjpgcovo10F/S50B/gqe1C6LubGajzCzTzDLbtm1bhWE555yLMhbTnloFdIpZ7hiWFTOz1QRXEEhqCpxnZpskrQIGlNh3WgJjdc45V0IiH36bDnSX1FVSA4LxnCbGbiCpjaSiGO4guKMJYDIwSFKrsHN6UFjmnHOumiQsQZhZPnAdwQf7IuBlM1sQ3v1U9JDdAOALSUuAdsDvwn03APcTJJnpwIiwzDnnXDUpt4kpnP/hXmC/cHsBZmb7l7evmU0CJpUouzvm9QRgQin7juH7KwrnnHPVLEofxGjgJmAmFehAds45V7tFSRCbzeythEfinHOuRomSIKZKegh4FcgtKjSzWQmLyjnnXNJFSRBFD7dlxpQZMLDqw3HOOVdTlJsgzOzE6gjEOedczVLuba6SWkh6RNKM8OdPklpUR3DOOeeSJ8pzEGOArcAF4c8W4O+JDMo551zyRemD6GZm58Us3ydpdoLicc45V0NEuYLYIelHRQvhg3M7EheSc865miDKFcTPgbFhv4MIZn4bmsignHPOJV+Uu5hmA30kNQ+XtyQ6KOecc8lXaoKQdKmZvSDp5hLlAJjZIwmOzTnnXBKVdQXRJPzdrDoCcc45V7OUNeXo0+Hv+6ovHOecczVFlAfl/iipuaR0Se9JWivp0uoIzjnnXPJEuc11UNgxfQaQBRwA3JLIoJxzziVflARR1Aw1GHjFzDYnMB7nnHM1RJQE8YakxcARwHuS2gI7oxxc0qmSvpC0TNLtcdZ3ljRV0ueS5ko6PSxPlzRW0jxJiyTdUZFKOeecq7xyE4SZ3Q4cC2SaWR6wHTirvP0kpQJPAKcBPYGLJfUssdlvCeaqPgy4CPhrWP4ToKGZHUKQmK6V1CVSjZxzzlWJsp6DGGhm70s6N6YsdpNXyzl2P2CZmS0P9x1PkFgWxmxjQPPwdQtgdUx5E0lpQCNgF8Eggc4556pJWc9B9AfeB4bEWWeUnyA6ACtjlrP5fvKhIvcC70i6nuC5i5PC8gkEyeQboDFwk5ltKOd8zjnnqlBZz0HcE/6+MoHnvxh41sz+JOkY4HlJvQmuPgqAfYFWwEeS3i26Giki6RrgGoDOnTsnMEznnKt/ojwH8XtJLWOWW0l6IMKxVwGdYpY7hmWxhgEvA5jZp0AG0Ab4KfC2meWZ2RrgE3af8pRwn1FmlmlmmW3bto0QknPOuaii3MV0mpltKlows43A6RH2mw50l9RVUgOCTuiJJbZZAfwYQNLBBAlibVg+MCxvAhwNLI5wTuecc1UkSoJIldSwaEFSI6BhGdsDYGb5wHXAZGARwd1KCySNkHRmuNmvgKslzQHGAUPNzAjufmoqaQFBovm7mc2tSMWcc85VTpT5IF4keP6haJrRK4GxUQ5uZpOASSXK7o55vRA4Ls5+2whudXXOOZckUeaD+EP4Db/oDqP7zWxyYsNyzjmXbFGuICBoIso3s3clNZbUzMy2JjIw55xzyRXlLqarCZ5LeDos6gC8lsCYnHPO1QBROql/SdBPsAXAzJYCeycyKOecc8kXJUHkmtmuooVw+AtLXEjOOedqgigJ4gNJvwEaSToZeAX4d2LDcs45l2xREsRtBA+vzQOuJbht9beJDMo551zylXkXUzhk9wIz6wE8Uz0hOeecqwnKvIIwswLgC0k+Ep5zztUzUZ6DaAUskPQZwWRBAJjZmaXv4pxzrraLkiDuSngUzjnnapyyZpTLAIYDBxB0UI8OB+BzzjlXD5TVBzGWYA6GeQTzSv+pWiJyzjlXI5TVxNTTzA4BkDQa+Kx6QnLOOVcTlHUFkVf0wpuWnHOu/inrCqKPpC3haxE8Sb0lfG1m1jzh0TnnnEuaUhOEmaVWZyDOOedqlihDbTjnnKuHok4YtEcknQo8CqQCfzOzkSXWdya4W6pluM3t4TSlSDqUYA6K5kAhcKSZ7azI+fPy8sjOzmbnzgrt5uq5jIwMOnbsSHp6erJDcS6pEpYgwnGcngBOBrKB6ZImhvNQF/kt8LKZPSmpJ8FAgF3CIcVfAC4zszmSWhPTaR5VdnY2zZo1o0uXLkiqdJ1c3WdmrF+/nuzsbLp27ZrscJxLqkQ2MfUDlpnZ8nA+ifHAWSW2MYIrBIAWwOrw9SBgrpnNATCz9eG4UBWyc+dOWrdu7cnBRSaJ1q1b+1WncyQ2QXQAVsYsZ4dlse4FLpWUTXD1cH1YfiBgkiZLmiXp1ngnkHSNpBmSZqxduzZuEJ4cXEX5vxnnAsnupL4YeNbMOgKnA89LSiFo+voRcEn4+xxJPy65s5mNMrNMM8ts27ZtdcbtnHN1XiITxCqgU8xyx7As1jDgZQAz+xTIANoQXG18aGbrzCyH4Ori8ATGmjDffvstF110Ed26deOII47g9NNPZ8mSJWRlZdG7d+8qO8/dd9/Nu+++C8BHH31Er1696Nu3L6tWreL888+v1LHNjIEDB7Jly5bistdeew1JLF68uLgsKyuLRo0a0bdvX3r27Mnw4cMpLCys1Llzc3O58MILOeCAAzjqqKPIysqKu92jjz5K79696dWrF//3f/9XXD5nzhyOOeYYDjnkEIYMGVJch3nz5jF06NBKxeZcXZfIBDEd6C6pq6QGwEXAxBLbrAB+DCDpYIIEsRaYDBwiqXHYYd0fWEgtY2acc845DBgwgC+//JKZM2fy4IMP8t1331X5uUaMGMFJJ50EwIsvvsgdd9zB7Nmz6dChAxMmTIh8nPz8Hz40P2nSJPr06UPz5t8/Gzlu3Dh+9KMfMW7cuN227datG7Nnz2bu3LksXLiQ1157bc8qFBo9ejStWrVi2bJl3HTTTdx2220/2Gb+/Pk888wzfPbZZ8yZM4c33niDZcuWAXDVVVcxcuRI5s2bxznnnMNDDz0EwCGHHEJ2djYrVqyoVHzO1WUJu4vJzPIlXUfwYZ8KjDGzBZJGADPMbCLwK+AZSTcRdFgPNTMDNkp6hCDJGDDJzN6sTDz3/XsBC1dvKX/DCui5b3PuGdKr1PVTp04lPT2d4cOHF5f16dMHYLdvwllZWVx22WVs3x5Mt/H4449z7LHH8s0333DhhReyZcsW8vPzefLJJzn22GMZNmwYM2bMQBI/+9nPuOmmmxg6dChnnHEGmzZt4uWXX2by5Mm89dZb/O53v+OMM85g/vz5FBQUcPvttzNt2jRyc3P55S9/ybXXXsu0adO46667aNWqFYsXL2bJkiW71ePFF1/kmmuuKV7etm0bH3/8MVOnTmXIkCHcd999P6h7Wloaxx57bPEH9Z56/fXXuffeewE4//zzue666zCz3foJFi1axFFHHUXjxo0B6N+/P6+++iq33norS5Ys4YQTTgDg5JNP5pRTTuH+++8HYMiQIYwfP55bb43bxeVcvZfQ5yDCZxomlSi7O+b1QuC4UvZ9geBW11pr/vz5HHHEEeVut/feezNlyhQyMjJYunQpF198MTNmzOAf//gHp5xyCnfeeScFBQXk5OQwe/ZsVq1axfz58wHYtGnTbse66qqr+PjjjznjjDM4//zzd0tEo0ePpkWLFkyfPp3c3FyOO+44Bg0aBMCsWbOYP39+3Fs7P/nkE55++uni5ddff51TTz2VAw88kNatWzNz5swf1DMnJ4f33nuPESNG/OB4xx9/PFu3bv1B+cMPP1x8FVRk1apVdOoUtFSmpaXRokUL1q9fT5s2bYq36d27N3feeSfr16+nUaNGTJo0iczMTAB69erF66+/ztlnn80rr7zCypXf3zeRmZnJyJEjPUE4V4qEJoiapKxv+smWl5fHddddx+zZs0lNTS3+Bn/kkUfys5/9jLy8PM4++2z69u3L/vvvz/Lly7n++usZPHhw8Qd8FO+88w5z584tbnLavHkzS5cupUGDBvTr16/U+/43bNhAs2bNipfHjRvHjTfeCMBFF13EuHHjihPEl19+Sd++fZHEWWedxWmnnfaD43300UeRY47i4IMP5rbbbmPQoEE0adKEvn37kpoajBQzZswYbrjhBu6//37OPPNMGjRoULzf3nvvzerVq0s7rHP1Xr1JEMnQq1evSO3/f/7zn2nXrh1z5syhsLCQjIwMAE444QQ+/PBD3nzzTYYOHcrNN9/M5Zdfzpw5c5g8eTJPPfUUL7/8MmPGjIkUj5nxl7/8hVNOOWW38mnTptGkSZNS90tLS6OwsJCUlBQ2bNjA+++/z7x585BEQUEBkorb9ov6IMpSkSuIDh06sHLlSjp27Eh+fj6bN2+mdevWP9h32LBhDBs2DIDf/OY3dOzYEYAePXrwzjvvALBkyRLefPP7lsqdO3fSqFGjMmN1rj5L9m2uddrAgQPJzc1l1KhRxWVz5879wTfozZs30759e1JSUnj++ecpKAieCfz6669p164dV199NVdddRWzZs1i3bp1FBYWct555/HAAw8wa9asyPGccsopPPnkk+TlBQ+lL1mypLjfoywHHXQQy5cvB2DChAlcdtllfP3112RlZbFy5Uq6du1aoauCjz76iNmzZ//gp2RyADjzzDMZO3Zs8bkHDhwY9zmFNWvWALBixQpeffVVfvrTn+5WXlhYyAMPPLBbf9CSJUuq9E4y5+oaTxAJJIl//etfvPvuu3Tr1o1evXpxxx13sM8+++y23S9+8QvGjh1Lnz59WLx4cfG3+WnTptGnTx8OO+wwXnrpJW688UZWrVrFgAED6Nu3L5deeikPPvhg5HiuuuoqevbsyeGHH07v3r259tpr4961VNLgwYOZNm0aEDQvnXPOObutP++8835wN1NVGTZsGOvXr+eAAw7gkUceYeTIYDiv1atXc/rpp+8WQ8+ePRkyZAhPPPEELVu2LI73wAMPpEePHuy7775ceeWVxftMnTqVwYMHJyRu5+oCBTcN1X6ZmZk2Y8aM3coWLVrEwQcfnKSI6o5vvvmGyy+/nClTpiQ7lCqTm5tL//79+fjjj0lL+2FLq//bcbXF4+8vJWdXAbee2mOP9pc008wy463zPghXrvbt23P11VezZcuW3Z6FqM1WrFjByJEj4yYH52qT6wZ2T9ix/X+Hi+SCCy5IdghVqnv37nTvnrj/WM7VBXW+D6KuNKG56uP/ZpwL1OkEkZGRwfr16/0/vIusaD6IoluNnavP6nQTU8eOHcnOzqa0ocCdi6doRjnn6rs6nSDS09N9VjDnnNtDdbqJyTnn3J7zBOGccy4uTxDOOefiqjNPUktaC3xdiUO0AdZVUTi1RX2rc32rL3id64vK1Hk/M4s7Z3OdSRCVJWlGaY+b11X1rc71rb7gda4vElVnb2JyzjkXlycI55xzcXmC+N6o8jepc+pbnetbfcHrXF8kpM7eB+Gccy4uv4JwzjkXlycI55xzcdWrBCHpVElfSFom6fY46xtKeilc/z9JXZIQZpWKUOebJS2UNFfSe5L2S0acVam8Osdsd54kk1Trb4mMUmdJF4R/6wWS/lHdMVa1CP+2O0uaKunz8N/36fGOU1tIGiNpjaT5payXpMfC92OupMMrfVIzqxc/QCrwJbA/0ACYA/Qssc0vgKfC1xcBLyU77mqo84lA4/D1z+tDncPtmgEfAv8FMpMddzX8nbsDnwOtwuW9kx13NdR5FPDz8HVPICvZcVeyzicAhwPzS1l/OvAWIOBo4H+VPWd9uoLoBywzs+VmtgsYD5xVYpuzgLHh6wnAjyWpGmOsauXW2cymmllOuPhfoLaPcx3l7wxwP/AHYGd1BpcgUep8NfCEmW0EMLM11RxjVYtSZwOK5shtAayuxviqnJl9CGwoY5OzgOcs8F+gpaT2lTlnfUoQHYCVMcvZYVncbcwsH9gMtK6W6BIjSp1jDSP4BlKblVvn8NK7k5m9WZ2BJVCUv/OBwIGSPpH0X0mnVlt0iRGlzvcCl0rKBiYB11dPaElT0f/v5arT80G46CRdCmQC/ZMdSyJJSgEeAYYmOZTqlkbQzDSA4CrxQ0mHmNmmZAaVYBcDz5rZnyQdAzwvqbeZFSY7sNqiPl1BrAI6xSx3DMvibiMpjeCydH21RJcYUeqMpJOAO4EzzSy3mmJLlPLq3AzoDUyTlEXQVjuxlndUR/k7ZwMTzSzPzL4ClhAkjNoqSp2HAS8DmNmnQAbBoHZ1VaT/7xVRnxLEdKC7pK6SGhB0Qk8ssc1E4Irw9fnA+xb2/tRS5dZZ0mHA0wTJoba3S0M5dTazzWbWxsy6mFkXgn6XM81sRnLCrRJR/m2/RnD1gKQ2BE1Oy6sxxqoWpc4rgB8DSDqYIEHU5fmHJwKXh3czHQ1sNrNvKnPAetPEZGb5kq4DJhPcATHGzBZIGgHMMLOJwGiCy9BlBJ1BFyUv4sqLWOeHgKbAK2F//AozOzNpQVdSxDrXKRHrPBkYJGkhUADcYma19uo4Yp1/BTwj6SaCDuuhtfkLn6RxBEm+Tdivcg+QDmBmTxH0s5wOLANygCsrfc5a/H4555xLoPrUxOScc64CPEE455yLyxOEc865uDxBOOeci8sThHPOubg8Qbg9JqlA0mxJ8yX9W1LLKj5+VnjPPpK2lbJNI0kfSEqV1EXSjjCmhZKeCp+crsg5MyU9Fr4eIOnYmHXDJV1emTqFx7lX0q/L2eZZSedX4JhdShvls6pJOrNo9FRJZ0vqGbNuRPjg5Z4cd7yk2vzwXp3jCcJVxg4z62tmvQmeG/llEmL4GfCqmRWEy1+aWV/gUIIRPM+uyMHMbIaZ3RAuDgCOjVn3lJk9V9mAazszm2hmI8PFswne56J1d5vZu3t46CeBWysZnqtCniBcVfmUcGAwSd0kvS1ppqSPJPUIy9tJ+pekOeHPsWH5a+G2CyRdU8HzXgK8XrIwHGzxP8AB4bfr9/X9nBedw/P+JLz6mSPpw7BsgKQ3FMwFMhy4KbwiOb7om7+kHpI+KzpXePx54esjwiuamZImq5zRNCVdLWl6GMM/JTWOWX2SpBmSlkg6I9w+VdJD4T5zJV1bkTdL0jZJfw7f6/cktQ3L+yoYxG9u+DdqFZbfoO/nCxkflg2V9Hj49zsTeCh8j7oVXfkomKvhlZjzDpD0Rvh6kKRPJc2S9IqkpuFmH4V1rjcP8NZ0niBcpUlKJRjSoOgp5VHA9WZ2BPBr4K9h+WPAB2bWh2Bc+wVh+c/CbTOBGyRFGkFXwRAL+5tZVpx1jcOY5gF/Acaa2aHAi2EcAHcDp4Tx7Pb0eHjMp4A/h1dJH8WsWww0kNQ1LLoQeElSeniu88P6jAF+V041XjWzI8MYFhGMH1SkC8Gw1oOBpyRlhOs3m9mRwJHA1TFxFNV9X0mTSjlfE4InjXsBHxA8jQvwHHBb+B7Niym/HTgsLB9e4j36D8Hf/JbwPfoyZvW7wFGSmoTLFwLjFTQZ/hY4ycwOB2YAN4fHKyR4CrhP6W+Xq06eIFxlNJI0G/gWaAdMCb8NHkswdMdsgnGeir5FDyRoRsDMCsxsc1h+g6Q5BOMidSL6IHJtgE0lyrqF5/0EeNPM3gKOAYpmUHse+FH4+hPgWUlXEwzXUBEvE3zoEf5+CTiIYCDAKWEMv6X8+TV6h1dZ8wiuhnrFnsPMCs1sKcG4ST2AQQTj7cwG/kcwHP1u75eZrTaz0mZPKwxjBXgB+JGkFkBLM/sgLB9LMDkNwFzgRQWj/eaXU5fYGPKBt4Eh4RXBYIIrvaMJmqQ+CetwBRA7i+EaYN+o53GJ5ZdyrjJ2mFnf8Nv6ZII+iGeBTWE/QLkkDQBOAo4xsxxJ0wgGVYt0/jjbfhn13GY2XNJRBB9eMyUdEfG8EHzIviLp1eBQtlTSIcACMzumAsd5FjjbzOZIGko4oF5RiCVDJpgt7Hozmxy7Qns+PW55Y+0MJkgWQ4A7wzpGNR64jqB/aoaZbZUkYIqZXVzKPhkEf1dXA/gVhKu0cEa6GwgGR8sBvpL0EyieJ7eoyeA9gmlNi9rSWxAMqb4xTA49CL5hRj3vRiA1bHopy3/4fuDFSwjaupHUzcz+Z2Z3E4zy2anEflsJhgePd+4vCQa9u4vvv5F/AbRVMPcAktIl9Yq3f4xmwDdh89QlJdb9RFKKpG4EU2t+QZCIfx5uj6QDY5pxokghGKkY4KfAx+GV3EZJx4fllwEfKLgDrJOZTQVuI/hbNS1xvFLfI4ImrMMJZrMbH5b9FzhO0gFh/E0kHRizz4FAtdyN5crnCcJVCTP7nKA54mKCD7phYbPRAr6fCvJG4MSwOWUmQVPD20CapEXASIIPkIp4h++bjEpzPXClpLkEH343huUPSZqn4PbQ/xDMaxzr38A5YQfs8fzQS8ClfD/nwC6CD98/hHWfTcxdUKW4i6Cp6BNgcYl1K4DPCGb5G25mO4G/AQuBWWHcT1OiJaCcPojtQL9w34HAiLD8CoL3Yy7QNyxPBV4I/16fA4/FmWBoPHCLpM/DRFYsvLPsDeC08DdmtpZgsqZx4bk+JWg6Q1I7gqvSb0uJ3VUzH83V1WoKpg+9ycwuS3YstYGkbWZW8iqgRlAwLPcWMxud7FhcwK8gXK1mZrOAqeGdVK5220TQQe5qCL+CcM45F5dfQTjnnIvLE4Rzzrm4PEE455yLyxOEc865uDxBOOeci+v/A7qNNh/O/Qb4AAAAAElFTkSuQmCC",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"from utils.embeddings_utils import cosine_similarity, get_embedding\n",
"from sklearn.metrics import PrecisionRecallDisplay\n",
"\n",
"def evaluate_embeddings_approach(\n",
" labels = ['negative', 'positive'],\n",
" model = EMBEDDING_MODEL,\n",
"):\n",
" label_embeddings = [get_embedding(label, engine=model) for label in labels]\n",
"\n",
" def label_score(review_embedding, label_embeddings):\n",
" return cosine_similarity(review_embedding, label_embeddings[1]) - cosine_similarity(review_embedding, label_embeddings[0])\n",
"\n",
" probas = df[\"embedding\"].apply(lambda x: label_score(x, label_embeddings))\n",
" preds = probas.apply(lambda x: 'positive' if x>0 else 'negative')\n",
"\n",
" report = classification_report(df.sentiment, preds)\n",
" print(report)\n",
"\n",
" display = PrecisionRecallDisplay.from_predictions(df.sentiment, probas, pos_label='positive')\n",
" _ = display.ax_.set_title(\"2-class Precision-Recall curve\")\n",
"\n",
"evaluate_embeddings_approach(labels=['negative', 'positive'], model=EMBEDDING_MODEL)\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"We can see that this classifier already performs extremely well. We used similarity embeddings, and the simplest possible label name. Let's try to improve on this by using more descriptive label names, and search embeddings."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" precision recall f1-score support\n",
"\n",
" negative 0.98 0.73 0.84 136\n",
" positive 0.96 1.00 0.98 789\n",
"\n",
" accuracy 0.96 925\n",
" macro avg 0.97 0.86 0.91 925\n",
"weighted avg 0.96 0.96 0.96 925\n",
"\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAzlUlEQVR4nO3deZyWVf3/8debRVEENUBEEEHUFFxIEbdMcldc03Lfcq3UfvqtvvqtXCjT0upRaW5J4hKGZIaKEiq4pcUiqxtoqIOoKJuKIMx8fn9cZ/BmvGfmGmbumWF4Px+Pecx9nXMtn3MP3Oc+51zXOYoIzMzMqmrV1AGYmVnz5ArCzMyKcgVhZmZFuYIwM7OiXEGYmVlRriDMzKwoVxDWoCSdKenZpo6jIUk6RdI/c+x3i6SfNkZMjUHSHEkHptdXSbqnqWOyxuUKwpC0vqQ7JL0p6SNJUyQd1tRx5ZE+xD6V9LGk9yTdKWmjhrxGRNwbEQfn2O+CiPhZQ167kqSQ9Ekq51xJv5HUuhTXMqvkCsIA2gBvA/sBGwM/AUZI6tWUQdXBkRGxEbArMIAs/tVIatPoUTW8XVI59wNOAL7dxPE0qBbyN2pRXEEYEfFJRFwVEXMioiIiHgb+C+xW3TGStpT0gKT5kj6UdGM1+/1O0tuSlkiaJGnfgryBkiamvPck/Salt5N0TzrvIkkTJHXNUY65wKPAjuk8Iel7kmYBs1LaEamFtEjSvyTtXFuZCrvNlPmtpPdT3NMlVV7vTkk/LzjfuZJmS1ogaZSkLQryQtIFkmalWG6SpNrKmMo5G3gO6F9wvjUpVx9JT6a0DyTdK2mTPDFUJenodP0lkl6XdGhKX9VNlbZXdVVJ6pXeh7MlvQU8KelRSRdWOfdUSd9Ir7eXNDa9p69K+taaxGv5uIKwL0gfxtsBM6vJbw08DLwJ9AK6A/dVc7oJZB9kXwL+AtwvqV3K+x3wu4joCPQBRqT0M8haMlsCnYALgE9zxL0lcDjwYkHyMcAeQF9JXwGGAuen894KjEpdbHnLdDDwNbL3Z2PgW8CHRWLZH7g25XdL5616viOA3YGd036H1FbGdO7tgX2B2Wl7TculFOMWwA5k7/dVeWKoEs9A4C7gh8AmZO/PnDqcYr90/UOA4cBJBefuC2wFPCKpPTCW7N/RZsCJwB/TPlYCriBsNZLaAvcCwyLilWp2G0j2ofLD1PpYFhFFB6Yj4p6I+DAiVkbEr4H1gS+n7BXANpI6R8THEfFCQXonYJuIKI+ISRGxpIawH5S0CHgWeAr4RUHetRGxICI+Bc4Dbo2If6fzDgOWA3vWoUwrgA7A9oAi4uWImFdkv1OAoRExOSKWA5cDe1XptrsuIhZFxFvAOApaBNWYLOkT4GVgPPDHlL5G5YqI2RExNiKWR8R84DdkH9Z1dXYq69jUAp1bw7+dYq5KsX0K/B3oL2mrlHcK8EB6D48A5kTEn9O/pxeBvwHfXIOYLQdXELaKpFbA3cBnwIUF6Y8qGxz9WNIpZN8034yIlTnO+QNJL0tanD7ENwY6p+yzyb6Jv5K6kY5I6XcDY4D7JL0j6Vep4qrOMRGxSURsFRHfTR80ld4ueL0V8D+pG2ZRimdLsg/QXGWKiCeBG4GbgPcl3SapY5FdtyD71l553MdkLY3uBfu8W/B6KbARgKSZBe/3vgX77Jr2OYGsVdS+PuWS1FXSfcoGvZcA9/D536YutgReX4PjKq36G0XER8AjZK0DyFoT96bXWwF7VCnnKcDm9bi21cAVhAFZ3zpwB9AVOC4iVlTmRcRhEbFR+rmX7D90T9UyqJg+3H5E1n2yaURsAiwm69ogImZFxElk3QW/BEZKah8RKyLi6ojoC+xN9s3x9DUsWuF0xW8D16TKpPJnw4gYnrdMKe7fR8RuQF+yCu6HRXZ7h+wDDYDUPdIJmJvj/P0K3u9nquRFRIwAngeuqGe5fkH2/uyUuvlOJf1t6uhtsi7CYj4BNizYLvZhXnVK6eHASZL2AtqRta4qr/NUlXJuFBHfWYOYLQdXEFbpZrJ+4COrfAMv5j/APOA6Se2VDSrvU2S/DsBKYD7QRtIVwKpv25JOldQlIiqARSm5QtLXJe2U+s+XkHXrVNSncMntwAWS9lCmvaTBkjrkLZOk3dPxbck+/JZVE9tw4CxJ/SWtT/Zh/O+ImNMA5QC4DjhX0ub1KFcH4GNgsaTuFK/o8riDrKwHSGolqXsaJwGYApwoqa2kAcDxOc43mqxyHQL8Nf37gGwsZTtJp6XztU1/jx3WMG6rhSsII/X3nk/WB/5ule6kL4iIcuBIYBvgLaCMrNujqjHAY8BrZN0ty1i9y+dQYKakj8kGrE9MldPmwEiyyuFlsnGFu+tZTCJiInAuWRfRQrJB3jPrWKaOZB/IC1OZPgSuL3Ktx4GfkvWRzyP7hn1i1f3qUZbpwNNkYwtrWq6rybqtFpN16zywhrH8BzgL+G0611N83nr6KVnZF6br/SXH+ZanWA4s3D91Px1M9j6+Q9ZF90uycS0rAXnBIDMzK8YtCDMzK8oVhJmZFeUKwszMinIFYWZmRbWYybE6d+4cvXr1auowzMzWKpMmTfogIroUy2sxFUSvXr2YOHFiU4dhZrZWkfRmdXnuYjIzs6JcQZiZWVGuIMzMrChXEGZmVpQrCDMzK6pkFYSkocqWZZxRTb4k/V7ZkozTJO1akHeGsqUYZ0k6o1QxmplZ9UrZgriTbLbO6hwGbJt+ziObbhpJXwKuJFsQZSBwpaRNSxinmZkVUbLnICLi6SrLK1Z1NHBXZNPJviBpE0ndgEHA2IhYACBpLFlFM7wUcS79bCW3jK/PYlhm1hzs2H1jDu7nxeUaUlM+KNed1dcGKEtp1aV/gaTzyFof9OzZc42C+PSzcv4wbvYaHWtmzUMEbN6xnSuIBrZWP0kdEbcBtwEMGDBgjRa26LTR+vz32sENGpeZNa5z75rI2JfeY8TEt4kIKgIq0u/5Hy1n0w3bUhGkvM/zI6CionB79WP3334zBvb+UlMXr8k0ZQUxl2yx80o9Utpcsm6mwvTxjRaVma11npv9AQA/GjmtXueRoJVEK8GK8mDmO4u5++w9GiLEtVJTVhCjgAsl3Uc2IL04IuZJGgP8omBg+mDg8qYK0syav6lXHkzZwk9p21rpAz77kJcEQNvWQimtMr+wMqjcrtwfoNdlj/DMrA/4v79Pp6IiKFv4KV06rM/KiqCiIiiviOx1ZL/LFiylc4f1s7yIVb9Xlmf7lFcEHdq15a6zB9KxXdumeqvqpGQVhKThZC2BzpLKyO5MagsQEbeQLUx+ONn6uUvJ1rQlIhZI+hkwIZ1qSOWAtZlZMW1bt6J35/YlOfc/Z75L61ZZxfHv/35Ij003pJWgTatWtGol2rQSrVqJTTZsy7zFn7LlphvSupWyH2V5rSXe/2gZk99axNyFn9Kx2zpeQUTESbXkB/C9avKGAkNLEZeZWR5zrmvYscnHZszjgnsmN+g5S22tHqQ2M1tbRLqNZtyr7zP/o+WsKK9gRXkFH3z8Ga1bKeuyKq9gZUWwojzo3bk9h+7YtHdluYIwM2sEcz5cCsCvHns11/7rt2nFqz8/rJQh1coVhJlZIzj/a1vTtrXo3bk9HTdoy3qtW9G2dSvWayNat2pF+/Va06Z1K9q0Fjc+OZuhz/63qUN2BWFm1hhatRLn7Lt1rn3btlbtOzUCz+ZqZmZFuQVhZtbMLFuRDVY/NmMey1ZUUF4RHNSva6M/P+EKwsysmRn70nsAq90We+Wyvpy1T+9GjcMVhJlZM/PwxV9l7Mz3+PLmHQA44g/PsqK8otHjcAVhZtbMdGzXluN26wHAJ8tXNlkcHqQ2M7OiXEGYmVlRubqY0syqWwCfAnMiovE7w8zMrFFVW0FI2phsMr2TgPWA+UA7oKukF4A/RsS4RonSzMwaXU0tiJHAXcC+EbGoMEPSbsBpkraOiDtKGJ+ZmTWRaiuIiDiohrxJwKSSRGRmZs1CrYPUypwq6Yq03VPSwNKHZmZmlRYuXdHo18xzF9Mfgb3IxiIAPgJuKllEZma2SuVqdjePf73Rr53nLqY9ImJXSS8CRMRCSeuVOC4zMwPatW0NQOeN1m/0a+dpQayQ1BoIAEldgFy3uUo6VNKrkmZLuqxI/laSnpA0TdJ4ST0K8n4laaaklyX9XoWriZuZrUMG79yNjTdo/Ikv8lQQvwf+Dmwm6RrgWeAXtR2UKpWbgMOAvsBJkvpW2e0G4K6I2BkYAlybjt0b2AfYGdgR2B3YL0+BzMxamkemzeP1+Z/wl3+/1ajXrbWCiIh7gR+RfXjPA46JiPtznHsgMDsi3oiIz4D7gKOr7NMXeDK9HleQH2TPXKwHrA+0Bd7LcU0zsxbnkgO3A+DGJ2fx2crGe045z11Mvwe+FBE3RcSNEfFyznN3B94u2C5LaYWmAt9Ir48FOkjqFBHPk1UY89LPmGLXlXSepImSJs6fPz9nWGZma5fvH7gtAO8sXsZjM99ttOvm6WKaBPxE0uuSbpA0oAGv/wNgvzQAvh8wFyiXtA2wA9CDrFLZX9K+VQ+OiNsiYkBEDOjSpUsDhmVm1rz86fTso3fZZ+WNds08XUzDIuJwsnGAV4FfSpqV49xzgS0LtnuktMJzvxMR34iIrwA/TmmLyFoTL0TExxHxMfAo2a22ZmbrpB226Njo16zLbK7bANsDWwGv5Nh/ArCtpN7pttgTgVGFO0jqLKkyhsuBoen1W2QtizaS2pK1LvJ2bZmZWQPIMwbxq9RiGALMAAZExJG1HRcRK4ELgTFkH+4jImKmpCGSjkq7DQJelfQa0BW4JqWPBF4HppONU0yNiIfqVDIzM6uXPDfWvg7sFREf1PXkETEaGF0l7YqC1yPJKoOqx5UD59f1emZm1nBqmu57+4h4hayrqKeknoX5ETG5+JFmZtYS1NSCuBQ4D/h1kbwA9i9JRGZmVq3yiEa7Vk3TfZ+XXh4WEcsK8yS1K2lUZma2mvLyrGK4/IHpnDSwZy17N4w8YxD/AnbNkWZmZiXSs9OGq14f/rtnANjvy13430O3L9k1q72LSdLmaeW4DSR9RdKu6WcQsGF1x5mZWWltsckGfPjJch6bUdqnqmtqQRwCnEn2gNtvCtI/Av6vhDGZmVkRc64bvOr1xcNfZPrcxSW9Xk1jEMOAYZKOi4i/lTQKMzNrdmq6zfXUiLgH6CXp0qr5EfGbIoeZmVkLUVMXU/v0e6PGCMTMzJqXmrqYbk2/r268cMzMrLnIOxdTR0lt0/Kg8yWd2hjBmZlZ08kzm+vBEbEEOAKYQzar6w9LGZSZmTW9PBVEZTfUYOD+iCjtfVVmZtYs5HmS+mFJrwCfAt+R1AVYVssxZma2lsuzotxlwN5k60CsAD4Bji51YGZm1rRqbUGkFd1OBb4mCeAp4JYSx2VmZk0sTxfTzUBb4I9p+7SUdk6pgjIzs6aXp4LYPSJ2Kdh+UtLUUgVkZmbNQ567mMol9anckLQ1UJ7n5JIOlfSqpNmSLiuSv1V6tmKapPGSehTk9ZT0T0kvS3pJUq881zQzs4aRp4L4ITAufYA/BTwJ/E9tB0lqDdwEHAb0BU6S1LfKbjcAd0XEzsAQ4NqCvLuA6yNiB2Ag8H6OWM3M1gmjpr7Dfz/4hHteeLNk18hzF9MTwLbAxcBFwJcjYlyOcw8EZkfEGxHxGXAfX7z7qS9ZhQMwrjI/VSRtImJsiuHjiFia45pmZuuEs/bpBcCLby0q2TXyTLXRDvgecBVwJdmzEHmWHO0OvF2wXZbSCk0FvpFeHwt0kNQJ2A5YJOkBSS9Kuj61SKrGdp6kiZImzp8/P0dIZmYtw5VH9qP7JhuU9Bp5upjuAvoBfwBuTK/vbqDr/wDYT9KLwH7AXLLxjTbAvil/d2BrssWLVhMRt0XEgIgY0KVLlwYKyczMIN9dTDtGROHYwThJL+U4bi6wZcF2j5S2SkS8Q2pBSNoIOC4iFkkqA6ZExBsp70FgT+COHNc1M7MGkKcFMVnSnpUbkvYAJuY4bgKwraTektYDTgRGFe4gqbOkyhguB4YWHLtJmtYDYH8gT6VkZmYNJE8FsRvwL0lzJM0Bngd2lzRd0rTqDoqIlcCFwBjgZWBERMyUNETSUWm3QcCrkl4DugLXpGPLybqXnpA0HRBw+5oU0MzM1kyeLqZD1/TkETEaGF0l7YqC1yOBkdUcOxbYeU2vbWbW0n26opyPlq0o2flrrSAionQ32ZqZ2Rpb8Mln/POl96ioCFq1UoOfP08Xk5mZNUNdO64PQJTo/K4gzMzWUqfssVVJz+8KwszMispVQUi6raZtMzNrefK2IG6tZdvMzFqYXBVEREyqadvMzFqeam9zlfQQNQyOR8RR1eWZmdnar6bnIG5otCjMzKzZqbaCiIinKl9L2gDoGRGvNkpUZmbW5PKsB3EkMAV4LG33lzSqxoPMzGytl2eQ+iqy1eEWAUTEFKB3ySIyM7NmIU8FsSIiFldJK9WT3WZm1kzkmc11pqSTgdaSKtem/ldpwzIzs6aWpwVxEdkyo8uB4cAS4P+VMCYzM2sG8kz3vRT4saRfZpvxUenDMjOzppbnLqbd06pu04DpkqZK2q30oZmZWVPKMwZxB/DdiHgGQNJXgT/j1d7MzFq0PGMQ5ZWVA0BEPAuszHNySYdKelXSbEmXFcnfStITkqZJGi+pR5X8jpLKJN2Y53pmZtZwapqLadf08ilJt5INUAdwAjC+thNLag3cBBwElAETJI2KiJcKdrsBuCsihknaH7gWOK0g/2fA0/mLY2ZmDaWmLqZfV9m+suB1nucgBgKzI+INAEn3AUcDhRVEX+DS9Hoc8GBlRhrn6Er2BPeAHNczM7MGVNNcTF+v57m7A28XbJcBe1TZZyrwDeB3wLFAB0mdgIVkFdSpwIHVXUDSecB5AD179qxnuGZmVijPIDWSBpM9C9GuMi0ihjTA9X8A3CjpTLKupLlAOfBdYHRElEmq9uCIuA24DWDAgAF+utvMrAHVWkFIugXYEPg68CfgeOA/Oc49F9iyYLtHSlslIt4ha0EgaSPguIhYJGkvYF9J3wU2AtaT9HFEfGGg28zMSiPPXUx7R8TpwMKIuBrYC9gux3ETgG0l9Za0HnAisNossJI6S6qM4XJgKEBEnBIRPSOiF1kr4y5XDmZmjStPBfFp+r1U0hbACqBbbQdFxErgQmAM8DIwIiJmShoiqXI1ukHAq5JeIxuQvqaO8ZuZWYnkGYN4WNImwPXAZLI7mP6U5+QRMRoYXSXtioLXI4GRtZzjTuDOPNczM7OGk2cupp+ll3+T9DDQrsj032Zm1sLU9KDcN2rIIyIeKE1IZmbWHNTUgjiyhrwAXEGYmbVgNT0od1ZjBmJmZs1LnruYzMxsHeQKwszMinIFYWZmReVZUW5DST+VdHva3lbSEaUPzczMmlKeFsSfgeVkU2xANp/Sz0sWkZmZNQt5Kog+EfErsik2iIilQPVTrJqZWYuQp4L4TNIGpEWCJPUha1GYmVkLlmcupqvIVnXbUtK9wD7AmSWMyczMmoE8czH9U9IkYE+yrqXvR8QHJY/MzMyaVJ4Fgx4C/gKMiohPSh+SmZnlMW9xthrDgk8+o0uH9Rv8/HnGIG4A9gVekjRS0vGS2tV2kJmZldb0udnE2k+8/F5Jzl9rBRERT0XEd4GtgVuBbwHvlyQaMzPL7fbTBwDpDqISyDNITbqL6UjgBGBXYFiJ4jEzs5xU4icO8oxBjAAGkt3JdCPwVERUlDQqMzNrcnnGIO4ge1jugogYV5fKQdKhkl6VNFvSZUXyt5L0hKRpksZL6pHS+0t6XtLMlHdC/iKZmVlDqGlFuf0j4kmgPXC0tHpTprYV5SS1Bm4CDgLKgAmSRkXESwW73QDcFRHDJO0PXAucBiwFTo+IWZK2ACZJGhMRi+pcQjMzWyM1dTHtBzxJ8ZXl8qwoNxCYHRFvAEi6DzgaKKwg+gKXptfjgAcBIuK1VReKeEfS+0AXYFEt1zQzswZS04pyV6aXQyLiv4V5knrnOHd34O2C7TJgjyr7TAW+AfwOOBboIKlTRHxYcK2BwHrA61UvIOk84DyAnj175gjJzMzyyjMG8bciaSMb6Po/APaT9CJZi2UuUF6ZKakbcDdwVrGxj4i4LSIGRMSALl26NFBIZmYGNY9BbA/0AzaW9I2CrI5Angfl5gJbFmz3SGmrRMQ7ZC0IJG0EHFc5ziCpI/AI8OOIeCHH9czMrAHVNAbxZeAIYBNWH4f4CDg3x7knANum7qi5wInAyYU7SOoMLEitg8uBoSl9PeDvZAPYDdVaMTOzOqhpDOIfwD8k7RURz9f1xBGxUtKFwBigNTA0ImZKGgJMjIhRwCDgWkkBPA18Lx3+LeBrQCdJZ6a0MyNiSl3jMDOzNVNTF9OP0kJBJ0s6qWp+RFxc28kjYjQwukraFQWvR1JkPCMi7gHuqe38ZmZWOjV1Mb2cfk9sjEDMzKx5qamL6aH0e9W8S5JaARtFxJJGiM3MzJpQrbe5SvqLpI6S2gMzyKb9/mHpQzMzs6aU5zmIvqnFcAzwKNCbbDoMMzNrwfJUEG0ltSWrIEZFxApKN/24mZk1E3kqiFuBOWST9j0taSvAYxBmZi1cnhXlfh8R3SPi8Mi8CXy9EWIzM7MaROrMefq1+SU5f55B6o0l/UbSxPTza7LWhJmZNaG2rbOP8EdnvFuS8+fpYhpKNr3Gt9LPEuDPJYnGzMxy67zR+nxtuy7033KTkpw/z5rUfSLiuILtqyVNKUk0ZmbWbORpQXwq6auVG5L2AT4tXUhmZtYc5GlBXADcJWnjtL0QOKN0IZmZWXNQYwUhqT+wDdlU3XMBPM2Gmdm6odouJklXACOA48gW7jnBlYOZ2bqjphbECUD/iFgqqRPwGHB744RlZmZNraZB6uURsRQgIj6sZV8zM2thampBbC1pVHotoE/BNhFxVEkjMzOzJlVTBXF0le0bShmImZk1LzUtGPRUfU8u6VDgd2RrUv8pIq6rkr8V2ZPaXYAFwKkRUZbyzgB+knb9eeHCRWZmVno13cX0kKQj01TfVfO2ljRE0rdrOL41cBNwGNAXOElS3yq73QDcFRE7A0OAa9OxXwKuBPYABgJXStq0bkUzM7P6qGng+VxgX+AVSRMkjZb0pKQ3yKYAnxQRQ2s4fiAwOyLeiIjPgPv4YrdVX+DJ9HpcQf4hwNiIWBARC4GxwKF1KpmZmdVLTV1M7wI/An4kqRfQjWyKjdcq726qRXfg7YLtMrIWQaGpwDfIuqGOBTqkW2qLHdu96gUknQecB9CzZ88cIZmZWV65bl2NiDkR8XxETMlZOeT1A2A/SS8C+5E9rV2e9+CIuC0iBkTEgC5dujRgWGZmlmcupjU1F9iyYLtHSlslIt4ha0EgaSPguIhYJGkuMKjKseNLGKuZmVVRyoffJgDbSuotaT2y+ZxGFe4gqbOkyhguJ7ujCWAMcLCkTdPg9MEpzczMGknJKoiIWAlcSPbB/jIwIiJmprufKh+yGwS8Kuk1oCtwTTp2AfAzskpmAjAkpZmZWSOptYsprf9wFbBV2l9ARMTWtR0bEaOB0VXSrih4PRIYWc2xQ/m8RWFmZo0szxjEHcAlwCTqMIBsZmZrtzwVxOKIeLTkkZiZWbOSp4IYJ+l64AFgeWViREwuWVRmZtbk8lQQlQ+3DShIC2D/hg/HzMyai1oriIj4emMEYmZmzUutt7lK2ljSbyRNTD+/lrRxYwRnZmZNJ89zEEOBj4BvpZ8lwJ9LGZSZmTW9PGMQfSLiuILtqyVNKVE8ZmbWTORpQXwq6auVG+nBuU9LF5KZmTUHeVoQ3wGGpXEHka38dmYpgzIzs6aX5y6mKcAukjqm7SWlDsrMzJpetRWEpFMj4h5Jl1ZJByAiflPi2MzMrAnV1IJon353aIxAzMysealpydFb0++rGy8cMzNrLvI8KPcrSR0ltZX0hKT5kk5tjODMzKzp5LnN9eA0MH0EMAfYBvhhKYMyM7Oml6eCqOyGGgzcHxGLSxiPmZk1E3kqiIclvQLsBjwhqQuwLM/JJR0q6VVJsyVdViS/p6Rxkl6UNE3S4Sm9raRhkqZLelnS5XUplJmZ1V+tFUREXAbsDQyIiBXAJ8DRtR0nqTVwE3AY0Bc4SVLfKrv9hGyt6q8AJwJ/TOnfBNaPiJ3IKqbzJfXKVSIzM2sQNT0HsX9EPCnpGwVphbs8UMu5BwKzI+KNdOx9ZBXLSwX7BNAxvd4YeKcgvb2kNsAGwGdkkwSamVkjqek5iP2AJ4Eji+QFtVcQ3YG3C7bL+HzxoUpXAf+UdBHZcxcHpvSRZJXJPGBD4JKIWFDL9czMrAHV9BzElen3WSW8/knAnRHxa0l7AXdL2pGs9VEObAFsCjwj6fHK1kglSecB5wH07NmzhGGama178jwH8QtJmxRsbyrp5znOPRfYsmC7R0ordDYwAiAingfaAZ2Bk4HHImJFRLwPPMfqS56SjrktIgZExIAuXbrkCMnMzPLKcxfTYRGxqHIjIhYCh+c4bgKwraTektYjG4QeVWWft4ADACTtQFZBzE/p+6f09sCewCs5rmlmZg0kTwXRWtL6lRuSNgDWr2F/ACJiJXAhMAZ4mexupZmShkg6Ku32P8C5kqYCw4EzIyLI7n7aSNJMsormzxExrS4FMzOz+smzHsS9ZM8/VC4zehYwLM/JI2I0MLpK2hUFr18C9ily3Mdkt7qamVkTybMexC/TN/zKO4x+FhFjShuWmZk1tTwtCMi6iFZGxOOSNpTUISI+KmVgZmbWtPLcxXQu2XMJt6ak7sCDJYzJzMyagTyD1N8jGydYAhARs4DNShmUmZk1vTwVxPKI+KxyI01/EaULyczMmoM8FcRTkv4P2EDSQcD9wEOlDcvMzJpangrif8keXpsOnE922+pPShmUmZk1vRrvYkpTds+MiO2B2xsnJDMzaw5qbEFERDnwqiTPhGdmto7J8xzEpsBMSf8hWywIgIg4qvpDzMxsbZengvhpyaMwM7Nmp6YV5doBFwDbkA1Q35Em4DMzs3VATWMQw8jWYJhOtq70rxslIjMzaxZq6mLqGxE7AUi6A/hP44RkZmbNQU0tiBWVL9y1ZGa27qmpBbGLpCXptciepF6SXkdEdCx5dGZm1mSqrSAionVjBmJmZs1Lnqk2zMxsHZR3waA1IulQ4HdAa+BPEXFdlfyeZHdLbZL2uSwtU4qkncnWoOgIVAC7R8Syulx/xYoVlJWVsWxZnQ6zdVy7du3o0aMHbdu2bepQzJpUySqINI/TTcBBQBkwQdKotA51pZ8AIyLiZkl9ySYC7JWmFL8HOC0ipkrqRMGgeV5lZWV06NCBXr16IaneZbKWLyL48MMPKSsro3fv3k0djlmTKmUX00BgdkS8kdaTuA84uso+QdZCANgYeCe9PhiYFhFTASLiwzQvVJ0sW7aMTp06uXKw3CTRqVMntzrNKG0F0R14u2C7LKUVugo4VVIZWevhopS+HRCSxkiaLOlHxS4g6TxJEyVNnD9/ftEgXDlYXfnfjFmmqQepTwLujIgewOHA3ZJakXV9fRU4Jf0+VtIBVQ+OiNsiYkBEDOjSpUtjxm1m1uKVsoKYC2xZsN0jpRU6GxgBEBHPA+2AzmStjacj4oOIWErWuti1hLGWzLvvvsuJJ55Inz592G233Tj88MN57bXXmDNnDjvuuGODXeeKK67g8ccfB+CZZ56hX79+9O/fn7lz53L88cfX69wRwf7778+SJUtWpT344INI4pVXXlmVNmfOHDbYYAP69+9P3759ueCCC6ioqKjXtZ9++ml23XVX2rRpw8iRI6vdb9KkSey0005ss802XHzxxURkq+IuWLCAgw46iG233ZaDDjqIhQsXAvDwww9zxRVX1Cs2s5aulBXEBGBbSb0lrQecCIyqss9bwAEAknYgqyDmA2OAnSRtmAas9wNeYi0TERx77LEMGjSI119/nUmTJnHttdfy3nvvNfi1hgwZwoEHHgjAvffey+WXX86UKVPo3r17jR+sVa1c+cWH5kePHs0uu+xCx46fPxs5fPhwvvrVrzJ8+PDV9u3Tpw9Tpkxh2rRpvPTSSzz44INrVqCkZ8+e3HnnnZx88sk17ved73yH22+/nVmzZjFr1iwee+wxAK677joOOOAAZs2axQEHHMB112U30g0ePJiHHnqIpUuX1is+s5asZHcxRcRKSReSfdi3BoZGxExJQ4CJETEK+B/gdkmXkA1YnxnZV7+Fkn5DVskEMDoiHqlPPFc/NJOX3llS+4510HeLjlx5ZL9q88eNG0fbtm254IILVqXtsssuQPZtu9KcOXM47bTT+OSTbLmNG2+8kb333pt58+ZxwgknsGTJElauXMnNN9/M3nvvzdlnn83EiRORxLe//W0uueQSzjzzTI444ggWLVrEiBEjGDNmDI8++ijXXHMNRxxxBDNmzKC8vJzLLruM8ePHs3z5cr73ve9x/vnnM378eH7605+y6aab8sorr/Daa6+tVo57772X8847b9X2xx9/zLPPPsu4ceM48sgjufrqq79Q9jZt2rD33nsze/bsNXpvK/Xq1QuAVq2q/y4zb948lixZwp577gnA6aefzoMPPshhhx3GP/7xD8aPHw/AGWecwaBBg/jlL3+JJAYNGsTDDz/Mt771rXrFaNZSlfQ5iPRMw+gqaVcUvH4J2KeaY+8hu9V1rTVjxgx22223WvfbbLPNGDt2LO3atWPWrFmcdNJJTJw4kb/85S8ccsgh/PjHP6a8vJylS5cyZcoU5s6dy4wZMwBYtGjRauc655xzePbZZzniiCM4/vjjV6uI7rjjDjbeeGMmTJjA8uXL2WeffTj44IMBmDx5MjNmzCh6a+dzzz3Hrbfeumr7H//4B4ceeijbbbcdnTp1YtKkSV8o59KlS3niiScYMmTIF86377778tFHH30h/YYbbljVCqqLuXPn0qNHj1XbPXr0YO7crDfzvffeo1u3bgBsvvnmq7XeBgwYwDPPPOMKwqwaJa0gmpOavuk3tRUrVnDhhRcyZcoUWrduveob/O677863v/1tVqxYwTHHHEP//v3ZeuuteeONN7jooosYPHjwqg/4PP75z38ybdq0VV1OixcvZtasWay33noMHDiw2vv+FyxYQIcOHVZtDx8+nO9///sAnHjiiQwfPnxVBfH666/Tv39/JHH00Udz2GGHfeF8zzzzTO6YG5Kk1e5Q2myzzXjnnXdqOMJs3bbOVBBNoV+/frn6/3/729/StWtXpk6dSkVFBe3atQPga1/7Gk8//TSPPPIIZ555Jpdeeimnn346U6dOZcyYMdxyyy2MGDGCoUOH5oonIvjDH/7AIYccslr6+PHjad++fbXHtWnThoqKClq1asWCBQt48sknmT59OpIoLy9HEtdffz3w+RhETRq6BdG9e3fKyspWbZeVldG9e3ZHddeuXZk3bx7dunVj3rx5bLbZZqv2W7ZsGRtssEGdr2e2rmjq21xbtP3335/ly5dz2223rUqbNm3aF75BL168mG7dutGqVSvuvvtuysuzZwLffPNNunbtyrnnnss555zD5MmT+eCDD6ioqOC4447j5z//OZMnT84dzyGHHMLNN9/MihXZQ+mvvfbaqnGPmnz5y1/mjTfeAGDkyJGcdtppvPnmm8yZM4e3336b3r1716lV8MwzzzBlypQv/KxJ5QDQrVs3OnbsyAsvvEBEcNddd3H00dkzmUcddRTDhg0DYNiwYavSISt/Q95JZtbSuIIoIUn8/e9/5/HHH6dPnz7069ePyy+/nM0333y1/b773e8ybNgwdtllF1555ZVV3+bHjx/PLrvswle+8hX++te/8v3vf5+5c+cyaNAg+vfvz6mnnsq1116bO55zzjmHvn37suuuu7Ljjjty/vnnF71rqarBgwevGugdPnw4xx577Gr5xx133BfuZmooEyZMoEePHtx///2cf/759Ov3eVdh//79V73+4x//yDnnnMM222xDnz59VnVtXXbZZYwdO5Ztt92Wxx9/nMsuu2zVMePGjWPw4MElidusJVDl/eJruwEDBsTEiRNXS3v55ZfZYYcdmiiilmPevHmcfvrpjB07tqlDaTDvvfceJ598Mk888UTRfP/bsbXFjU/OYuln5fzo0O3X6HhJkyJiQLE8j0FYrbp168a5557LkiVLVnsWYm321ltv8etfe5l1W/tduP+2JTu3KwjLpaXdCrr77rs3dQhmzV6LH4NoKV1o1nj8b8Ys06IriHbt2vHhhx/6P7zlVrkeROWtxmbrshbdxdSjRw/Kysqobipws2IqV5QzW9e16Aqibdu2XhXMzGwNteguJjMzW3OuIMzMrChXEGZmVlSLeZJa0nzgzXqcojPwQQOFs7ZY18q8rpUXXOZ1RX3KvFVEFF2zucVUEPUlaWJ1j5u3VOtamde18oLLvK4oVZndxWRmZkW5gjAzs6JcQXzuttp3aXHWtTKva+UFl3ldUZIyewzCzMyKcgvCzMyKcgVhZmZFrVMVhKRDJb0qabaky4rkry/pryn/35J6NUGYDSpHmS+V9JKkaZKekLRVU8TZkGorc8F+x0kKSWv9LZF5yizpW+lvPVPSXxo7xoaW4992T0njJL2Y/n0f3hRxNhRJQyW9L2lGNfmS9Pv0fkyTtGu9LxoR68QP0Bp4HdgaWA+YCvStss93gVvS6xOBvzZ13I1Q5q8DG6bX31kXypz26wA8DbwADGjquBvh77wt8CKwadrerKnjboQy3wZ8J73uC8xp6rjrWeavAbsCM6rJPxx4FBCwJ/Dv+l5zXWpBDARmR8QbEfEZcB9wdJV9jgaGpdcjgQMkqRFjbGi1ljkixkXE0rT5ArC2z3Od5+8M8DPgl8CyxgyuRPKU+VzgpohYCBAR7zdyjA0tT5kDqFwjd2PgnUaMr8FFxNPAghp2ORq4KzIvAJtI6lafa65LFUR34O2C7bKUVnSfiFgJLAY6NUp0pZGnzIXOJvsGsjartcyp6b1lRDzSmIGVUJ6/83bAdpKek/SCpEMbLbrSyFPmq4BTJZUBo4GLGie0JlPX/++1atHrQVh+kk4FBgD7NXUspSSpFfAb4MwmDqWxtSHrZhpE1kp8WtJOEbGoKYMqsZOAOyPi15L2Au6WtGNEVDR1YGuLdakFMRfYsmC7R0oruo+kNmTN0g8bJbrSyFNmJB0I/Bg4KiKWN1JspVJbmTsAOwLjJc0h66sdtZYPVOf5O5cBoyJiRUT8F3iNrMJYW+Up89nACICIeB5oRzapXUuV6/97XaxLFcQEYFtJvSWtRzYIParKPqOAM9Lr44EnI43+rKVqLbOkrwC3klUOa3u/NNRS5ohYHBGdI6JXRPQiG3c5KiImNk24DSLPv+0HyVoPSOpM1uX0RiPG2NDylPkt4AAASTuQVRAtef3hUcDp6W6mPYHFETGvPidcZ7qYImKlpAuBMWR3QAyNiJmShgATI2IUcAdZM3Q22WDQiU0Xcf3lLPP1wEbA/Wk8/q2IOKrJgq6nnGVuUXKWeQxwsKSXgHLghxGx1raOc5b5f4DbJV1CNmB95tr8hU/ScLJKvnMaV7kSaAsQEbeQjbMcDswGlgJn1fuaa/H7ZWZmJbQudTGZmVkduIIwM7OiXEGYmVlRriDMzKwoVxBmZlaUKwhbY5LKJU2RNEPSQ5I2aeDzz0n37CPp42r22UDSU5JaS+ol6dMU00uSbklPTtflmgMk/T69HiRp74K8CySdXp8ypfNcJekHtexzp6Tj63DOXtXN8tnQJB1VOXuqpGMk9S3IG5IevFyT894naW1+eK/FcQVh9fFpRPSPiB3Jnhv5XhPE8G3ggYgoT9uvR0R/YGeyGTyPqcvJImJiRFycNgcBexfk3RIRd9U34LVdRIyKiOvS5jFk73Nl3hUR8fganvpm4Ef1DM8akCsIayjPkyYGk9RH0mOSJkl6RtL2Kb2rpL9Lmpp+9k7pD6Z9Z0o6r47XPQX4R9XENNniv4Bt0rfrJ/X5mhc903W/mVo/UyU9ndIGSXpY2VogFwCXpBbJvpXf/CVtL+k/lddK55+eXu+WWjSTJI1RLbNpSjpX0oQUw98kbViQfaCkiZJek3RE2r+1pOvTMdMknV+XN0vSx5J+m97rJyR1Sen9lU3iNy39jTZN6Rfr8/VC7ktpZ0q6Mf39jgKuT+9Rn8qWj7K1Gu4vuO4gSQ+n1wdLel7SZEn3S9oo7fZMKvM68wBvc+cKwupNUmuyKQ0qn1K+DbgoInYDfgD8MaX/HngqInYhm9d+Zkr/dtp3AHCxpFwz6CqbYmHriJhTJG/DFNN04A/AsIjYGbg3xQFwBXBIime1p8fTOW8BfptaSc8U5L0CrCepd0o6AfirpLbpWsen8gwFrqmlGA9ExO4phpfJ5g+q1ItsWuvBwC2S2qX8xRGxO7A7cG5BHJVl30LS6Gqu157sSeN+wFNkT+MC3AX8b3qPphekXwZ8JaVfUOU9+hfZ3/yH6T16vSD7cWAPSe3T9gnAfcq6DH8CHBgRuwITgUvT+SrIngLepfq3yxqTKwirjw0kTQHeBboCY9O3wb3Jpu6YQjbPU+W36P3JuhGIiPKIWJzSL5Y0lWxepC3JP4lcZ2BRlbQ+6brPAY9ExKPAXkDlCmp3A19Nr58D7pR0Ltl0DXUxguxDj/T7r8CXySYCHJti+Am1r6+xY2plTSdrDfUrvEZEVETELLJ5k7YHDiabb2cK8G+y6ehXe78i4p2IqG71tIoUK8A9wFclbQxsEhFPpfRhZIvTAEwD7lU22+/KWspSGMNK4DHgyNQiGEzW0tuTrEvquVSGM4DCVQzfB7bIex0rLTflrD4+jYj+6dv6GLIxiDuBRWkcoFaSBgEHAntFxFJJ48kmVct1/SL7vp732hFxgaQ9yD68JknaLed1IfuQvV/SA9mpYpaknYCZEbFXHc5zJ3BMREyVdCZpQr3KEKuGTLZa2EURMaYwQ2u+PG5tc+0MJqssjgR+nMqY133AhWTjUxMj4iNJAsZGxEnVHNOO7O9qzYBbEFZvaUW6i8kmR1sK/FfSN2HVOrmVXQZPkC1rWtmXvjHZlOoLU+WwPdk3zLzXXQi0Tl0vNfkXn0+8eApZXzeS+kTEvyPiCrJZPresctxHZNODF7v262ST3v2Uz7+Rvwp0Ubb2AJLaSupX7PgCHYB5qXvqlCp535TUSlIfsqU1XyWriL+T9kfSdgXdOHm0IpupGOBk4NnUklsoad+UfhrwlLI7wLaMiHHA/5L9rTaqcr5q3yOyLqxdyVazuy+lvQDsI2mbFH97SdsVHLMd0Ch3Y1ntXEFYg4iIF8m6I04i+6A7O3UbzeTzpSC/D3w9dadMIutqeAxoI+ll4DqyD5C6+CefdxlV5yLgLEnTyD78vp/Sr5c0Xdntof8iW9e40EPAsWkAdl++6K/AqXy+5sBnZB++v0xln0LBXVDV+ClZV9FzwCtV8t4C/kO2yt8FEbEM+BPwEjA5xX0rVXoCahmD+AQYmI7dHxiS0s8gez+mAf1TemvgnvT3ehH4fZEFhu4DfijpxVSRrZLuLHsYOCz9JiLmky3WNDxd63myrjMkdSVrlb5bTezWyDybq63VlC0feklEnNbUsawNJH0cEVVbAc2Csmm5l0TEHU0di2XcgrC1WkRMBsalO6ls7baIbIDcmgm3IMzMrCi3IMzMrChXEGZmVpQrCDMzK8oVhJmZFeUKwszMivr/42vvGa3GwGsAAAAASUVORK5CYII=",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"evaluate_embeddings_approach(labels=['An Amazon review with a negative sentiment.', 'An Amazon review with a positive sentiment.'])\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Using the search embeddings and descriptive names leads to an additional improvement in performance."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" precision recall f1-score support\n",
"\n",
" negative 0.98 0.73 0.84 136\n",
" positive 0.96 1.00 0.98 789\n",
"\n",
" accuracy 0.96 925\n",
" macro avg 0.97 0.86 0.91 925\n",
"weighted avg 0.96 0.96 0.96 925\n",
"\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAze0lEQVR4nO3dd5hV1dn+8e9NUSygBhCRIogaBQtRxB6JXbFr7L2gie2nb8yLb2IjMWosuWLsRiKowSAxBhVFFLBFE4pUG2hQB1FRmkoRZp7fH3sPHsYzM3uYOXOG4f5c11yz99rtWWdgr7PW2nstRQRmZmYVNSl2AGZm1jC5gDAzs7xcQJiZWV4uIMzMLC8XEGZmlpcLCDMzy8sFhNUpSWdJerXYcdQlSadKej7DfvdKuro+YqoPkmZJOiBdvk7SI8WOyeqXCwhD0rqSHpT0oaSvJE2SdGix48oivYktkfS1pM8kPSRpw7q8RkQ8GhEHZdjvwoj4TV1eu5ykkPRNms/Zkm6X1LQQ1zIr5wLCAJoBHwP7AhsBvwaGSupSzKBq4IiI2BDYGehFEv8qJDWr96jq3k5pPvcFTgTOKXI8daqR/I0aFRcQRkR8ExHXRcSsiCiLiKeB/wK7VHaMpE6SnpA0V9KXku6sZL8/SvpY0iJJEyTtk7Ott6Tx6bbPJN2epreQ9Eh63gWSxklqlyEfs4Fnge3T84SkiyTNAGakaYenNaQFkv4lacfq8pTbbKbEHyR9nsY9VVL59R6S9Nuc850vaaakeZKGS9o8Z1tIulDSjDSWuySpujym+ZwJvAb0zDnf6uSrm6TRadoXkh6VtHGWGCqSdFR6/UWS3pd0SJq+spkqXV/ZVCWpS/o5nCvpI2C0pGclXVzh3JMlHZsubytpVPqZvivphNWJ17JxAWHfk96MtwGmV7K9KfA08CHQBegAPFbJ6caR3Mh+APwVeFxSi3TbH4E/RkQroBswNE0/k6Qm0wloDVwILMkQdyfgMODNnOSjgd2A7pJ+BAwELkjPex8wPG1iy5qng4Afk3w+GwEnAF/miWU/4MZ0e/v0vBXPdziwK7Bjut/B1eUxPfe2wD7AzHR9dfOlNMbNge1IPu/rssRQIZ7ewGDgSmBjks9nVg1OsW96/YOBIcDJOefuDmwBPCNpA2AUyb+jTYGTgLvTfawAXEDYKiQ1Bx4FBkXEO5Xs1pvkpnJlWvtYGhF5O6Yj4pGI+DIiVkTEbcC6wA/TzcuBrSS1iYivI+KNnPTWwFYRURoREyJiURVhPylpAfAq8BLwu5xtN0bEvIhYAvQD7ouIf6fnHQQsA3avQZ6WAy2BbQFFxNsRMSfPfqcCAyNiYkQsA64C9qjQbHdTRCyIiI+AMeTUCCoxUdI3wNvAWODuNH218hURMyNiVEQsi4i5wO0kN+uaOjfN66i0Bjq7in87+VyXxrYE+AfQU9IW6bZTgSfSz/BwYFZE/CX99/Qm8Hfgp6sRs2XgAsJWktQEeBj4Frg4J/1ZJZ2jX0s6leSb5ocRsSLDOX8h6W1JC9Ob+EZAm3TzuSTfxN9Jm5EOT9MfBkYCj0n6RNLv04KrMkdHxMYRsUVE/Dy90ZT7OGd5C+B/0maYBWk8nUhuoJnyFBGjgTuBu4DPJd0vqVWeXTcn+dZeftzXJDWNDjn7fJqzvBjYEEDS9JzPe5+cfXZO9zmRpFa0QW3yJamdpMeUdHovAh7hu79NTXQC3l+N48qt/BtFxFfAMyS1A0hqE4+my1sAu1XI56nAZrW4tlXBBYQBSds68CDQDjguIpaXb4uIQyNiw/TnUZL/0J1VTadienP7JUnzySYRsTGwkKRpg4iYEREnkzQX3AwMk7RBRCyPiOsjojuwJ8k3xzNWM2u5wxV/DNyQFiblP+tHxJCseUrjviMidgG6kxRwV+bZ7ROSGxoAafNIa2B2hvP3yPm8X6mwLSJiKPA6cE0t8/U7ks9nh7SZ7zTSv00NfUzSRJjPN8D6Oev5buYVh5QeApwsaQ+gBUntqvw6L1XI54YR8bPViNkycAFh5e4haQc+osI38Hz+A8wBbpK0gZJO5b3y7NcSWAHMBZpJugZY+W1b0mmS2kZEGbAgTS6T9BNJO6Tt54tImnXKapO51APAhZJ2U2IDSX0ltcyaJ0m7psc3J7n5La0ktiHA2ZJ6SlqX5Gb874iYVQf5ALgJOF/SZrXIV0vga2ChpA7kL+iyeJAkr/tLaiKpQ9pPAjAJOElSc0m9gOMznG8ESeE6APhb+u8Dkr6UbSSdnp6vefr32G4147ZquIAw0vbeC0jawD+t0Jz0PRFRChwBbAV8BJSQNHtUNBJ4DniPpLllKas2+RwCTJf0NUmH9Ulp4bQZMIykcHibpF/h4Vpmk4gYD5xP0kQ0n6ST96wa5qkVyQ15fpqnL4Fb8lzrBeBqkjbyOSTfsE+quF8t8jIVeJmkb2F183U9SbPVQpJmnSdWM5b/AGcDf0jP9RLf1Z6uJsn7/PR6f81wvmVpLAfk7p82Px1E8jl+QtJEdzNJv5YVgDxhkJmZ5eMahJmZ5eUCwszM8nIBYWZmebmAMDOzvBrN4Fht2rSJLl26FDsMM7M1yoQJE76IiLb5tjWaAqJLly6MHz++2GGYma1RJH1Y2TY3MZmZWV4uIMzMLC8XEGZmlpcLCDMzy8sFhJmZ5VWwAkLSQCXTMk6rZLsk3aFkSsYpknbO2XamkqkYZ0g6s1AxmplZ5QpZg3iIZLTOyhwKbJ3+9CMZbhpJPwCuJZkQpTdwraRNChinmZnlUbD3ICLi5QrTK1Z0FDA4kuFk35C0saT2QB9gVETMA5A0iqSgGVKIOBd/u4J7x9ZmMiwzawi277ARB/Xw5HJ1qZgvynVg1bkBStK0ytK/R1I/ktoHnTt3Xq0glnxbyp/GzFytY82sYYiAzVq1cAFRx9boN6kj4n7gfoBevXqt1sQWrTdcl//e2LdO4zKz+nX+4PGMeuszho7/mIigLKAs/T33q2Vssn5zyoJ023fbI6CsLHd91WP323ZTenf9QbGzVzTFLCBmk0x2Xq5jmjabpJkpN31svUVlZmuc12Z+AcAvh02p1XkkaCLRRLC8NJj+yUIePne3ughxjVTMAmI4cLGkx0g6pBdGxBxJI4Hf5XRMHwRcVawgzazhm3ztQZTMX0Lzpkpv8MlNXhIAzZsKpWnl23MLg/L18v0BuvR/hldmfMH//WMqZWVByfwltG25LivKgrKyoLQskuVIfpfMW0yblusm2yJW/l5RmuxTWha0bNGcwef2plWL5sX6qGqkYAWEpCEkNYE2kkpInkxqDhAR95JMTH4Yyfy5i0nmtCUi5kn6DTAuPdWA8g5rM7N8mjdtQtc2GxTk3M9P/5SmTZKC49///ZKOm6xPE0GzJk1o0kQ0ayKaNBEbr9+cOQuX0GmT9WnaRMmPkm1NJT7/aikTP1rA7PlLaNV+LS8gIuLkarYHcFEl2wYCAwsRl5lZFrNuqtu+yeemzeHCRybW6TkLbY3upDYzW1NE+hjNmHc/Z+5Xy1heWsby0mB5aRlzFi5h/XWasaK0jBVlwfLSoGubDThk++I+leUCwsysHsz6cjEAv3/u3Uz7r9usCe/+9tBChlQtFxBmZvXggh9vSfOmYrv2rSgtCzZarznNmzZhnWZJp/mG6zajWdMmNGsq7hw9k4Gv/rfYIbuAMDOrD02aiPP22TLTvs2bqvqd6oFHczUzs7xcgzAza2CWLk86q5+bNoely8soLQsO7NGu3t+fcAFhZtbAjHrrM4BVHou9dml3zt6ra73G4QLCzKyBefrSvRk1/TN+uFlLAA7/06ssLy2r9zhcQJiZNTCtWjTnuF06AvDNshVFi8Od1GZmlpcLCDMzyytTE1M6surmwBJgVkTUf2OYmZnVq0oLCEkbkQymdzKwDjAXaAG0k/QGcHdEjKmXKM3MrN5VVYMYBgwG9omIBbkbJO0CnC5py4h4sIDxmZlZkVRaQETEgVVsmwBMKEhEZmbWIFTbSa3EaZKuSdc7S+pd+NDMzKyYsjzFdDewB0lfBMBXwF0Fi8jMzBqELAXEbhFxEbAUICLmk3Ram5lZPRnzztx6v2aWAmK5pKZAAEhqC2R6zFXSIZLelTRTUv8827eQ9KKkKZLGSuqYs+33kqZLelvSHcqdTdzMbC2xTrPkNv36B1/W+7WzFBB3AP8ANpV0A/Aq8LvqDkoLlbuAQ4HuwMmSulfY7VZgcETsCAwAbkyP3RPYC9gR2B7YFdg3S4bMzBqT5k2/u03/9d8f1eu1qy0gIuJR4JckN+85wNER8XiGc/cGZkbEBxHxLfAYcFSFfboDo9PlMTnbg+Sdi3WAdYHmwGcZrmlm1uhcfsA2ANw5egbfrqi/95SzPMV0B/CDiLgrIu6MiLcznrsD8HHOekmalmsycGy6fAzQUlLriHidpMCYk/6MzHddSf0kjZc0fu7c+m+fMzOrD5cdsDUAnyxcynPTP62362ZpYpoA/FrS+5JuldSrDq//C2BfSW+SNCHNBkolbQVsB3QkKVT2k7RPxYMj4v6I6BURvdq2bVuHYZmZNSx/PiO59S79trTerpmliWlQRBxG0g/wLnCzpBkZzj0b6JSz3jFNyz33JxFxbET8CPhVmraApDbxRkR8HRFfA8+SPGprZrZW2m7zVvV+zZqM5roVsC2wBfBOhv3HAVtL6ippHeAkYHjuDpLaSCqP4SpgYLr8EUnNopmk5iS1i6xNW2ZmVgey9EH8Pq0xDACmAb0i4ojqjouIFcDFwEiSm/vQiJguaYCkI9Pd+gDvSnoPaAfckKYPA94HppL0U0yOiKdqlDMzM6uVLMN9vw/sERFf1PTkETECGFEh7Zqc5WEkhUHF40qBC2p6PTMzqztVDfe9bUS8Q9JU1FlS59ztETEx/5FmZtYYVFWDuALoB9yWZ1sA+xUkIjMzq1RpRL1dq6rhvvuli4dGxNLcbZJaFDQqMzNbRWlpUjBc9cRUTu7duZq960aWPoh/ATtnSDMzswLp3Hr9lcuH/fEVAPb9YVv+95BtC3bNSp9ikrRZOnPcepJ+JGnn9KcPsH5lx5mZWWFtvvF6fPnNMp6bVti3qquqQRwMnEXygtvtOelfAf9XwJjMzCyPWTf1Xbl86ZA3mTp7YUGvV1UfxCBgkKTjIuLvBY3CzMwanKoecz0tIh4Buki6ouL2iLg9z2FmZtZIVNXEtEH6e8P6CMTMzBqWqpqY7kt/X19/4ZiZWUORdSymVpKap9ODzpV0Wn0EZ2ZmxZNlNNeDImIRcDgwi2RU1ysLGZSZmRVflgKivBmqL/B4RBT2uSozM2sQsrxJ/bSkd4AlwM8ktQWWVnOMmZmt4bLMKNcf2JNkHojlwDfAUYUOzMzMiqvaGkQ6o9tpwI8lAbwE3FvguMzMrMiyNDHdAzQH7k7XT0/TzitUUGZmVnxZCohdI2KnnPXRkiYXKiAzM2sYsjzFVCqpW/mKpC2B0iwnl3SIpHclzZTUP8/2LdJ3K6ZIGiupY862zpKel/S2pLckdclyTTMzqxtZCogrgTHpDfwlYDTwP9UdJKkpcBdwKNAdOFlS9wq73QoMjogdgQHAjTnbBgO3RMR2QG/g8wyxmpmtFYZP/oT/fvENj7zxYcGukeUppheBrYFLgUuAH0bEmAzn7g3MjIgPIuJb4DG+//RTd5ICB2BM+fa0IGkWEaPSGL6OiMUZrmlmtlY4e68uALz50YKCXSPLUBstgIuA64BrSd6FyDLlaAfg45z1kjQt12Tg2HT5GKClpNbANsACSU9IelPSLWmNpGJs/SSNlzR+7ty5GUIyM2scrj2iBx02Xq+g18jSxDQY6AH8CbgzXX64jq7/C2BfSW8C+wKzSfo3mgH7pNt3BbYkmbxoFRFxf0T0iohebdu2raOQzMwMsj3FtH1E5PYdjJH0VobjZgOdctY7pmkrRcQnpDUISRsCx0XEAkklwKSI+CDd9iSwO/BghuuamVkdyFKDmChp9/IVSbsB4zMcNw7YWlJXSesAJwHDc3eQ1EZSeQxXAQNzjt04HdYDYD8gS6FkZmZ1JEsBsQvwL0mzJM0CXgd2lTRV0pTKDoqIFcDFwEjgbWBoREyXNEDSkelufYB3Jb0HtANuSI8tJWleelHSVEDAA6uTQTMzWz1ZmpgOWd2TR8QIYESFtGtylocBwyo5dhSw4+pe28yssVuyvJSvli4v2PmrLSAionAP2ZqZ2Wqb9823PP/WZ5SVBU2aqM7Pn6WJyczMGqB2rdYFIAp0fhcQZmZrqFN326Kg53cBYWZmeWUqICTdX9W6mZk1PllrEPdVs25mZo1MpgIiIiZUtW5mZo1PpY+5SnqKKjrHI+LIyraZmdmar6r3IG6ttyjMzKzBqbSAiIiXypclrQd0joh36yUqMzMruizzQRwBTAKeS9d7Shpe5UFmZrbGy9JJfR3J7HALACJiEtC1YBGZmVmDkKWAWB4RCyukFerNbjMzayCyjOY6XdIpQFNJ5XNT/6uwYZmZWbFlqUFcQjLN6DJgCLAI+H8FjMnMzBqALMN9LwZ+JenmZDW+KnxYZmZWbFmeYto1ndVtCjBV0mRJuxQ+NDMzK6YsfRAPAj+PiFcAJO0N/AXP9mZm1qhl6YMoLS8cACLiVWBFlpNLOkTSu5JmSuqfZ/sWkl6UNEXSWEkdK2xvJalE0p1ZrmdmZnWnqrGYdk4XX5J0H0kHdQAnAmOrO7GkpsBdwIFACTBO0vCIeCtnt1uBwRExSNJ+wI3A6TnbfwO8nD07ZmZWV6pqYrqtwvq1OctZ3oPoDcyMiA8AJD0GHAXkFhDdgSvS5THAk+Ub0n6OdiRvcPfKcD0zM6tDVY3F9JNanrsD8HHOegmwW4V9JgPHAn8EjgFaSmoNzCcpoE4DDqjsApL6Af0AOnfuXMtwzcwsV5ZOaiT1JXkXokV5WkQMqIPr/wK4U9JZJE1Js4FS4OfAiIgokVTpwRFxP3A/QK9evfx2t5lZHaq2gJB0L7A+8BPgz8DxwH8ynHs20ClnvWOatlJEfEJSg0DShsBxEbFA0h7APpJ+DmwIrCPp64j4Xke3mZkVRpanmPaMiDOA+RFxPbAHsE2G48YBW0vqKmkd4CRglVFgJbWRVB7DVcBAgIg4NSI6R0QXklrGYBcOZmb1K0sBsST9vVjS5sByoH11B0XECuBiYCTwNjA0IqZLGiCpfDa6PsC7kt4j6ZC+oYbxm5lZgWTpg3ha0sbALcBEkieY/pzl5BExAhhRIe2anOVhwLBqzvEQ8FCW65mZWd3JMhbTb9LFv0t6GmiRZ/hvMzNrZKp6Ue7YKrYREU8UJiQzM2sIqqpBHFHFtgBcQJiZNWJVvSh3dn0GYmZmDUuWp5jMzGwt5ALCzMzycgFhZmZ5ZZlRbn1JV0t6IF3fWtLhhQ/NzMyKKUsN4i/AMpIhNiAZT+m3BYvIzMwahCwFRLeI+D3JEBtExGKg8iFWzcysUchSQHwraT3SSYIkdSOpUZiZWSOWZSym60hmdesk6VFgL+CsAsZkZmYNQJaxmJ6XNAHYnaRp6bKI+KLgkZmZWVFlmTDoKeCvwPCI+KbwIZmZWRZzFiazMcz75lvatly3zs+fpQ/iVmAf4C1JwyQdL6lFdQeZmVlhTZ2dDKz94tufFeT81RYQEfFSRPwc2BK4DzgB+Lwg0ZiZWWYPnNELSJ8gKoAsndSkTzEdAZwI7AwMKlA8ZmaWkQr8xkGWPoihQG+SJ5nuBF6KiLKCRmVmZkWXpQ/iQZKX5S6MiDE1KRwkHSLpXUkzJfXPs30LSS9KmiJprKSOaXpPSa9Lmp5uOzF7lszMrC5UNaPcfhExGtgAOEpatSpT3YxykpoCdwEHAiXAOEnDI+KtnN1uBQZHxCBJ+wE3AqcDi4EzImKGpM2BCZJGRsSCGufQzMxWS1VNTPsCo8k/s1yWGeV6AzMj4gMASY8BRwG5BUR34Ip0eQzwJEBEvLfyQhGfSPocaAssqOaaZmZWR6qaUe7adHFARPw3d5ukrhnO3QH4OGe9BNitwj6TgWOBPwLHAC0ltY6IL3Ou1RtYB3i/4gUk9QP6AXTu3DlDSGZmllWWPoi/50kbVkfX/wWwr6Q3SWoss4HS8o2S2gMPA2fn6/uIiPsjoldE9Grbtm0dhWRmZlB1H8S2QA9gI0nH5mxqBWR5UW420ClnvWOatlJEfEJSg0DShsBx5f0MkloBzwC/iog3MlzPzMzqUFV9ED8EDgc2ZtV+iK+A8zOcexywddocNRs4CTgldwdJbYB5ae3gKmBgmr4O8A+SDuy6qq2YmVkNVNUH8U/gn5L2iIjXa3riiFgh6WJgJNAUGBgR0yUNAMZHxHCgD3CjpABeBi5KDz8B+DHQWtJZadpZETGppnGYmdnqqaqJ6ZfpREGnSDq54vaIuLS6k0fECGBEhbRrcpaHkac/IyIeAR6p7vxmZlY4VTUxvZ3+Hl8fgZiZWcNSVRPTU+nvleMuSWoCbBgRi+ohNjMzK6JqH3OV9FdJrSRtAEwjGfb7ysKHZmZmxZTlPYjuaY3haOBZoCvJcBhmZtaIZSkgmktqTlJADI+I5RRu+HEzM2sgshQQ9wGzSAbte1nSFoD7IMzMGrksM8rdEREdIuKwSHwI/KQeYjMzsypE2pjz8ntzC3L+LJ3UG0m6XdL49Oc2ktqEmZkVUfOmyS382WmfFuT8WZqYBpIMr3FC+rMI+EtBojEzs8zabLguP96mLT07bVyQ82eZk7pbRByXs369pEkFicbMzBqMLDWIJZL2Ll+RtBewpHAhmZlZQ5ClBnEhMFjSRun6fODMwoVkZmYNQZUFhKSewFYkQ3XPBvAwG2Zma4dKm5gkXQMMBY4jmbjnRBcOZmZrj6pqECcCPSNisaTWwHPAA/UTlpmZFVtVndTLImIxQER8Wc2+ZmbWyFRVg9hS0vB0WUC3nHUi4siCRmZmZkVVVQFxVIX1WwsZiJmZNSxVTRj0Um1PLukQ4I8kc1L/OSJuqrB9C5I3tdsC84DTIqIk3XYm8Ot019/mTlxkZmaFV9VTTE9JOiId6rviti0lDZB0ThXHNwXuAg4FugMnS+peYbdbgcERsSMwALgxPfYHwLXAbkBv4FpJm9Qsa2ZmVhtVdTyfD+wDvCNpnKQRkkZL+oBkCPAJETGwiuN7AzMj4oOI+BZ4jO83W3UHRqfLY3K2HwyMioh5ETEfGAUcUqOcmZlZrVTVxPQp8Evgl5K6AO1Jhth4r/zppmp0AD7OWS8hqRHkmgwcS9IMdQzQMn2kNt+xHSpeQFI/oB9A586dM4RkZmZZZXp0NSJmRcTrETEpY+GQ1S+AfSW9CexL8rZ2adaDI+L+iOgVEb3atm1bh2GZmVmWsZhW12ygU856xzRtpYj4hKQGgaQNgeMiYoGk2UCfCseOLWCsZmZWQSFffhsHbC2pq6R1SMZzGp67g6Q2kspjuIrkiSaAkcBBkjZJO6cPStPMzKyeFKyAiIgVwMUkN/a3gaERMT19+qn8Jbs+wLuS3gPaATekx84DfkNSyIwDBqRpZmZWT6ptYkrnf7gO2CLdX0BExJbVHRsRI4ARFdKuyVkeBgyr5NiBfFejMDOzepalD+JB4HJgAjXoQDYzszVblgJiYUQ8W/BIzMysQclSQIyRdAvwBLCsPDEiJhYsKjMzK7osBUT5y229ctIC2K/uwzEzs4ai2gIiIn5SH4GYmVnDUu1jrpI2knS7pPHpz22SNqqP4MzMrHiyvAcxEPgKOCH9WQT8pZBBmZlZ8WXpg+gWEcflrF8vaVKB4jEzswYiSw1iiaS9y1fSF+eWFC4kMzNrCLLUIH4GDEr7HUQy89tZhQzKzMyKL8tTTJOAnSS1StcXFTooMzMrvkoLCEmnRcQjkq6okA5ARNxe4NjMzKyIqqpBbJD+blkfgZiZWcNS1ZSj96W/r6+/cMzMrKHI8qLc7yW1ktRc0ouS5ko6rT6CMzOz4snymOtBacf04cAsYCvgykIGZWZmxZelgChvhuoLPB4RCwsYj5mZNRBZCoinJb0D7AK8KKktsDTLySUdIuldSTMl9c+zvbOkMZLelDRF0mFpenNJgyRNlfS2pKtqkikzM6u9aguIiOgP7An0iojlwDfAUdUdJ6kpcBdwKNAdOFlS9wq7/ZpkruofAScBd6fpPwXWjYgdSAqmCyR1yZQjMzOrE1W9B7FfRIyWdGxOWu4uT1Rz7t7AzIj4ID32MZKC5a2cfQJolS5vBHySk76BpGbAesC3JIMEmplZPanqPYh9gdHAEXm2BdUXEB2Aj3PWS/hu8qFy1wHPS7qE5L2LA9L0YSSFyRxgfeDyiJhXzfXMzKwOVfUexLXp77MLeP2TgYci4jZJewAPS9qepPZRCmwObAK8IumF8tpIOUn9gH4AnTt3LmCYZmZrnyzvQfxO0sY565tI+m2Gc88GOuWsd0zTcp0LDAWIiNeBFkAb4BTguYhYHhGfA6+x6pSnpMfcHxG9IqJX27ZtM4RkZmZZZXmK6dCIWFC+EhHzgcMyHDcO2FpSV0nrkHRCD6+wz0fA/gCStiMpIOam6ful6RsAuwPvZLimmZnVkSwFRFNJ65avSFoPWLeK/QGIiBXAxcBI4G2Sp5WmSxog6ch0t/8Bzpc0GRgCnBURQfL004aSppMUNH+JiCk1yZiZmdVOlvkgHiV5/6F8mtGzgUFZTh4RI4ARFdKuyVl+C9grz3FfkzzqamZmRZJlPoib02/45U8Y/SYiRhY2LDMzK7YsNQhImohWRMQLktaX1DIivipkYGZmVlxZnmI6n+S9hPvSpA7AkwWMyczMGoAsndQXkfQTLAKIiBnApoUMyszMii9LAbEsIr4tX0mHv4jChWRmZg1BlgLiJUn/B6wn6UDgceCpwoZlZmbFlqWA+F+Sl9emAheQPLb660IGZWZmxVflU0zpkN3TI2Jb4IH6CcnMzBqCKmsQEVEKvCvJI+GZma1lsrwHsQkwXdJ/SCYLAiAijqz8EDMzW9NlKSCuLngUZmbW4FQ1o1wL4EJgK5IO6gfTAfjMzGwtUFUfxCCSORimkswrfVu9RGRmZg1CVU1M3SNiBwBJDwL/qZ+QzMysIaiqBrG8fMFNS2Zma5+qahA7SVqULovkTepF6XJERKuCR2dmZkVTaQEREU3rMxAzM2tYsgy1YWZma6GsEwatFkmHAH8EmgJ/joibKmzvTPK01MbpPv3TaUqRtCPJHBStgDJg14hYWpPrL1++nJKSEpYurdFhtpZr0aIFHTt2pHnz5sUOxayoClZApOM43QUcCJQA4yQNT+ehLvdrYGhE3COpO8lAgF3SIcUfAU6PiMmSWpPTaZ5VSUkJLVu2pEuXLkiqdZ6s8YsIvvzyS0pKSujatWuxwzErqkI2MfUGZkbEB+l8Eo8BR1XYJ0hqCAAbAZ+kywcBUyJiMkBEfJmOC1UjS5cupXXr1i4cLDNJtG7d2rVOMwpbQHQAPs5ZL0nTcl0HnCaphKT2cEmavg0QkkZKmijpl/kuIKmfpPGSxs+dOzdvEC4crKb8b8YsUexO6pOBhyKiI3AY8LCkJiRNX3sDp6a/j5G0f8WDI+L+iOgVEb3atm1bn3GbmTV6hSwgZgOdctY7pmm5zgWGAkTE60ALoA1JbePliPgiIhaT1C52LmCsBfPpp59y0kkn0a1bN3bZZRcOO+ww3nvvPWbNmsX2229fZ9e55ppreOGFFwB45ZVX6NGjBz179mT27Nkcf/zxtTp3RLDffvuxaNGilWlPPvkkknjnnXdWps2aNYv11luPnj170r17dy688ELKyspqde2XX36ZnXfemWbNmjFs2LBK95swYQI77LADW221FZdeeikRyay48+bN48ADD2TrrbfmwAMPZP78+QA8/fTTXHPNNbWKzayxK2QBMQ7YWlJXSesAJwHDK+zzEbA/gKTtSAqIucBIYAdJ66cd1vsCb7GGiQiOOeYY+vTpw/vvv8+ECRO48cYb+eyzz+r8WgMGDOCAAw4A4NFHH+Wqq65i0qRJdOjQocoba0UrVnz/pfkRI0aw00470arVd+9GDhkyhL333pshQ4assm+3bt2YNGkSU6ZM4a233uLJJ59cvQylOnfuzEMPPcQpp5xS5X4/+9nPeOCBB5gxYwYzZszgueeeA+Cmm25i//33Z8aMGey///7cdFPyIF3fvn156qmnWLx4ca3iM2vMCvYUU0SskHQxyc2+KTAwIqZLGgCMj4jhwP8AD0i6nKTD+qxIvvrNl3Q7SSETwIiIeKY28Vz/1HTe+mRR9TvWQPfNW3HtET0q3T5mzBiaN2/OhRdeuDJtp512ApJv2+VmzZrF6aefzjffJNNt3Hnnney5557MmTOHE088kUWLFrFixQruuece9txzT84991zGjx+PJM455xwuv/xyzjrrLA4//HAWLFjA0KFDGTlyJM8++yw33HADhx9+ONOmTaO0tJT+/fszduxYli1bxkUXXcQFF1zA2LFjufrqq9lkk0145513eO+991bJx6OPPkq/fv1Wrn/99de8+uqrjBkzhiOOOILrr7/+e3lv1qwZe+65JzNnzlytz7Zcly5dAGjSpPLvMnPmzGHRokXsvvvuAJxxxhk8+eSTHHroofzzn/9k7NixAJx55pn06dOHm2++GUn06dOHp59+mhNOOKFWMZo1VgV9DyJ9p2FEhbRrcpbfAvaq5NhHSB51XWNNmzaNXXbZpdr9Nt10U0aNGkWLFi2YMWMGJ598MuPHj+evf/0rBx98ML/61a8oLS1l8eLFTJo0idmzZzNt2jQAFixYsMq5zjvvPF599VUOP/xwjj/++FUKogcffJCNNtqIcePGsWzZMvbaay8OOuggACZOnMi0adPyPtr52muvcd99961c/+c//8khhxzCNttsQ+vWrZkwYcL38rl48WJefPFFBgwY8L3z7bPPPnz11VffS7/11ltX1oJqYvbs2XTs2HHleseOHZk9O2nN/Oyzz2jfvj0Am2222Sq1t169evHKK6+4gDCrREELiIakqm/6xbZ8+XIuvvhiJk2aRNOmTVd+g991110555xzWL58OUcffTQ9e/Zkyy235IMPPuCSSy6hb9++K2/wWTz//PNMmTJlZZPTwoULmTFjBuussw69e/eu9Ln/efPm0bJly5XrQ4YM4bLLLgPgpJNOYsiQISsLiPfff5+ePXsiiaOOOopDDz30e+d75ZVXMsdclySt8oTSpptuyieffFLFEWZrt7WmgCiGHj16ZGr//8Mf/kC7du2YPHkyZWVltGjRAoAf//jHvPzyyzzzzDOcddZZXHHFFZxxxhlMnjyZkSNHcu+99zJ06FAGDhyYKZ6I4E9/+hMHH3zwKuljx45lgw02qPS4Zs2aUVZWRpMmTZg3bx6jR49m6tSpSKK0tBRJ3HLLLcB3fRBVqesaRIcOHSgpKVm5XlJSQocOyRPV7dq1Y86cObRv3545c+aw6aabrtxv6dKlrLfeejW+ntnaotiPuTZq++23H8uWLeP+++9fmTZlypTvfYNeuHAh7du3p0mTJjz88MOUlibvBH744Ye0a9eO888/n/POO4+JEyfyxRdfUFZWxnHHHcdvf/tbJk6cmDmegw8+mHvuuYfly5OX0t97772V/R5V+eEPf8gHH3wAwLBhwzj99NP58MMPmTVrFh9//DFdu3atUa3glVdeYdKkSd/7WZ3CAaB9+/a0atWKN954g4hg8ODBHHVU8k7mkUceyaBBgwAYNGjQynRI8l+XT5KZNTYuIApIEv/4xz944YUX6NatGz169OCqq65is802W2W/n//85wwaNIiddtqJd955Z+W3+bFjx7LTTjvxox/9iL/97W9cdtllzJ49mz59+tCzZ09OO+00brzxxszxnHfeeXTv3p2dd96Z7bffngsuuCDvU0sV9e3bd2VH75AhQzjmmGNW2X7cccd972mmujJu3Dg6duzI448/zgUXXECPHt81Ffbs2XPl8t133815553HVlttRbdu3VY2bfXv359Ro0ax9dZb88ILL9C/f/+Vx4wZM4a+ffsWJG6zxkDlz4uv6Xr16hXjx49fJe3tt99mu+22K1JEjcecOXM444wzGDVqVLFDqTOfffYZp5xyCi+++GLe7f63Y2uKO0fPYPG3pfzykG1X63hJEyKiV75t7oOwarVv357zzz+fRYsWrfIuxJrso48+4rbbPM26rfku3m/rgp3bBYRl0tgeBd11112LHYJZg9fo+yAaSxOa1R//mzFLNOoCokWLFnz55Zf+D2+Zlc8HUf6osdnarFE3MXXs2JGSkhIqGwrcLJ/yGeXM1naNuoBo3ry5ZwUzM1tNjbqJyczMVp8LCDMzy8sFhJmZ5dVo3qSWNBf4sBanaAN8UUfhrCnWtjyvbfkF53ltUZs8bxEReedsbjQFRG1JGl/Z6+aN1dqW57Utv+A8ry0KlWc3MZmZWV4uIMzMLC8XEN+5v/pdGp21Lc9rW37BeV5bFCTP7oMwM7O8XIMwM7O8XECYmVlea1UBIekQSe9Kmimpf57t60r6W7r935K6FCHMOpUhz1dIekvSFEkvStqiGHHWperynLPfcZJC0hr/SGSWPEs6If1bT5f01/qOsa5l+LfdWdIYSW+m/74PK0acdUXSQEmfS5pWyXZJuiP9PKZI2rnWF42IteIHaAq8D2wJrANMBrpX2OfnwL3p8knA34oddz3k+SfA+unyz9aGPKf7tQReBt4AehU77nr4O28NvAlskq5vWuy46yHP9wM/S5e7A7OKHXct8/xjYGdgWiXbDwOeBQTsDvy7ttdcm2oQvYGZEfFBRHwLPAYcVWGfo4BB6fIwYH9JqscY61q1eY6IMRGxOF19A1jTx7nO8ncG+A1wM7C0PoMrkCx5Ph+4KyLmA0TE5/UcY13LkucAyufI3Qj4pB7jq3MR8TIwr4pdjgIGR+INYGNJ7WtzzbWpgOgAfJyzXpKm5d0nIlYAC4HW9RJdYWTJc65zSb6BrMmqzXNa9e4UEc/UZ2AFlOXvvA2wjaTXJL0h6ZB6i64wsuT5OuA0SSXACOCS+gmtaGr6/71ajXo+CMtO0mlAL2DfYsdSSJKaALcDZxU5lPrWjKSZqQ9JLfFlSTtExIJiBlVgJwMPRcRtkvYAHpa0fUSUFTuwNcXaVIOYDXTKWe+YpuXdR1Izkmrpl/USXWFkyTOSDgB+BRwZEcvqKbZCqS7PLYHtgbGSZpG01Q5fwzuqs/ydS4DhEbE8Iv4LvEdSYKypsuT5XGAoQES8DrQgGdSuscr0/70m1qYCYhywtaSuktYh6YQeXmGf4cCZ6fLxwOhIe3/WUNXmWdKPgPtICoc1vV0aqslzRCyMiDYR0SUiupD0uxwZEeOLE26dyPJv+0mS2gOS2pA0OX1QjzHWtSx5/gjYH0DSdiQFRGOef3g4cEb6NNPuwMKImFObE641TUwRsULSxcBIkicgBkbEdEkDgPERMRx4kKQaOpOkM+ik4kVcexnzfAuwIfB42h//UUQcWbSgayljnhuVjHkeCRwk6S2gFLgyItbY2nHGPP8P8ICky0k6rM9ak7/wSRpCUsi3SftVrgWaA0TEvST9LIcBM4HFwNm1vuYa/HmZmVkBrU1NTGZmVgMuIMzMLC8XEGZmlpcLCDMzy8sFhJmZ5eUCwlabpFJJkyRNk/SUpI3r+Pyz0mf2kfR1JfusJ+klSU0ldZG0JI3pLUn3pm9O1+SavSTdkS73kbRnzrYLJZ1Rmzyl57lO0i+q2echScfX4JxdKhvls65JOrJ89FRJR0vqnrNtQPri5eqc9zFJa/LLe42OCwirjSUR0TMitid5b+SiIsRwDvBERJSm6+9HRE9gR5IRPI+uyckiYnxEXJqu9gH2zNl2b0QMrm3Aa7qIGB4RN6WrR5N8zuXbromIF1bz1PcAv6xleFaHXEBYXXmddGAwSd0kPSdpgqRXJG2bpreT9A9Jk9OfPdP0J9N9p0vqV8Prngr8s2JiOtjiv4Ct0m/Xo/XdnBed0+v+NK39TJb0cprWR9LTSuYCuRC4PK2R7FP+zV/StpL+U36t9PxT0+Vd0hrNBEkjVc1ompLOlzQujeHvktbP2XyApPGS3pN0eLp/U0m3pMdMkXRBTT4sSV9L+kP6Wb8oqW2a3lPJIH5T0r/RJmn6pfpuvpDH0rSzJN2Z/v2OBG5JP6Nu5TUfJXM1PJ5z3T6Snk6XD5L0uqSJkh6XtGG62ytpnteaF3gbOhcQVmuSmpIMaVD+lvL9wCURsQvwC+DuNP0O4KWI2IlkXPvpafo56b69gEslZRpBV8kQC1tGxKw829ZPY5oK/AkYFBE7Ao+mcQBcAxycxrPK2+PpOe8F/pDWkl7J2fYOsI6krmnSicDfJDVPr3V8mp+BwA3VZOOJiNg1jeFtkvGDynUhGda6L3CvpBbp9oURsSuwK3B+Thzled9c0ohKrrcByZvGPYCXSN7GBRgM/G/6GU3NSe8P/ChNv7DCZ/Qvkr/5leln9H7O5heA3SRtkK6fCDympMnw18ABEbEzMB64Ij1fGclbwDtV/nFZfXIBYbWxnqRJwKdAO2BU+m1wT5KhOyaRjPNU/i16P5JmBCKiNCIWpumXSppMMi5SJ7IPItcGWFAhrVt63deAZyLiWWAPoHwGtYeBvdPl14CHJJ1PMlxDTQwluemR/v4b8EOSgQBHpTH8murn19g+rWVNJakN9ci9RkSURcQMknGTtgUOIhlvZxLwb5Lh6Ff5vCLik4iobPa0sjRWgEeAvSVtBGwcES+l6YNIJqcBmAI8qmS03xXV5CU3hhXAc8ARaY2gL0lNb3eSJqnX0jycCeTOYvg5sHnW61hhuSpntbEkInqm39ZHkvRBPAQsSPsBqiWpD3AAsEdELJY0lmRQtUzXz7Pv+1mvHREXStqN5OY1QdIuGa8LyU32cUlPJKeKGZJ2AKZHxB41OM9DwNERMVnSWaQD6pWHWDFkktnCLomIkbkbtPrT41Y31k5fksLiCOBXaR6zegy4mKR/anxEfCVJwKiIOLmSY1qQ/F2tAXANwmotnZHuUpLB0RYD/5X0U1g5T255k8GLJNOalrelb0QypPr8tHDYluQbZtbrzgeapk0vVfkX3w28eCpJWzeSukXEvyPiGpJRPjtVOO4rkuHB8137fZJB767mu2/k7wJtlcw9gKTmknrkOz5HS2BO2jx1aoVtP5XURFI3kqk13yUpiH+W7o+kbXKacbJoQjJSMcApwKtpTW6+pH3S9NOBl5Q8AdYpIsYA/0vyt9qwwvkq/YxImrB2JpnN7rE07Q1gL0lbpfFvIGmbnGO2AerlaSyrngsIqxMR8SZJc8TJJDe6c9Nmo+l8NxXkZcBP0uaUCSRNDc8BzSS9DdxEcgOpief5rsmoMpcAZ0uaQnLzuyxNv0XSVCWPh/6LZF7jXE8Bx6QdsPvwfX8DTuO7OQe+Jbn53pzmfRI5T0FV4mqSpqLXgHcqbPsI+A/JLH8XRsRS4M/AW8DENO77qNASUE0fxDdA7/TY/YABafqZJJ/HFKBnmt4UeCT9e70J3JFngqHHgCslvZkWZCulT5Y9DRya/iYi5pJM1jQkvdbrJE1nSGpHUiv9tJLYrZ55NFdboymZPvTyiDi92LGsCSR9HREVawENgpJhuRdFxIPFjsUSrkHYGi0iJgJj0iepbM22gKSD3BoI1yDMzCwv1yDMzCwvFxBmZpaXCwgzM8vLBYSZmeXlAsLMzPL6/+I+7HODzXRXAAAAAElFTkSuQmCC",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"evaluate_embeddings_approach(labels=['An Amazon review with a negative sentiment.', 'An Amazon review with a positive sentiment.'])\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"As shown above, zero-shot classification with embeddings can lead to great results, especially when the labels are more descriptive than just simple words."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "openai",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.9"
},
"orig_nbformat": 4,
"vscode": {
"interpreter": {
"hash": "365536dcbde60510dc9073d6b991cd35db2d9bac356a11f5b64279a5e6708b97"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}