openai-cookbook/examples/Zero-shot_classification_with_embeddings.ipynb
2022-06-03 12:56:03 -07:00

225 lines
56 KiB
Plaintext

{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Zero-shot classification with embeddings\n",
"\n",
"In this notebook we will classify the sentiment of reviews using embeddings and zero labeled data! The dataset is created in the [Obtain_dataset Notebook](Obtain_dataset.ipynb).\n",
"\n",
"We'll define positive sentiment to be 4 and 5-star reviews, and negative sentiment to be 1 and 2-star reviews. 3-star reviews are considered neutral and we won't use them for this example.\n",
"\n",
"We will perform zero-shot classification by embedding descriptions of each class and then comparing new samples to those class embeddings."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"\n",
"from sklearn.metrics import classification_report\n",
"\n",
"df = pd.read_csv('output/embedded_1k_reviews.csv')\n",
"df['babbage_similarity'] = df.babbage_similarity.apply(eval).apply(np.array)\n",
"df['babbage_search'] = df.babbage_search.apply(eval).apply(np.array)\n",
"\n",
"df= df[df.Score!=3]\n",
"df['sentiment'] = df.Score.replace({1:'negative', 2:'negative', 4:'positive', 5:'positive'})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Zero-Shot Classification\n",
"To perform zero shot classification, we want to predict labels for our samples without any training. To do this, we can simply embed short descriptions of each label, such as positive and negative, and then compare the cosine distance between embeddings of samples and label descriptions. \n",
"\n",
"The highest similarity label to the sample input is the predicted label. We can also define a prediction score to be the difference between the cosine distance to the positive and to the negative label. This score can be used for plotting a precision-recall curve, which can be used to select a different tradeoff between precision and recall, by selecting a different threshold."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" precision recall f1-score support\n",
"\n",
" negative 0.67 0.88 0.76 136\n",
" positive 0.98 0.93 0.95 789\n",
"\n",
" accuracy 0.92 925\n",
" macro avg 0.82 0.90 0.86 925\n",
"weighted avg 0.93 0.92 0.92 925\n",
"\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Z1A+gAAAACXBIWXMAAAsTAAALEwEAmpwYAAAyIElEQVR4nO3dd5xV1b3//9ebJhZsgIoggqgxYAQVNaJGYlewm9iV2Fv0F6/eq9dIIolRY8kvxthyNWLDQoxdCSIoGk0oAoIoLaiMqAjSRNrM5/vH3oOH4czMBubMmfJ+Ph7zmLP7Z52B8zlrrb3XUkRgZmZWUZNiB2BmZnWTE4SZmeXlBGFmZnk5QZiZWV5OEGZmlpcThJmZ5eUEYTVKUj9JbxU7jpok6XRJ/8iw372Srq+NmGqDpJmSDklf/1rSo8WOyWqXE4QhaQNJD0j6WNIiSeMkHVnsuLJIP8S+lbRY0heSHpK0SU1eIyIei4jDMux3UUT8piavXU5SSPomLWeJpDskNS3EtczKOUEYQDPgU+BAYDPgl8BTkjoVM6i1cHREbALsAfQkiX81kprVelQ1r3tazgOBk4FzihxPjWogf6MGxQnCiIhvIuLXETEzIsoi4kXgP8CelR0jaTtJz0iaI2mupLsq2e+Pkj6VtFDSGEkH5GzbW9LodNsXku5I17eU9Gh63vmSRknaOkM5SoBXgF3T84SkSyVNBaam6/qmNaT5kv4pabfqypTbbKbEHyR9mcb9vqTy6z0k6bc55ztf0jRJ8yQ9L2nbnG0h6SJJU9NY/ixJ1ZUxLec04G2gR8751qVcXSS9nq77StJjkjbPEkNFko5Nr79Q0nRJR6TrVzVTpcurmqokdUrfh3MlfQK8LukVSZdVOPd4SSekr3eRNDR9Tz+S9NN1ideycYKwNaQfxjsDkyrZ3hR4EfgY6AS0B56o5HSjSD7ItgQeB56W1DLd9kfgjxGxKdAFeCpdfzZJTWY7oDVwEfBthri3A44C3stZfRywD9BV0u7Ag8CF6XnvA55Pm9iylukw4Eck789mwE+BuXliOQi4Kd3eLj1vxfP1BfYCdkv3O7y6Mqbn3gU4AJiWLq9ruZTGuC3wfZL3+9dZYqgQz97Aw8DVwOYk78/MtTjFgen1DwcGAafmnLsrsD3wkqSNgaEk/462Ak4B7k73sQJwgrDVSGoOPAYMjIgPK9ltb5IPlavT2sfSiMjbMR0Rj0bE3IhYGRG3AxsA30s3rwB2lNQmIhZHxLs561sDO0ZEaUSMiYiFVYT9rKT5wFvAG8DvcrbdFBHzIuJb4ALgvoj4V3regcAy4IdrUaYVQCtgF0ARMTkiZufZ73TgwYgYGxHLgGuBfSs0290cEfMj4hNgODk1gkqMlfQNMBkYAdydrl+nckXEtIgYGhHLImIOcAfJh/XaOjct69C0BlpSxb+dfH6dxvYt8Hegh6Tt022nA8+k72FfYGZE/DX99/Qe8DfgJ+sQs2XgBGGrSGoCPAIsBy7LWf+Kks7RxZJOJ/mm+XFErMxwzqskTZa0IP0Q3wxok24+l+Sb+IdpM1LfdP0jwBDgCUmfSfp9mrgqc1xEbB4R20fEJekHTblPc15vD/xX2gwzP41nO5IP0ExliojXgbuAPwNfSrpf0qZ5dt2W5Ft7+XGLSWoa7XP2+Tzn9RJgEwBJk3Le7wNy9tkj3edkklrRxutTLklbS3pCSaf3QuBRvvvbrI3tgOnrcFy5VX+jiFgEvERSO4CkNvFY+np7YJ8K5Twd2GY9rm1VcIIwIGlbBx4AtgZOjIgV5dsi4siI2CT9eYzkP3RHVdOpmH64/TdJ88kWEbE5sICkaYOImBoRp5I0F9wCDJa0cUSsiIgbIqIr0Ivkm+NZ61i03OGKPwVuTJNJ+c9GETEoa5nSuO+MiD2BriQJ7uo8u31G8oEGQNo80hooyXD+bjnv98gK2yIingLeAfqvZ7l+R/L+/CBt5juD9G+zlj4laSLM5xtgo5zlfB/mFYeUHgScKmlfoCVJ7ar8Om9UKOcmEXHxOsRsGThBWLl7SNqBj67wDTyffwOzgZslbaykU3m/PPu1AlYCc4BmkvoDq75tSzpDUtuIKAPmp6vLJP1Y0g/S9vOFJM06ZetTuNRfgIsk7aPExpL6SGqVtUyS9kqPb07y4be0ktgGAT+T1EPSBiQfxv+KiJk1UA6Am4HzJW2zHuVqBSwGFkhqT/5El8UDJGU9WFITSe3TfhKAccApkppL6gmclOF8L5Mk1wHAk+m/D0j6UnaWdGZ6vubp3+P76xi3VcMJwkjbey8kaQP/vEJz0hoiohQ4GtgR+ASYRdLsUdEQ4FVgCklzy1JWb/I5ApgkaTFJh/UpaXLaBhhMkhwmk/QrPLKexSQiRgPnkzQRfU3SydtvLcu0KckH8tdpmeYCt+a51mvA9SRt5LNJvmGfUnG/9SjL+8CbJH0L61quG0iarRaQNOs8s46x/Bv4GfCH9Fxv8F3t6XqSsn+dXu/xDOdblsZySO7+afPTYSTv42ckTXS3kPRrWQHIEwaZmVk+rkGYmVleThBmZpaXE4SZmeXlBGFmZnk1mMGx2rRpE506dSp2GGZm9cqYMWO+ioi2+bY1mATRqVMnRo8eXewwzMzqFUkfV7bNTUxmZpaXE4SZmeXlBGFmZnk5QZiZWV5OEGZmllfBEoSkB5VMyzixku2SdKeSKRknSNojZ9vZSqZinCrp7ELFaGZmlStkDeIhktE6K3MksFP6cwHJcNNI2hL4FcmEKHsDv5K0RQHjNDOzPAr2HEREvFlhesWKjgUejmQ42XclbS6pHdAbGBoR8wAkDSVJNIMKEeeS5Su5d8T6TIZlZnVVyxZN6derExu1aDCPfNWqYr5r7Vl9boBZ6brK1q9B0gUktQ86duy4TkF8u7yUPw2ftk7HmlndVT6TwS7btOKgXbYubjD1VL1OqxFxP3A/QM+ePddpYovWm2zAf27qU6NxmVnxTSxZQN8/vUVpTcxF2EgVM0GUkEx2Xq5Duq6EpJkpd/2IWovKzBqE8hrEKxNns2xlKaVlQVkEpWWwT+ct2W7Ljao+gRU1QTwPXCbpCZIO6QURMVvSEOB3OR3ThwHXFitIM6ufgiRDPDO2hGfGlqy27Yhu23DvmXsWI6x6pWAJQtIgkppAG0mzSO5Mag4QEfeSTEx+FMn8uUtI5rQlIuZJ+g0wKj3VgPIOazOzrHbrsDl/v6QXLZs3pWkT0USiaRNx4SOjWe52p0wKeRfTqdVsD+DSSrY9CDxYiLjMrPHYveOad8jPX7KC6XPm8NoHXzD3m2W0bN6UFaXBitIyVpaW8dmCpWzasvmq5RVlwafzlrDlxi1W229FWbBR86bccGy3BnuXVMMslZlZJb5ctAyA8x7ONj1Ai6ZNkGDZyjLabLIBLZqKZk2bsHxlGZ8vXMope3dkz+0b5qNaThBm1qiMuKo3H32xiHabtaRZkyY0aQIbNW9G82aiedMmNG/ShBbNmtC8adIkJSnved6YMoezH/x3LUdfu5wgzKxR6dRmYzq12bjYYdQLHqzPzMzycoIwM7O8nCDMzCwvJwgzM8vLCcLMzPJygjAzs7ycIMzMLC8nCDMzyyvTg3LpyKrbAt8CMyPCI12ZmTVwlSYISZuRDKZ3KtACmAO0BLaW9C5wd0QMr5Uozcys1lVVgxgMPAwcEBHzczdI2hM4U9IOEfFAAeMzM7MiqTRBRMShVWwbA4wpSERmZlYnVNtJrcQZkvqnyx0l7V340MzMrJiydFLfDZQBBwEDgEXA34C9ChiXmVmdtmJlcq/OdX9/n45bbkTTJmLSZwtZ8O0KIoLlpWXccEw3Tt6rY5EjXXdZEsQ+EbGHpPcAIuJrSS0KHJeZWZ22siyZ8/rDzxfx4eeLaNG0Cd/bphVbbNScTTdszjvT5zL1i8VFjnL9ZEkQKyQ1hWQGcEltSWoUZmaN1hG7bsP7vz6MDZo1pUWzNVvru/V/tQhR1awsD8rdCfwd2ErSjcBbwO8KGpWZWT3QqmXzvMmhoai2BhERj0kaAxwMCDguIiYXPDIzMyuqahOEpDuBJyLiz7UQj5mZ1RFZ6kZjgF9Kmi7pNkk9Cx2UmZkVX7UJIiIGRsRRJLe1fgTcImlqwSMzM7OiWpvelR2BXYDtgQ8LE46ZWcPwzfJS/u+t/9DrpmH811Pjix3OOsnyJPXv0xrDAGAi0DMiji54ZGZmDcBnC5Yyaua8YoexTrI8BzEd2Dcivip0MGZmDcXMm/sA8IsnxzHm46+LHM26qWq4710i4kNgFNBR0mrPi0fE2EIHZ2ZmxVNVDeJK4ALg9jzbgmRsJjMza6CqGu77gvTlkRGxNHebpJYFjcrMzIouy11M/8y4zszMGpCq+iC2AdoDG0ranWSYDYBNgY1qITYzMyuiqmoQhwO3AR2AO0j6Im4n6Zv43ywnl3SEpI8kTZN0TZ7t20saJmmCpBGSOuRs+72kSZImS7pTkioeb2ZW170w/jM+mbeE0//vXcrSIcLri0oTRPoE9Y+BfhHx45yfYyLimepOnA4R/mfgSKArcKqkrhV2uw14OCJ2I3nO4qb02F7AfsBuwK4kT3EfuPbFMzMrrp/0TL73vj1t7qo5JOqLqpqYzoiIR4FOkq6suD0i7qjm3HsD0yJiRnq+J4BjgQ9y9ulKUiMBGA48W356oCXQgqRpqznwRXWFMTOra246YTc6bLERtw75qNihrLWqmpg2Tn9vArTK81Od9sCnOcuz0nW5xgMnpK+PB1pJah0R75AkjNnpz5B8Q4xLukDSaEmj58yZkyEkMzPLqqrbXO9Lf99QwOtfBdwlqR/wJlAClEraEfg+Sf8HwFBJB0TEyAox3g/cD9CzZ8/6VXczM6vjso7FtKmk5mmH8hxJZ2Q4dwmwXc5yh3TdKhHxWUScEBG7A9el6+aT1CbejYjFEbEYeAXYN1uRzMysJmR5DuKwiFgI9AVmkozqenWG40YBO0nqLKkFcArwfO4OktpIKo/hWuDB9PUnwIGSmklqTtJB7VnszMxqUZYEUd4M1Qd4OiIWZDlxRKwELgOGkHy4PxURkyQNkHRMultv4CNJU4CtgRvT9YNJBgl8n6SfYnxEvJDlumZmVjOyjOb6oqQPgW+BiyW1BZZWcwwAEfEy8HKFdf1zXg8mSQYVjysFLsxyDTMzK4wsM8pdA/QimQdiBfANye2qZmbWgFVbg0j7AM4AfpQ+zPwGcG+B4zIzsyLL0sR0D8mDaneny2em684rVFBmZlZ8WRLEXhHRPWf5dUn1c4JVMzPLLEuCKJXUJSKmA0jaASgtbFhmZo3PitIy5i5ezorSMlaWBRtv0JStWhVv+p0sCeJqYLikGSTjIm0P/KygUZmZNRLLV5YxYdZ83p42lz+8NmW1bU2biH//78G03mSDosRWbYKIiGGSdgK+l676KCKWFTYsM7OGZ8oXi5j25WKGTv6CIRM/B1hjhNddtmnFuft35r1P5/P4vz5h8bKVdTdBpNOLXgLsTzLK6khJ91achtTMzPIbNjkZjLrvn95abf1RP9iGLm03QRL7dWlNj46bs0GzpkBSe3j8X5/Ueqy5sjQxPQwsAv6ULp8GPAL8pFBBmZk1JP3268zYT97j6sO/R4ctNmSPjluw3ZZ1f2LOLAli14jInehnuKQPKt3bzMxWc0z3bTmm+7bFDmOtZRmLaaykH5YvSNoHGF24kMzMrC7IUoPYE/inpPLGsI4kA+y9D0Q6XaiZmTUwWRLEEQWPwszM6pwst7l+XBuBmJlZ3ZKlD8LMzBohJwgzM8vLCcLMzPLKlCAk3V/VspmZNTxZaxD3VbNsZmYNTKYEERFjqlo2M7OGp9LbXCW9QDI4X14RcUxBIjIzszqhqucgbqu1KMzMrM6pNEFExBvlryVtCHSMiI9qJSozMyu6avsgJB0NjANeTZd7SHq+wHGZmVmRZemk/jWwNzAfICLGAZ0LFpGZmdUJWRLEiohYUGFdpZ3XZmbWMGQZzXWSpNOApunc1JcD/yxsWGZmVmxZahA/B7oBy4BBwELg/ytgTGZmVgdkGe57CXCdpFuSxVhU+LDMzKzYstzFtFc6e9wE4H1J4yXtWfjQzMysmLL0QTwAXBIRIwEk7Q/8FfBUo2ZmDViWPojS8uQAEBFvASsLF5KZmdUFlSYISXtI2gN4Q9J9knpLOlDS3cCILCeXdISkjyRNk3RNnu3bSxomaYKkEZI65GzrKOkfkiZL+kBSp7UvnplZ/VTy9bcA3DNietFiqKqJ6fYKy7/KeV3tcxCSmgJ/Bg4FZgGjJD0fER/k7HYb8HBEDJR0EHATcGa67WHgxogYKmkToKy6a5qZNRSbbdQcgCdGfcrNJxanRb+qsZh+vJ7n3huYFhEzACQ9ARwL5CaIrsCV6evhwLPpvl2BZhExNI1l8XrGYmZWr5y1byc++nwRQyZ9XrQYsnRSI6kPybMQLcvXRcSAag5rD3yaszwL2KfCPuOBE4A/AscDrSS1BnYG5kt6hmRYj9eAayKitEJcFwAXAHTs2DFLUczMLKMst7neC5xM8sCcgJ8A29fQ9a8CDpT0HnAgUAKUkiSuA9LtewE7AP0qHhwR90dEz4jo2bZt2xoKyczMINtdTL0i4izg64i4AdiX5Bt+dUqA7XKWO6TrVomIzyLihIjYHbguXTefpLYxLiJmRMRKkqanPTJc08zMakiWBPFt+nuJpG2BFUC7DMeNAnaS1FlSC+AUYLVhwiW1kVQew7XAgznHbi6pvFpwEKv3XZiZWYFlSRAvStocuBUYC8wkGZOpSuk3/8uAIcBk4KmImCRpgKTy6Up7Ax9JmgJsDdyYHltK0rw0LH2KW8BfshfLzMzWV5axmH6TvvybpBeBlnmG/67s2JeBlyus65/zejAwuJJjh+Kntc3MiqbSBCHphCq2ERHPFCYkMzOrC6qqQRxdxbYAnCDMzBqwqh6U+1ltBmJmZnVLlk5qMzMrgulzFvPV4uUsWV6c8VGdIMzM6qh3Z8wD4F//mVeU6ztBmJnVUc9duh8A4z6ZX5TrZxlqYyNJ10v6S7q8k6S+hQ/NzKxxm7dkOQB/HDa1KM1MWWoQfwWWkQyxAclwGb8tWERmZgZA752/G2NuxcpqZ1mocVkSRJeI+D3JEBtExBKSJ5vNzKyAJNG/b9eiXT9LglguaUPSSYIkdSGpUZiZWQOWZT6IXwOvAttJegzYjzxDb5uZWcOSZSymf0gaA/yQpGnpioj4quCRmZlZUVWbICS9ADwOPB8R3xQ+JDMzqwuy9EHcRjK72weSBks6SVLL6g4yM7P6LUsT0xvAG5Kakkzccz7JxD6bFjg2MzNLjZjyJUuWl3Jsj23ZqEWW7uP1l+kq6V1MR5PMTb0HMLCQQZmZWWJiSTL9zhVPjAOgZfMmHL97h1q5dpYnqZ8imRHuIOAukucifl7owMzMDHp03Hy15fvemMGcRbXzpEGWPogHSJLCRRExPCLKCh2UmZklztq3EzNv7sPwq3oD8OHni7jvjem1cu2qZpQ7KCJeBzYGjpVWf3jaM8qZmdWezm025oCd2jBy6ld8U0vjMlVVgzgw/X10nh8P1mdmVsseOXcfAAb9+9NauV5VM8r9Kn05ICL+k7tNUueCRmVmZkWXpQ/ib3nWDa7pQMzMrHpn77s9m2/UvFauVVUfxC5AN2AzSSfkbNoU8INyZmYNXFXPQXyPpK9hc5J+h3KLSB6WMzOzWvbOjLnMX7KCFyd8Rt/dti3otarqg3gOeE7SvhHxTkGjMDOzTJatTJ40ePa9kuIlCEn/nU4UdJqkUytuj4jLCxqZmZmt4Y2rf0yna17itclfFvxaVTUxTU5/jy54FGZmttbmL1nO5hu1KNj5q2pieiH9vWrcJUlNgE0iYmHBIjIzsypdd9T3ufHlyZSWFXae6ixjMT0uaVNJGwMTSYb9vrqgUZmZWaU2aJ7lCYX1l+UqXdMaw3HAK0Bn4MxCBmVmZtUr77AulCwJormk5iQJ4vmIWAEUtl5jZmaVGjk1mfX5mLveKuh1siSI+4CZJIP2vSlpe8B9EGZmRXL+ATsAsHhZYQftqzZBRMSdEdE+Io6KxMfAj7OcXNIRkj6SNE3SNXm2by9pmKQJkkZI6lBh+6aSZkm6K3OJzMwauL07bwnA0hVllMz/tmDXydJJvZmkOySNTn9uJ6lNVHdcU+DPwJFAV+BUSV0r7HYb8HBE7AYMAG6qsP03wJsZymFm1qjs2j6Z9bnk6yImCJL5pxcBP01/FgJ/zXDc3sC0iJgREcuBJ4BjK+zTFXg9fT08d7ukPYGtgX9kuJaZWaNy7ZHfL/g1siSILhHxq/SDfkZE3ADskOG49kDuoOWz0nW5xgPlAwEeD7SS1Dp93uJ24KqqLiDpgvKazZw5czKEZGZmWWVJEN9K2r98QdJ+QE3Vaa4CDpT0HskERSVAKXAJ8HJEzKrq4Ii4PyJ6RkTPtm3b1lBIZmYGVQ+1Ue4i4GFJm6XLXwNnZziuBNguZ7lDum6ViPiMtAYhaRPgxIiYL2lf4ABJlwCbAC0kLY6INTq6zcysMKpMEJJ6ADsCp5B+uK/FMBujgJ3S2edK0nOcVuH8bYB5EVEGXEvS30FEnJ6zTz+gp5ODmVntqrSJSVJ/4CngROAl4OS1GYMpIlYClwFDSAb+eyoiJkkaIOmYdLfewEeSppB0SN+4TqUwM7MaV1UN4mSgR0QskdQaeBX4y9qcPCJeBl6usK5/zuvBVDN9aUQ8BDy0Ntc1M7P1V1Un9bKIWAIQEXOr2dfMzBqYqmoQO0h6Pn0toEvOMhFxTP7DzMysIagqQVR8qO22QgZiZmZ1S1UTBr1Rm4GYmVndUtVdTC9IOjod6rvith3Su5HOKWx4ZmZWLFU1MZ0PXAn8/5LmAXOAlkAnYDpwV0Q8V/AIzcysKKpqYvoc+G/gvyV1AtqRDLExpfzuJjMza7iyDLVBRMwkmTTIzMwaCT/bYGZmeTlBmJlZXk4QZmaWV5YpR/eTNFTSFEkzJP1H0ozaCM7MzPJbvrIMgGlfLi7YNbLUIB4A7gD2B/YCeqa/zcysSJYsLwVg4D9nFuwaWRLEgoh4JSK+jIi55T8Fi8jMzKrVZ7d2AOzafrNq9lx3WW5zHS7pVuAZYFn5yogYW7CozMysWu0337Cg58+SIPZJf/fMWRfAQTUfjpmZZbVk+UoWfLuiYOevNkFExI8LdnUzM1tnXy9ZwWuTv6CsLGjSRDV+/ix3MW0m6Q5Jo9Of2yUVrtHLzMzWShTovFk6qR8EFgE/TX8WAn8tUDxmZpbRlYfuXNDzZ+mD6BIRJ+Ys3yBpXIHiMTOzOiJLDeJbSfuXL0jaj2RUVzMza8Cy1CAuBgam/Q4C5gH9ChmUmZkVX5a7mMYB3SVtmi4vLHRQZmZWfJUmCElnRMSjkq6ssB6AiLijwLGZmVkRVVWD2Dj93ao2AjEzs7qlqilH70t/31B74ZiZWV2R5UG530vaVFJzScMkzZF0Rm0EZ2ZmxZPlNtfD0o7pviTzUu8IXF3IoMzMrPiyJIjyZqg+wNMRsaCA8ZiZWR2R5TmIFyV9SPJw3MWS2gJLCxuWmZkVW7U1iIi4BugF9IyIFcA3wLGFDszMzIqrqucgDoqI1yWdkLMud5dnChmYmZkVV1U1iAPT30fn+emb5eSSjpD0kaRpkq7Js3379M6oCZJGSOqQru8h6R1Jk9JtJ69VqczMbL1V9RzEr9LfP1uXE0tqCvwZOBSYBYyS9HxEfJCz223AwxExUNJBwE3AmcAS4KyImCppW2CMpCERMX9dYjEzs7WX5TmI30naPGd5C0m/zXDuvYFpETEjIpYDT7Bm30VX4PX09fDy7RExJSKmpq8/A74E2ma4ppmZ1ZAst7kemfvNPSK+Bo7KcFx74NOc5VnpulzjgfI+juOBVpJa5+4gaW+gBTC94gUkXVA+092cOXMyhGRmZlllSRBNJW1QviBpQ2CDKvZfG1cBB0p6j6TPowQozblWO+AR4GcRUVbx4Ii4PyJ6RkTPtm1dwTAzq0lZnoN4DBgmqXya0Z8BAzMcVwJsl7PcIV23Stp8dAKApE2AE8trK+nw4i8B10XEuxmuZ2ZmNSjLfBC3SBoPHJKu+k1EDMlw7lHATpI6kySGU4DTcneQ1AaYl9YOriWZ/xpJLYC/k3RgD85aGDMzqzlZahAAk4GVEfGapI0ktYqIRVUdEBErJV0GDAGaAg9GxCRJA4DREfE80Bu4SVIAbwKXpof/FPgR0FpSv3Rdv3TyIjMzqwXVJghJ5wMXAFsCXUg6mu8FDq7u2Ih4GXi5wrr+Oa8HA2vUECLiUeDR6s5vZmaFk6WT+lJgP2AhQHr76VaFDMrMzIovS4JYlj7HAICkZkAULiQzM6sLsiSINyT9L7ChpEOBp4EXChuWmZkVW5YE8T/AHOB94EKSPoVfFjIoMzMrvio7qdPxlCZFxC7AX2onJDMzqwuqrEFERCnwkaSOtRSPmZnVEVmeg9gCmCTp3ySTBQEQEccULCozMyu6LAni+oJHYWZmdU5VM8q1BC4CdiTpoH4gIlbWVmBmZlZcVfVBDAR6kiSHI4HbayUiMzOrE6pqYuoaET8AkPQA8O/aCcnMzOqCqmoQK8pfuGnJzKzxqaoG0V3SwvS1SJ6kXpi+jojYtODRmZlZ0VSaICKiaW0GYmZmdUuWoTbMzKwRyjphUL20YsUKZs2axdKlS4sditUjLVu2pEOHDjRv3rzYoZgVVYNOELNmzaJVq1Z06tQJScUOx+qBiGDu3LnMmjWLzp07FzscsyqVfP0tAHO/WcZWrVrW+PkbdBPT0qVLad26tZODZSaJ1q1bu9Zp9cI7M+YCMGzylwU5f4NOEICTg601/5ux+uLRc/cp6PkbfIIwM2uoWjQr7Ee4E0SBff7555xyyil06dKFPffck6OOOoopU6Ywc+ZMdt111xq7Tv/+/XnttdcAGDlyJN26daNHjx6UlJRw0kknrde5I4KDDjqIhQsXrlr37LPPIokPP/xw1bqZM2ey4YYb0qNHD7p27cpFF11EWVnZel37zTffZI899qBZs2YMHjy40v3GjBnDD37wA3bccUcuv/xyIpJZcefNm8ehhx7KTjvtxKGHHsrXX38NwIsvvkj//v3XKzazhs4JooAiguOPP57evXszffp0xowZw0033cQXX3xR49caMGAAhxxyCACPPfYY1157LePGjaN9+/ZVfrBWtHLlmg/Nv/zyy3Tv3p1NN/3u2chBgwax//77M2jQoNX27dKlC+PGjWPChAl88MEHPPvss+tWoFTHjh156KGHOO2006rc7+KLL+Yvf/kLU6dOZerUqbz66qsA3HzzzRx88MFMnTqVgw8+mJtvvhmAPn368MILL7BkyZL1is+sIWvQdzHluuGFSXzw2cLqd1wLXbfdlF8d3a3S7cOHD6d58+ZcdNFFq9Z1794dSL5tl5s5cyZnnnkm33yTTLdx11130atXL2bPns3JJ5/MwoULWblyJffccw+9evXi3HPPZfTo0UjinHPO4Re/+AX9+vWjb9++zJ8/n6eeeoohQ4bwyiuvcOONN9K3b18mTpxIaWkp11xzDSNGjGDZsmVceumlXHjhhYwYMYLrr7+eLbbYgg8//JApU6asVo7HHnuMCy64YNXy4sWLeeuttxg+fDhHH300N9xwwxplb9asGb169WLatGnr9N6W69SpEwBNmlT+XWb27NksXLiQH/7whwCcddZZPPvssxx55JE899xzjBgxAoCzzz6b3r17c8sttyCJ3r178+KLL/LTn/50vWI0a6gaTYIohokTJ7LnnntWu99WW23F0KFDadmyJVOnTuXUU09l9OjRPP744xx++OFcd911lJaWsmTJEsaNG0dJSQkTJ04EYP78+aud67zzzuOtt96ib9++nHTSSaslogceeIDNNtuMUaNGsWzZMvbbbz8OO+wwAMaOHcvEiRPz3tr59ttvc999961afu655zjiiCPYeeedad26NWPGjFmjnEuWLGHYsGEMGDBgjfMdcMABLFq0aI31t91226pa0NooKSmhQ4cOq5Y7dOhASUkJAF988QXt2rUDYJtttlmt9tazZ09GjhzpBGFWiUaTIKr6pl9sK1as4LLLLmPcuHE0bdp01Tf4vfbai3POOYcVK1Zw3HHH0aNHD3bYYQdmzJjBz3/+c/r06bPqAz6Lf/zjH0yYMGFVk9OCBQuYOnUqLVq0YO+99670vv958+bRqlWrVcuDBg3iiiuuAOCUU05h0KBBqxLE9OnT6dGjB5I49thjOfLII9c438iRIzPHXJMkrXaH0lZbbcVnn31WlFjM6oNGkyCKoVu3bpna///whz+w9dZbM378eMrKymjZMnng5Uc/+hFvvvkmL730Ev369ePKK6/krLPOYvz48QwZMoR7772Xp556igcffDBTPBHBn/70Jw4//PDV1o8YMYKNN9640uOaNWtGWVkZTZo0Yd68ebz++uu8//77SKK0tBRJ3HrrrcB3fRBVqekaRPv27Zk1a9aq5VmzZtG+fXsAtt56a2bPnk27du2YPXs2W2211ar9li5dyoYbbrjW1zNrLNxJXUAHHXQQy5Yt4/7771+1bsKECWt8g16wYAHt2rWjSZMmPPLII5SWlgLw8ccfs/XWW3P++edz3nnnMXbsWL766ivKyso48cQT+e1vf8vYsWMzx3P44Ydzzz33sGJFMpL7lClTVvV7VOV73/seM2bMAGDw4MGceeaZfPzxx8ycOZNPP/2Uzp07r1WtYOTIkYwbN26Nn3VJDgDt2rVj00035d133yUiePjhhzn22GMBOOaYYxg4cCAAAwcOXLUekvLX5J1kZsXyzbLCzMjgBFFAkvj73//Oa6+9RpcuXejWrRvXXnst22yzzWr7XXLJJQwcOJDu3bvz4Ycfrvo2P2LECLp3787uu+/Ok08+yRVXXEFJSQm9e/emR48enHHGGdx0002Z4znvvPPo2rUre+yxB7vuuisXXnhh3ruWKurTp8+qjt5BgwZx/PHHr7b9xBNPXONuppoyatQoOnTowNNPP82FF15It27fNRX26NFj1eu7776b8847jx133JEuXbqsatq65pprGDp0KDvttBOvvfYa11xzzapjhg8fTp8+fQoSt1ltWLI8+f/725cmF+T8Kr9fvL7r2bNnjB49erV1kydP5vvf/36RImo4Zs+ezVlnncXQoUOLHUqN+eKLLzjttNMYNmxY3u3+t2P1QVlZsMP/vsx5+3fml327rtM5JI2JiJ75trkPwqrVrl07zj//fBYuXLjasxD12SeffMLtt3uadavfmjQRM28uXC3YCcIyaWi3gu61117FDsGszmvwfRANpQnNao//zZglCpogJB0h6SNJ0yRdk2f79pKGSZogaYSkDjnbzpY0Nf05e12u37JlS+bOnev/8JZZ+XwQ5bcamzVmBWtiktQU+DNwKDALGCXp+Yj4IGe324CHI2KgpIOAm4AzJW0J/AroCQQwJj3267WJoUOHDsyaNYs5c+bURJGskSifUc6ssStkH8TewLSImAEg6QngWCA3QXQFrkxfDweeTV8fDgyNiHnpsUOBI4C1upeyefPmnhXMzGwdFbKJqT3wac7yrHRdrvHACenr44FWklpnPBZJF0gaLWm0awlmZjWr2J3UVwEHSnoPOBAoAUqzHhwR90dEz4jo2bZt20LFaGbWKBWyiakE2C5nuUO6bpWI+Iy0BiFpE+DEiJgvqQToXeHYEQWM1czMKijYk9SSmgFTgINJEsMo4LSImJSzTxtgXkSUSboRKI2I/mkn9Rhgj3TXscCe5X0SlVxvDvDxeoTcBvhqPY6vjxpbmRtbecFlbizWp8zbR0TeJpiC1SAiYqWky4AhQFPgwYiYJGkAMDoiniepJdwkKYA3gUvTY+dJ+g1JUgEYUFVySI9ZrzYmSaMre9y8oWpsZW5s5QWXubEoVJkL+iR1RLwMvFxhXf+c14OBvONhR8SDQLZxrM3MrMYVu5PazMzqKCeI79xf/S4NTmMrc2MrL7jMjUVBytxghvs2M7Oa5RqEmZnl5QRhZmZ5NaoEkWF02Q0kPZlu/5ekTkUIs0ZlKPOVkj5IR9QdJmn7YsRZk6orc85+J0oKSfX+lsgsZZb00/RvPUnS47UdY03L8G+7o6Thkt5L/30fVYw4a4qkByV9KWliJdsl6c70/ZggaY98+62ViGgUPyTPYkwHdgBakIwD1bXCPpcA96avTwGeLHbctVDmHwMbpa8vbgxlTvdrRfLszbtAz2LHXQt/552A94At0uWtih13LZT5fuDi9HVXYGax417PMv+I5OHhiZVsPwp4BRDwQ+Bf63vNxlSDWDW6bEQsB8pHl811LDAwfT0YOFiSajHGmlZtmSNieEQsSRffJRnWpD7L8ncG+A1wC7C0NoMrkCxlPh/4c6RD5kfEl7UcY03LUuYAyufI3Qz4rBbjq3ER8SZQ1QPDx5JMnxAR8S6wuaR263PNxpQgsowQu2qfiFgJLABa10p0hZFpVNwc55J8A6nPqi1zWvXeLiJeqs3ACijL33lnYGdJb0t6V9IRtRZdYWQp86+BMyTNInlg9+e1E1rRrO3/92p5TmoDQNIZJBM0HVjsWApJUhPgDqBfkUOpbc1Impl6k9QS35T0g4iYX8ygCuxU4KGIuF3SvsAjknaNiLJiB1ZfNKYaRLWjy+bukw42uBkwt1aiK4wsZUbSIcB1wDERsayWYiuU6srcCtgVGCFpJklb7fP1vKM6y995FvB8RKyIiP+QDKS5Uy3FVwhZynwu8BRARLwDtCQZ1K6hyvT/fW00pgQxCthJUmdJLUg6oZ+vsM/zQPn81ycBr0fa+1NPVVtmSbsD95Ekh/reLg3VlDkiFkREm4joFBGdSPpdjomI0cUJt0Zk+bf9LOkQ+ukoyjsDM2oxxpqWpcyfkIwmjaTvkySIhjyz2PPAWendTD8EFkTE7PU5YaNpYopso8s+QFINnUbSGXRK8SJefxnLfCuwCfB02h//SUQcU7Sg11PGMjcoGcs8BDhM0gckk3JdHRH1tnacscz/BfxF0i9IOqz71ecvfJIGkST5Nmm/yq+A5gARcS9JP8tRwDRgCfCz9b5mPX6/zMysgBpTE5OZma0FJwgzM8vLCcLMzPJygjAzs7ycIMzMLC8nCFtnkkoljZM0UdILkjav4fPPTO/ZR9LiSvbZUNIbkppK6iTp2zSmDyTdmz45vTbX7CnpzvR1b0m9crZdJOms9SlTep5fS7qqmn0eknTSWpyzU2WjfNY0SceUj54q6ThJXXO2DUgfvFyX8z4hqT4/vNfgOEHY+vg2InpExK4kz41cWoQYzgGeiYjSdHl6RPQAdiMZwfO4tTlZRIyOiMvTxd5Ar5xt90bEw+sbcH0XEc9HxM3p4nEk73P5tv4R8do6nvoe4L/XMzyrQU4QVlPeIR0YTFIXSa9KGiNppKRd0vVbS/q7pPHpT690/bPpvpMkXbCW1z0deK7iynSwxX8CO6bfrl/Xd3NedEyv+5O09jNe0pvput6SXlQyF8hFwC/SGskB5d/8Je0i6d/l10rP/376es+0RjNG0hBVM5qmpPMljUpj+JukjXI2HyJptKQpkvqm+zeVdGt6zARJF67NmyVpsaQ/pO/1MElt0/U9lAziNyH9G22Rrr9c380X8kS6rp+ku9K/3zHArel71KW85qNkroanc67bW9KL6evDJL0jaaykpyVtku42Mi1zo3mAt65zgrD1JqkpyZAG5U8p3w/8PCL2BK4C7k7X3wm8ERHdSca1n5SuPyfdtydwuaRMI+gqGWJhh4iYmWfbRmlM7wN/AgZGxG7AY2kcAP2Bw9N4Vnt6PD3nvcAf0lrSyJxtHwItJHVOV50MPCmpeXqtk9LyPAjcWE0xnomIvdIYJpOMH1SuE8mw1n2AeyW1TLcviIi9gL2A83PiKC/7tpJeruR6G5M8adwNeIPkaVyAh4H/Sd+j93PWXwPsnq6/qMJ79E+Sv/nV6Xs0PWfza8A+kjZOl08GnlDSZPhL4JCI2AMYDVyZnq+M5Cng7pW/XVabnCBsfWwoaRzwObA1MDT9NtiLZOiOcSTjPJV/iz6IpBmBiCiNiAXp+ssljScZF2k7sg8i1waYX2Fdl/S6bwMvRcQrwL5A+QxqjwD7p6/fBh6SdD7JcA1r4ymSDz3S308C3yMZCHBoGsMvqX5+jV3TWtb7JLWhbrnXiIiyiJhKMm7SLsBhJOPtjAP+RTIc/WrvV0R8FhGVzZ5WlsYK8Ciwv6TNgM0j4o10/UCSyWkAJgCPKRntd2U1ZcmNYSXwKnB0WiPoQ1LT+yFJk9TbaRnOBnJnMfwS2DbrdaywXJWz9fFtRPRIv60PIemDeAiYn/YDVEtSb+AQYN+IWCJpBMmgapmun2ff6VmvHREXSdqH5MNrjKQ9M14Xkg/ZpyU9k5wqpkr6ATApIvZdi/M8BBwXEeMl9SMdUK88xIohk8wW9vOIGJK7Qes+PW51Y+30IUkWRwPXpWXM6gngMpL+qdERsUiSgKERcWolx7Qk+btaHeAahK23dEa6y0kGR1sC/EfST2DVPLnlTQbDSKY1LW9L34xkSPWv0+SwC8k3zKzX/Rpomja9VOWffDfw4ukkbd1I6hIR/4qI/iSjfG5X4bhFJMOD57v2dJJB767nu2/kHwFtlcw9gKTmkrrlOz5HK2B22jx1eoVtP5HURFIXkqk1PyJJxBen+yNp55xmnCyakIxUDHAa8FZak/ta0gHp+jOBN5TcAbZdRAwH/ofkb7VJhfNV+h6RNGHtQTKb3RPpuneB/STtmMa/saSdc47ZGaiVu7Gsek4QViMi4j2S5ohTST7ozk2bjSbx3VSQVwA/TptTxpA0NbwKNJM0GbiZ5ANkbfyD75qMKvNz4GeSJpB8+F2Rrr9V0vtKbg/9J8m8xrleAI5PO2APYE1PAmfw3ZwDy0k+fG9Jyz6OnLugKnE9SVPR28CHFbZ9AvybZJa/iyJiKfB/wAfA2DTu+6jQElBNH8Q3wN7psQcBA9L1Z5O8HxOAHun6psCj6d/rPeDOPBMMPQFcLem9NJGtkt5Z9iJwZPqbiJhDMlnToPRa75A0nSFpa5Ja6eeVxG61zKO5Wr2mZPrQX0TEmcWOpT6QtDgiKtYC6gQlw3IvjIgHih2LJVyDsHotIsYCw9M7qax+m0/SQW51hGsQZmaWl2sQZmaWlxOEmZnl5QRhZmZ5OUGYmVleThBmZpbX/wPov24If76fLgAAAABJRU5ErkJggg==",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"from openai.embeddings_utils import cosine_similarity, get_embedding\n",
"from sklearn.metrics import PrecisionRecallDisplay\n",
"\n",
"def evaluate_emeddings_approach(\n",
" labels = ['negative', 'positive'], \n",
" engine = 'text-similarity-babbage-001',\n",
"):\n",
" label_embeddings = [get_embedding(label, engine=engine) for label in labels]\n",
"\n",
" def label_score(review_embedding, label_embeddings):\n",
" return cosine_similarity(review_embedding, label_embeddings[1]) - cosine_similarity(review_embedding, label_embeddings[0])\n",
"\n",
" engine_col_name = engine.replace('-','_').replace('_query','')\n",
" probas = df[engine_col_name].apply(lambda x: label_score(x, label_embeddings))\n",
" preds = probas.apply(lambda x: 'positive' if x>0 else 'negative')\n",
"\n",
" report = classification_report(df.sentiment, preds)\n",
" print(report)\n",
"\n",
" display = PrecisionRecallDisplay.from_predictions(df.sentiment, probas, pos_label='positive')\n",
" _ = display.ax_.set_title(\"2-class Precision-Recall curve\")\n",
"\n",
"evaluate_emeddings_approach(labels=['negative', 'positive'], engine='text-similarity-babbage-001')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can see that this classifier already performs extremely well. We used similarity embeddings, and the simplest possible label name. Let's try to improve on this by using more descriptive label names, and search embeddings."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" precision recall f1-score support\n",
"\n",
" negative 0.65 0.93 0.76 136\n",
" positive 0.99 0.91 0.95 789\n",
"\n",
" accuracy 0.92 925\n",
" macro avg 0.82 0.92 0.86 925\n",
"weighted avg 0.94 0.92 0.92 925\n",
"\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Z1A+gAAAACXBIWXMAAAsTAAALEwEAmpwYAAAwoUlEQVR4nO3deZwU1bn/8c+XAQRRwCAigiyiRkEFFY2iROJucNeoGBc04pKo+cWruXqNRsmiRk3uNSYuuXpFY3Ahxl0JKihuUUBAcAMMIoiKsokswszz+6NqxmbsmSmY6emZ4ft+veY1Vaeqq57TA/10nVN1jiICMzOzypoVOwAzM2uYnCDMzCwvJwgzM8vLCcLMzPJygjAzs7ycIMzMLC8nCKtTkoZKerHYcdQlST+U9M8M+90q6Yr6iKk+SJot6cB0+SpJfy12TFa/nCAMSRtJukPSB5K+kDRZ0mHFjiuL9ENshaRlkj6RdJekTeryHBFxb0QcnGG/cyPiV3V57nKSQtKXaT3nSfq9pJJCnMusnBOEATQHPgT2A9oBvwAekNSjmEGtgyMiYhNgN6A/SfxrkdS83qOqe33Teu4HnAicWeR46lQT+Rs1KU4QRkR8GRFXRcTsiCiLiMeBfwO7V/UaSVtLekjSAkmfS7q5iv3+R9KHkpZKmihpYM62PSVNSLd9Iun3aXkrSX9Nj7tY0uuSOmWoxzzgKWCn9Dgh6SeSZgAz0rLD0yukxZJelrRLTXXKbTZT4g+SPk3jflNS+fnukvTrnOMNkzRT0kJJj0raKmdbSDpX0ow0lj9JUk11TOs5E3gJ6JdzvPWpVy9Jz6Vln0m6V1L7LDFUJumo9PxLJc2SdGhaXtFMla5XNFVJ6pG+Dz+SNAd4TtJTks6vdOwpko5Nl3eQNCZ9T9+VdML6xGvZOEHYN6QfxtsD06vYXgI8DnwA9AC6APdVcbjXST7IvgX8DXhQUqt02/8A/xMRbYFewANp+ekkVzJbAx2Ac4EVGeLeGvg+8EZO8dHAd4DeknYF7gTOSY97G/Bo2sSWtU4HA98leX/aAScAn+eJZX/gmnR75/S4lY93OLAHsEu63yE11TE99g7AQGBmur6+9VIa41bAjiTv91VZYqgUz57A3cAlQHuS92f2Ohxiv/T8hwAjgSE5x+4NdAeekNQGGEPy72gL4CTgz+k+VgBOELYWSS2Ae4EREfFOFbvtSfKhckl69bEyIvJ2TEfEXyPi84hYExE3AhsB3043rwa2lbR5RCyLiFdzyjsA20ZEaURMjIil1YT9sKTFwIvA88Bvc7ZdExELI2IFcDZwW0T8Kz3uCGAVsNc61Gk1sCmwA6CIeDsi5ufZ74fAnRExKSJWAZcBe1dqtrs2IhZHxBxgLDlXBFWYJOlL4G1gHPDntHy96hURMyNiTESsiogFwO9JPqzX1Y/Suo5Jr0DnVfNvJ5+r0thWAP8A+knqnm77IfBQ+h4eDsyOiP9L/z29Afwd+MF6xGwZOEFYBUnNgHuAr4Dzc8qfUtI5ukzSD0m+aX4QEWsyHPNiSW9LWpJ+iLcDNk83/4jkm/g7aTPS4Wn5PcBo4D5JH0n6XZq4qnJ0RLSPiO4R8eP0g6bchznL3YH/SJthFqfxbE3yAZqpThHxHHAz8CfgU0m3S2qbZ9etSL61l79uGcmVRpecfT7OWV4ObAIgaXrO+z0wZ5/d0n1OJLkqalObeknqJOk+JZ3eS4G/8vXfZl1sDcxaj9eVq/gbRcQXwBMkVweQXE3cmy53B75TqZ4/BLasxbmtGk4QBiRt68AdQCfguIhYXb4tIg6LiE3Sn3tJ/kN3Uw2diumH289Jmk82i4j2wBKSpg0iYkZEDCFpLrgOGCWpTUSsjoirI6I3MIDkm+Np61m13OGKPwR+kyaT8p+NI2Jk1jqlcd8UEbsDvUkS3CV5dvuI5AMNgLR5pAMwL8Px++S83+MrbYuIeAB4BbiylvX6Lcn7s3PazHcK6d9mHX1I0kSYz5fAxjnr+T7MKw8pPRIYImlvoBXJ1VX5eZ6vVM9NIuK89YjZMnCCsHK3kLQDH1HpG3g+rwHzgWsltVHSqbxPnv02BdYAC4Dmkq4EKr5tSzpFUseIKAMWp8Vlkr4naee0/XwpSbNOWW0ql/oLcK6k7yjRRtJgSZtmrZOkPdLXtyD58FtZRWwjgTMk9ZO0EcmH8b8iYnYd1APgWmCYpC1rUa9NgWXAEkldyJ/osriDpK4HSGomqUvaTwIwGThJUgtJ/YHjMxzvSZLkOhy4P/33AUlfyvaSTk2P1yL9e+y4nnFbDZwgjLS99xySNvCPKzUnfUNElAJHANsCc4C5JM0elY0GngbeI2luWcnaTT6HAtMlLSPpsD4pTU5bAqNIksPbJP0K99SymkTEBGAYSRPRIpJO3qHrWKe2JB/Ii9I6fQ5cn+dczwBXkLSRzyf5hn1S5f1qUZc3gRdI+hbWt15XkzRbLSFp1nloPWN5DTgD+EN6rOf5+urpCpK6L0rP97cMx1uVxnJg7v5p89PBJO/jRyRNdNeR9GtZAcgTBpmZWT6+gjAzs7ycIMzMLC8nCDMzy8sJwszM8moyg2Ntvvnm0aNHj2KHYWbWqEycOPGziOiYb1uTSRA9evRgwoQJxQ7DzKxRkfRBVdvcxGRmZnk5QZiZWV5OEGZmlpcThJmZ5eUEYWZmeRUsQUi6U8m0jNOq2C5JNymZknGqpN1ytp2uZCrGGZJOL1SMZmZWtUJeQdxFMlpnVQ4Dtkt/ziYZbhpJ3wJ+STIhyp7ALyVtVsA4zcwsj4I9BxERL1SaXrGyo4C7IxlO9lVJ7SV1BgYBYyJiIYCkMSSJZmQh4lz+1RpuHVebybDMrKnYeKPmDB3Qg1YtSoodSoNQzAflurD23ABz07Kqyr9B0tkkVx9069ZtvYJY8VUpfxw7c71ea2ZNR/nMB327tmfvXh2KG0wD0aifpI6I24HbAfr3779eE1t02GQj/n3N4DqNy8wan9f+vZATbnuFMs+RU6GYCWIeyWTn5bqmZfNImplyy8fVW1RmtkEqTwyPT53PZ8tWURZBWRmURrBqdSnLVpXSZqMSSsuCsoCysqA0IllPy0ojaNOyhDP26UnL5o3/JtFiJohHgfMl3UfSIb0kIuZLGg38Nqdj+mDgsmIFaWYbhtWlydTXI1+bw8jX5tTqWP17fIvduzf+e2sKliAkjSS5Ethc0lySO5NaAETErSQTk3+fZP7c5SRz2hIRCyX9Cng9PdTw8g5rM7NCGbhdRx75yT60bllCM4mSZqJEQoKSZsnvFiXNKJFo1kw0S8tz931p1mecesdrTaaZqpB3MQ2pYXsAP6li253AnYWIy8ysKn23bl+r1wvVTSANRONvJDMzs4JwgjAzs7ycIMzMLC8nCDMzy8sJwszM8nKCMDOzvJwgzMwsLycIMzPLywnCzMzycoIwM7O8nCDMzCyvTGMxpSOrbgWsAGZHRFlBozIzs6KrMkFIakcymN4QoCWwAGgFdJL0KvDniBhbL1GamVm9q+4KYhRwNzAwIhbnbpC0O3CqpG0i4o4CxmdmZkVSZYKIiIOq2TYRmFiQiMzMGqnPv1wFwA9ufYWn/99AdtiybZEjqp0aO6mVOEXSlel6N0l7Fj40M7PGZZeu7SuW35izuGhx1JUsdzH9GdibpC8C4AvgTwWLyMyskeq5eRteveyAYodRZ7LcxfSdiNhN0hsAEbFIUssCx2VmZkWW5QpitaQSIAAkdQR8m6uZWROXJUHcBPwD2ELSb4AXgd8WNCozMyu6GpuYIuJeSROBAwABR0fE2wWPzMzMiqrGBCHpJuC+iHDHtJnZBiRLJ/VE4BeSvk3S1HRfREwobFhmZo3bG3MW0aKkGV+tKaOkGRy7W1dalDSu4e+yNDGNAEZI+hZwHHCdpG4RsV3BozMza2TKIgB4YMJcHpgwt6K8V8dN6N/jW8UKa71kGqwvtS2wA9AdcB+EmVkeW7VvzdH9tqL3Vm3ZdotN+HzZV1wyaiqrS6PYoa2zLH0QvwOOAWYB9wO/qjw2k5mZfe2/T9q1YvmVWZ8XMZLayXIFMQvYOyI+K3QwZmbWcFQ33PcOEfEO8DrQTVK33O0RManQwZmZWfFUdwVxEXA2cGOebQHsX5CIzMyakPIRXof85VW2ateKxy8cyLfaNI7Riqob7vvsdPGwiFiZu01Sq4JGZWbWRPTbuj0AJc3ER0tW8snSlY0mQWS5KffljGVmZlZJ1802Zva1g/nTybvWvHMDU10fxJZAF6C1pF1JhtkAaAtsXA+xmZlZEVXXB3EIMBToCvw+p/wL4L8KGJOZ2QZjTWkZzRvoE9bV9UGUP0F9XET8vR5jMjNrcr5KH5Q74dZX6LJZa5Z/VcqchcsB+MXgHTlr4DbFDC+vKtOWpFPSxR6SLqr8k+Xgkg6V9K6kmZIuzbO9u6RnJU2VNE5S15xtv5M0XdLbkm6SpMqvNzNrLD77Irmb6YtVa3jn4y/ou3V7jt21C0BFomhoqmtiapP+3mR9DpxOMvQn4CBgLvC6pEcj4q2c3W4A7o6IEZL2B64BTpU0ANgH2CXd70VgP2Dc+sRiZlZsZ+7bk127tWfHzm1p1aKkonzsu58WMarqVdfEdFv6++r1PPaewMyIeB9A0n3AUUBuguhN8rwFwFjg4fLTA62AliSd4y2AT9YzDjOzBmHXbpsVO4R1UmPPSNrU01ZSi7Q5aEFO81N1ugAf5qzPTctyTQGOTZePATaV1CEiXiFJGPPTn9H5JimSdLakCZImLFiwIENIZmaWVZau84MjYilwODCbZFTXS+ro/BcD+0l6g6QJaR5QKmlbYEeSO6i6APtLGlj5xRFxe0T0j4j+HTt2rKOQzMzqz6Llq7n7lQ94+I15xQ7lG7IkiPJmqMHAgxGxJOOx5wFb56x3TcsqRMRHEXFsROwKXJ6WLSa5mng1IpZFxDLgKWDvjOc1M2t07n/9w5p3qmdZRnN9XNI7wArgPEkdgZU1vAaSQf62k9STJDGcBJycu4OkzYGFEVEGXAbcmW6aAwyTdA1JH8R+wH9nOKeZWaMy+9rBHH/Ly8xcsIxT7/gXAKVlwZrS4Ih+W3HqXt2LFluNVxARcSkwAOgfEauBL0k6m2t63RrgfGA0yQRDD0TEdEnDJR2Z7jYIeFfSe0An4Ddp+SiSYcbfJOmnmBIRj61LxczMGosJHyxiwRerGD/jM17790JWl5bx9sdLeWzKR0WNK8uEQS2AU4Dvpo8iPA/cmuXgEfEk8GSlsitzlkeRJIPKrysFzslyDjOzxu4Xg3fkuXc+5dS9unNIny1p1kycdPsrlBV5ErosTUy3kNxm+ud0/dS07KxCBWVmtiE5a+A2DfJJ6iwJYo+I6Juz/pykKYUKyMzMGoYsdzGVSupVviJpG6C0cCGZmVlDkOUK4hJgrKT3Se4o6g6cUdCozMys6GpMEBHxrKTtgG+nRe9GxKrChmVmZsWW5S6mVsCPgX1JxkgaL+nWytOQmplZ05KlielukkmC/piunwzcA/ygUEGZmVnxZUkQO0VE75z1sZLeqnJvMzNrErLcxTRJ0l7lK5K+A0woXEhmZtYQZLmC2B14WdKcdL0byfAYbwIREbtU/VIzM2ussiSIQwsehZmZrWXq3CUs/6qUiKBYMy5nuc31g/oIxMzMvrb8q+R55HmLV9B1s42LEkOWPggzM6tnvz8hGeGotIgj9jlBmJk1QNM/WgrAE2/OL1oMThBmZg3Q8bt3BWDJ8tVFiyFTgpB0e3XrZmZWt3bs3JYWJaKkWXE6qCH7FcRtNaybmVkTkylBRMTE6tbNzKzpqfI2V0mPkQzOl1dEHFnVNjMza/yqew7ihnqLwszMGpwqE0REPF++LKk10C0i3q2XqMzMjNWlwVdryop2/hr7ICQdAUwGnk7X+0l6tMBxmZkZ8L8v/rto587SSX0VsCewGCAiJgM9CxaRmZmtJSKY+ekXPDrlI2Z+uqzezptlsL7VEbGk0mBRxXv228xsA9F1s9bMXbSCnpc9WVE2oFcH/jZsr2peVXeyXEFMl3QyUCJpO0l/BF4ucFxmZhu8FiXJR/TgnTtzwf7bArBydWm9nT/LFcQFwOXAKmAkMBr4VSGDMjMzGHvxoLXWJ81ZxKrV9ddpnWW47+XA5ZKuS1bji8KHZWZmxZblLqY90tnjpgJvSpoiaffCh2ZmZsWUpYnpDuDHETEeQNK+wP8BnmrUzKwJy9JJXVqeHAAi4kVgTeFCMjOzhqC6sZh2Sxefl3QbSQd1ACcC4wofmpmZ5Xpp5udAcidTqxYlBT9fdU1MN1Za/2XOsp+DMDMrknmLV9Cr4yYFP091YzF9r+BnNzOzzG78QV/+48EpvPbvhcVNELkkDQb6AK3KyyJieKGCMjOzb3rv0+Qpg8seepMhe3Yr+Pmy3OZ6K0m/wwWAgB8A3bMcXNKhkt6VNFPSpXm2d5f0rKSpksZJ6pqzrZukf0p6W9JbknpkrZSZWVP080N2oF3rFrRpWfj+B8h2F9OAiDgNWBQRVwN7A9vX9CJJJcCfgMOA3sAQSb0r7XYDcHdE7AIMB67J2XY3cH1E7EgyWOCnGWI1M2uySpqJo/ttRYvmWWeLrp0sZ1mR/l4uaStgNdA5w+v2BGZGxPsR8RVwH3BUpX16A8+ly2PLt6eJpHlEjAGIiGXpE91mZlZPsiSIxyW1B64HJgGzSW55rUkX4MOc9blpWa4pwLHp8jHAppI6kFyhLJb0kKQ3JF2fXpGsRdLZkiZImrBgwYIMIZmZWVY1JoiI+FVELI6Iv5P0PewQEVfU0fkvBvaT9AawHzAPKCXpPB+Ybt8D2AYYmie22yOif0T079ixYx2FZGZmUP2DcsdWs42IeKiGY88Dts5Z75qWVYiIj0ivICRtAhwXEYslzQUmR8T76baHgb1Ihv0wM7N6UN1trkdUsy2AmhLE68B2knqSJIaTgJNzd5C0ObAwIsqAy4A7c17bXlLHiFgA7A9MqOF8ZmZWh6p7UO6M2hw4ItZIOp9k/ogS4M6ImC5pODAhIh4FBgHXSArgBeAn6WtLJV0MPKtkKruJwF9qE4+Zma2bTA/Kra+IeBJ4slLZlTnLo4BRVbx2DB4x1sxsLZ8sXcXi5atZtaaUjZoX9nmI+rmZ1szM6sTT0z8G4OVZnxf8XE4QZmaNyH1n7wXA6jWFn3o0y1AbG0u6QtJf0vXtJB1e8MjMzOwbNtmooD0Da8lyBfF/wCqSITYguSPp1wWLyMzMGoQsCaJXRPyOZIgN0iEvVNCozMwsr7JIpuOZ9tHSgp8rS4L4SlJr0kmCJPUiuaIwM7N61kzJ9/P/Hf9+wc+VpTHrKuBpYGtJ9wL7kGfYCzMzK7ydurRj62+1pnO71gU/V40JIiL+KWkiyVAXAn4aEZ8VPDIzM8vrw4Ur+HDhClZ8VUrrAs4NkeUupseAg4FxEfG4k4OZWXF1+9bGACz/ak1Bz5OlD+IGkpFV35I0StLxklrV9CIzMyuMswb2rJfzZGlieh54Pp2PYX9gGMmgem0LHJuZmRVRpicu0ruYjiCZm3o3YEQhgzIzs+KrMUFIeoBk+tCngZuB59Phuc3MrAnLcgVxBzAkIkoLHYyZmTUc1c0ot39EPAe0AY6S1n54OsOMcmZmVgDT5yVPUU+as5iDencq2Hmqu4tpv/T3EXl+PFifmVmR7Ny1HQDTP1pS0PNUN6PcL9PF4RHx79xt6TSiZmZWBMfv3pVfPDyNls0LO2NDlqP/PU9Z3lngzMys6aiuD2IHoA/QTtKxOZvaAn5QzsysiavuLqZvk/Q1tCfpdyj3BcnDcmZm1oRV1wfxCPCIpL0j4pV6jMnMzBqA6pqYfp5OFHSypCGVt0fEhQWNzMzMiqq6Jqa3098T6iMQMzNrWKprYnos/V0x7pKkZsAmEVH4ue7MzKxaI1+bw/G7dWWLtoW5byjLfBB/k9RWUhtgGsmw35cUJBozM6tROi01Hy5cwVWPTS/YebI8B9E7vWI4GngK6AmcWrCIzMysWq1blrBHj80AaNWiiDPKAS0ktSBJEI9GxGogChaRmZnV6MFzB9ClfWuEat55PWVJELcBs0kG7XtBUnfAfRBmZk1clhnlbgJuyin6QNL3CheSmZk1BFk6qdtJ+r2kCenPjSRXE2Zm1oRlaWK6k2R4jRPSn6XA/xUyKDMzK74sM8r1iojjctavljS5QPGYmVkDkeUKYoWkfctXJO0DrChcSGZm1hBkuYI4F7hbUrt0fRFweuFCMjOzhqDaKwhJ/YDtgJOAXYBdImLXiJia5eCSDpX0rqSZki7Ns727pGclTZU0TlLXStvbSpor6ebMNTIzszpRZYKQdCXwAHAc8ARw4rqMwSSpBPgTcBjQGxgiqXel3W4A7o6IXYDhwDWVtv8KeCHrOc3MrO5UdwVxItAvIoYAewBnr+Ox9wRmRsT7EfEVcB9wVKV9egPPpctjc7dL2h3oBPxzHc9rZmZ1oLoEsSoilgNExOc17JtPF+DDnPW5aVmuKUD5dKbHAJtK6pCOGnsjcHF1J5B0dvnzGQsWLFjH8MzMrDrVdVJvI+nRdFlAr5x1IuLIOjj/xcDNkoaSNCXNA0qBHwNPRsRcqepxRiLiduB2gP79+3t8KDOzOlRdgqjcHHTDOh57HrB1znrXtKxCRHxEegUhaRPguIhYLGlvYKCkHwObAC0lLYuIb3R0m5lZYVQ3YdDztTz268B2knqSJIaTgJNzd5C0ObAwIsqAy0ie2iYifpizz1Cgv5ODmVn9qu4upsckHZEO9V152zaShks6s6rXR8Qa4HxgNMn0pQ9ExPT0deXNU4OAdyW9R9Ih/Zta1MXMzOpQdU1Mw4CLgP+WtBBYALQCegCzgJsj4pHqDh4RTwJPViq7Mmd5FDCqhmPcBdxV3T5mZlb3qmti+hj4OfBzST2AziRDbLxXfneTmZk1XVmG2iAiZpNMGmRmZhuIdX22wczMGoh5i1fw90lziSjMXf5OEGZmjVxZgZ4Cc4IwM2ukLjpo+4Iev8Y+iHT+h6uA7un+AiIitiloZGZmVlRZOqnvAH4GTCQZBsPMzDYAWRLEkoh4quCRmJlZg5IlQYyVdD3wELCqvDAiJhUsKjMzK7osCeI76e/+OWUB7F/34ZiZWUNRY4KIiO/VRyBmZtaw1Hibq6R2kn5fPjGPpBsltauP4MzMrHiyPAdxJ/AFcEL6sxT4v0IGZWZmxZelD6JXRByXs361pMkFisfMzBqILFcQKyTtW76SPji3onAhmZlZQ5DlCuI8YETa7yBgITC0kEGZmVnxZbmLaTLQV1LbdH1poYMyM7PiqzJBSDolIv4q6aJK5QBExO8LHJuZmRVRdVcQbdLfm9ZHIGZm1rBUN+Xobenvq+svHDMzayiyPCj3O0ltJbWQ9KykBZJOqY/gzMyseLLc5npw2jF9OMm81NsClxQyKDMzK74sCaK8GWow8GBELClgPGZm1kBkeQ7icUnvkDwcd56kjsDKwoZlZmbFVuMVRERcCgwA+kfEauBL4KhCB2ZmZsVV3XMQ+0fEc5KOzSnL3eWhQgZmZmbFVV0T037Ac8ARebYFThBmZk1adc9B/DL9fUb9hWNmZg1Flucgfiupfc76ZpJ+XdCozMys6LLc5npYRCwuX4mIRcD3CxaRmZk1CFkSRImkjcpXJLUGNqpmfzMzawKyPAdxL/CspPJpRs8ARhQuJDMzawiyzAdxnaQpwIFp0a8iYnRhwzIzs2LLcgUB8DawJiKekbSxpE0j4otCBmZmZsWV5S6mYcAo4La0qAvwcJaDSzpU0ruSZkq6NM/27ukIsVMljZPUNS3vJ+kVSdPTbSdmrpGZmdWJLJ3UPwH2AZYCRMQMYIuaXiSpBPgTcBjQGxgiqXel3W4A7o6IXYDhwDVp+XLgtIjoAxwK/HfurbZmZlZ4WRLEqoj4qnxFUnOSJ6lrsicwMyLeT19/H98cw6k3ydPaAGPLt0fEe2kiIiI+Aj4FOmY4p5mZ1ZEsCeJ5Sf8FtJZ0EPAg8FiG13UBPsxZn5uW5ZoClI/1dAywqaQOuTtI2hNoCcyqfAJJZ0uaIGnCggULMoRkZmZZZUkQ/wksAN4EzgGeBH5RR+e/GNhP0hskYz/NA0rLN0rqDNwDnBERZZVfHBG3R0T/iOjfsaMvMMzM6lK1dzGl/QjTI2IH4C/reOx5wNY5613Tsgpp89Gx6bk2AY4rf2pbUlvgCeDyiHh1Hc9tZma1VO0VRESUAu9K6rYex34d2E5ST0ktgZOAR3N3kLS5pPIYLgPuTMtbAv8g6cAetR7nNjOzWsryHMRmwHRJr5FMFgRARBxZ3YsiYo2k84HRQAlwZ0RMlzQcmBARjwKDgGskBfACyR1TACcA3wU6SBqalg2NiMlZK2ZmZrWTJUFcsb4Hj4gnSfoscsuuzFkeRfKMReXX/RX46/qe18zMaq+6GeVaAecC25J0UN8REWvqKzAzMyuu6vogRgD9SZLDYcCN9RKRmZk1CNU1MfWOiJ0BJN0BvFY/IZmZWUNQ3RXE6vIFNy2ZmW14qruC6Ctpaboskiepl6bLERFtCx6dmZkVTZUJIiJK6jMQMzNrWLIMtWFmZhugrBMGNUqrV69m7ty5rFy5stihWCPSqlUrunbtSosWLYodillRNekEMXfuXDbddFN69OiBpGKHY41ARPD5558zd+5cevbsWexwzIqqSTcxrVy5kg4dOjg5WGaS6NChg686zWjiCQJwcrB15n8zZokmnyDMzGz9OEEU2Mcff8xJJ51Er1692H333fn+97/Pe++9x+zZs9lpp53q7DxXXnklzzzzDADjx4+nT58+9OvXj3nz5nH88cfX6tgRwf7778/SpUsryh5++GEk8c4771SUzZ49m9atW9OvXz969+7NueeeS1nZN+Z5WicvvPACu+22G82bN2fUqKpHfp84cSI777wz2267LRdeeCERyay4Cxcu5KCDDmK77bbjoIMOYtGiRQA8/vjjXHnllVUez8ycIAoqIjjmmGMYNGgQs2bNYuLEiVxzzTV88skndX6u4cOHc+CBBwJw7733ctlllzF58mS6dOlS7QdrZWvWfPOh+SeffJK+ffvStu3Xz0aOHDmSfffdl5EjR661b69evZg8eTJTp07lrbfe4uGHH16/CqW6devGXXfdxcknn1ztfueddx5/+ctfmDFjBjNmzODpp58G4Nprr+WAAw5gxowZHHDAAVx77bUADB48mMcee4zly5fXKj6zpqxJ38WU6+rHpvPWR0tr3nEd9N6qLb88ok+V28eOHUuLFi0499xzK8r69u0LJN+2y82ePZtTTz2VL79Mptu4+eabGTBgAPPnz+fEE09k6dKlrFmzhltuuYUBAwbwox/9iAkTJiCJM888k5/97GcMHTqUww8/nMWLF/PAAw8wevRonnrqKX7zm99w+OGHM23aNEpLS7n00ksZN24cq1at4ic/+QnnnHMO48aN44orrmCzzTbjnXfe4b333lurHvfeey9nn312xfqyZct48cUXGTt2LEcccQRXX331N+revHlzBgwYwMyZM9frvS3Xo0cPAJo1q/q7zPz581m6dCl77bUXAKeddhoPP/wwhx12GI888gjjxo0D4PTTT2fQoEFcd911SGLQoEE8/vjjnHDCCbWK0ayp2mASRDFMmzaN3Xffvcb9tthiC8aMGUOrVq2YMWMGQ4YMYcKECfztb3/jkEMO4fLLL6e0tJTly5czefJk5s2bx7Rp0wBYvHjxWsc666yzePHFFzn88MM5/vjj10pEd9xxB+3ateP1119n1apV7LPPPhx88MEATJo0iWnTpuW9tfOll17itttuq1h/5JFHOPTQQ9l+++3p0KEDEydO/EY9ly9fzrPPPsvw4cO/cbyBAwfyxRdffKP8hhtuqLgKWhfz5s2ja9euFetdu3Zl3rxkdttPPvmEzp07A7DllluudfXWv39/xo8f7wRhVoUNJkFU902/2FavXs3555/P5MmTKSkpqfgGv8cee3DmmWeyevVqjj76aPr168c222zD+++/zwUXXMDgwYMrPuCz+Oc//8nUqVMrmpyWLFnCjBkzaNmyJXvuuWeV9/0vXLiQTTfdtGJ95MiR/PSnPwXgpJNOYuTIkRUJYtasWfTr1w9JHHXUURx22GHfON748eMzx1yXJK11h9IWW2zBRx99VJRYzBqDDSZBFEOfPn0ytf//4Q9/oFOnTkyZMoWysjJatWoFwHe/+11eeOEFnnjiCYYOHcpFF13EaaedxpQpUxg9ejS33norDzzwAHfeeWemeCKCP/7xjxxyyCFrlY8bN442bdpU+brmzZtTVlZGs2bNWLhwIc899xxvvvkmkigtLUUS119/PfB1H0R16voKokuXLsydO7dife7cuXTp0gWATp06MX/+fDp37sz8+fPZYostKvZbuXIlrVu3XufzmTUUf/vXHADGvvMpB/buVOfHdyd1Ae2///6sWrWK22+/vaJs6tSp3/gGvWTJEjp37kyzZs245557KC0tBeCDDz6gU6dODBs2jLPOOotJkybx2WefUVZWxnHHHcevf/1rJk2alDmeQw45hFtuuYXVq5OR3N97772Kfo/qfPvb3+b9998HYNSoUZx66ql88MEHzJ49mw8//JCePXuu01XB+PHjmTx58jd+1ic5AHTu3Jm2bdvy6quvEhHcfffdHHXUUQAceeSRjBgxAoARI0ZUlENS/7q8k8ysvp2//7YAzP685v/H68MJooAk8Y9//INnnnmGXr160adPHy677DK23HLLtfb78Y9/zIgRI+jbty/vvPNOxbf5cePG0bdvX3bddVfuv/9+fvrTnzJv3jwGDRpEv379OOWUU7jmmmsyx3PWWWfRu3dvdtttN3baaSfOOeecvHctVTZ48OCKjt6RI0dyzDHHrLX9uOOO+8bdTHXl9ddfp2vXrjz44IOcc8459OnzdVNhv379Kpb//Oc/c9ZZZ7HtttvSq1eviqatSy+9lDFjxrDddtvxzDPPcOmll1a8ZuzYsQwePLggcZvVhwG9OgDw6yfeLsjxVX6/eGPXv3//mDBhwlplb7/9NjvuuGORImo65s+fz2mnncaYMWOKHUqd+eSTTzj55JN59tln8273vx1rDMrKgm3+60mGDezJ5YN7r9cxJE2MiP75trkPwmrUuXNnhg0bxtKlS9d6FqIxmzNnDjfe6GnWrXFr1kzMvrZwV8FOEJZJU7sVdI899ih2CGYNXpPvg2gqTWhWf/xvxizRpBNEq1at+Pzzz/0f3jIrnw+i/FZjsw1Zk25i6tq1K3PnzmXBggXFDsUakfIZ5cw2dE06QbRo0cKzgpmZracm3cRkZmbrzwnCzMzycoIwM7O8msyT1JIWAB/U4hCbA5/VUTiNxYZW5w2tvuA6byhqU+fuEdEx34YmkyBqS9KEqh43b6o2tDpvaPUF13lDUag6u4nJzMzycoIwM7O8nCC+dnvNuzQ5G1qdN7T6guu8oShInd0HYWZmefkKwszM8nKCMDOzvDaoBCHpUEnvSpop6dI82zeSdH+6/V+SehQhzDqVoc4XSXpL0lRJz0rqXow461JNdc7Z7zhJIanR3xKZpc6STkj/1tMl/a2+Y6xrGf5td5M0VtIb6b/v7xcjzroi6U5Jn0qaVsV2SbopfT+mStqt1ieNiA3iBygBZgHbAC2BKUDvSvv8GLg1XT4JuL/YcddDnb8HbJwun7ch1Dndb1PgBeBVoH+x466Hv/N2wBvAZun6FsWOux7qfDtwXrrcG5hd7LhrWefvArsB06rY/n3gKUDAXsC/anvODekKYk9gZkS8HxFfAfcBR1Xa5yhgRLo8CjhAkuoxxrpWY50jYmxELE9XXwUa+zjXWf7OAL8CrgNW1mdwBZKlzsOAP0XEIoCI+LSeY6xrWeocQPkcue2Aj+oxvjoXES8AC6vZ5Sjg7ki8CrSX1Lk259yQEkQX4MOc9blpWd59ImINsAToUC/RFUaWOuf6Eck3kMasxjqnl95bR8QT9RlYAWX5O28PbC/pJUmvSjq03qIrjCx1vgo4RdJc4EnggvoJrWjW9f97jZr0fBCWnaRTgP7AfsWOpZAkNQN+Dwwtcij1rTlJM9MgkqvEFyTtHBGLixlUgQ0B7oqIGyXtDdwjaaeIKCt2YI3FhnQFMQ/YOme9a1qWdx9JzUkuSz+vl+gKI0udkXQgcDlwZESsqqfYCqWmOm8K7ASMkzSbpK320UbeUZ3l7zwXeDQiVkfEv4H3SBJGY5Wlzj8CHgCIiFeAViSD2jVVmf6/r4sNKUG8DmwnqaekliSd0I9W2udR4PR0+XjguUh7fxqpGussaVfgNpLk0NjbpaGGOkfEkojYPCJ6REQPkn6XIyNiQnHCrRNZ/m0/THL1gKTNSZqc3q/HGOtaljrPAQ4AkLQjSYJoyvMPPwqclt7NtBewJCLm1+aAG0wTU0SskXQ+MJrkDog7I2K6pOHAhIh4FLiD5DJ0Jkln0EnFi7j2Mtb5emAT4MG0P35ORBxZtKBrKWOdm5SMdR4NHCzpLaAUuCQiGu3VccY6/wfwF0k/I+mwHtqYv/BJGkmS5DdP+1V+CbQAiIhbSfpZvg/MBJYDZ9T6nI34/TIzswLakJqYzMxsHThBmJlZXk4QZmaWlxOEmZnl5QRhZmZ5OUHYepNUKmmypGmSHpPUvo6PPzu9Zx9Jy6rYp7Wk5yWVSOohaUUa01uSbk2fnF6Xc/aXdFO6PEjSgJxt50o6rTZ1So9zlaSLa9jnLknHr8Mxe1Q1ymddk3Rk+eipko6W1Dtn2/D0wcv1Oe59khrzw3tNjhOE1caKiOgXETuRPDfykyLEcCbwUESUpuuzIqIfsAvJCJ5Hr8vBImJCRFyYrg4CBuRsuzUi7q5twI1dRDwaEdemq0eTvM/l266MiGfW89C3AD+vZXhWh5wgrK68QjowmKRekp6WNFHSeEk7pOWdJP1D0pT0Z0Ba/nC673RJZ6/jeX8IPFK5MB1s8WVg2/Tb9XP6es6Lbul5f5Be/UyR9EJaNkjS40rmAjkX+Fl6RTKw/Ju/pB0kvVZ+rvT4b6bLu6dXNBMljVYNo2lKGibp9TSGv0vaOGfzgZImSHpP0uHp/iWSrk9fM1XSOevyZklaJukP6Xv9rKSOaXk/JYP4TU3/Rpul5Rfq6/lC7kvLhkq6Of37HQlcn75HvcqvfJTM1fBgznkHSXo8XT5Y0iuSJkl6UNIm6W7j0zpvMA/wNnROEFZrkkpIhjQof0r5duCCiNgduBj4c1p+E/B8RPQlGdd+elp+Zrpvf+BCSZlG0FUyxMI2ETE7z7aN05jeBP4IjIiIXYB70zgArgQOSeNZ6+nx9Ji3An9Ir5LG52x7B2gpqWdadCJwv6QW6bmOT+tzJ/CbGqrxUETskcbwNsn4QeV6kAxrPRi4VVKrdPuSiNgD2AMYlhNHed23kvRkFedrQ/KkcR/geZKncQHuBv4zfY/ezCm/FNg1LT+30nv0Msnf/JL0PZqVs/kZ4DuS2qTrJwL3KWky/AVwYETsBkwALkqPV0byFHDfqt8uq09OEFYbrSVNBj4GOgFj0m+DA0iG7phMMs5T+bfo/UmaEYiI0ohYkpZfKGkKybhIW5N9ELnNgcWVynql530JeCIingL2BspnULsH2Dddfgm4S9IwkuEa1sUDJB96pL/vB75NMhDgmDSGX1Dz/Bo7pVdZb5JcDfXJPUdElEXEDJJxk3YADiYZb2cy8C+S4ejXer8i4qOIqGr2tLI0VoC/AvtKage0j4jn0/IRJJPTAEwF7lUy2u+aGuqSG8Ma4GngiPSKYDDJld5eJE1SL6V1OB3IncXwU2CrrOexwvKlnNXGiojol35bH03SB3EXsDjtB6iRpEHAgcDeEbFc0jiSQdUynT/PvrOynjsizpX0HZIPr4mSds94Xkg+ZB+U9FByqJghaWdgekTsvQ7HuQs4OiKmSBpKOqBeeYiVQyaZLeyCiBidu0HrPz1uTWPtDCZJFkcAl6d1zOo+4HyS/qkJEfGFJAFjImJIFa9pRfJ3tQbAVxBWa+mMdBeSDI62HPi3pB9AxTy55U0Gz5JMa1relt6OZEj1RWly2IHkG2bW8y4CStKml+q8zNcDL/6QpK0bSb0i4l8RcSXJKJ9bV3rdFyTDg+c79yySQe+u4Otv5O8CHZXMPYCkFpL65Ht9jk2B+Wnz1A8rbfuBpGaSepFMrfkuSSI+L90fSdvnNONk0YxkpGKAk4EX0yu5RZIGpuWnAs8ruQNs64gYC/wnyd9qk0rHq/I9ImnC2o1kNrv70rJXgX0kbZvG30bS9jmv2R6ol7uxrGZOEFYnIuINkuaIISQfdD9Km42m8/VUkD8Fvpc2p0wkaWp4Gmgu6W3gWpIPkHXxT75uMqrKBcAZkqaSfPj9NC2/XtKbSm4PfZlkXuNcjwHHpB2wA/mm+4FT+HrOga9IPnyvS+s+mZy7oKpwBUlT0UvAO5W2zQFeI5nl79yIWAn8L/AWMCmN+zYqtQTU0AfxJbBn+tr9geFp+ekk78dUoF9aXgL8Nf17vQHclGeCofuASyS9kSayCumdZY8Dh6W/iYgFJJM1jUzP9QpJ0xmSOpFclX5cRexWzzyaqzVqSqYP/VlEnFrsWBoDScsiovJVQIOgZFjupRFxR7FjsYSvIKxRi4hJwNj0Tipr3BaTdJBbA+ErCDMzy8tXEGZmlpcThJmZ5eUEYWZmeTlBmJlZXk4QZmaW1/8HwFhzzjh8/JsAAAAASUVORK5CYII=",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"evaluate_emeddings_approach(labels=['An Amazon review with a negative sentiment.', 'An Amazon review with a positive sentiment.'], engine='text-similarity-babbage-001')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Using the search embeddings and descriptive names leads to an additional improvement in performance."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" precision recall f1-score support\n",
"\n",
" negative 0.77 0.79 0.78 136\n",
" positive 0.96 0.96 0.96 789\n",
"\n",
" accuracy 0.94 925\n",
" macro avg 0.87 0.88 0.87 925\n",
"weighted avg 0.94 0.94 0.94 925\n",
"\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Z1A+gAAAACXBIWXMAAAsTAAALEwEAmpwYAAAsFUlEQVR4nO3deZhU1bnv8e+PQVEBNUwiqCBqDCgQRaIYI3FWcDbOGmdxvvGYHL25TmiixiEnMSZqjkQ0hmiIUVQUUUAxiVFQQHAARFQGFWVSmeG9f+zdbdFUd22gq6uH3+d5+mFPtfe7qpt6a62191qKCMzMzCpqVOoAzMysdnKCMDOzvJwgzMwsLycIMzPLywnCzMzycoIwM7O8nCCsWkk6S9IrpY6jOkk6TdLzGY67V9K1NRFTTZA0U9JB6fINkv5c6pisZjlBGJI2lfSApA8lfSlpgqTDSx1XFumH2FJJX0n6VNKDkppX5zUi4pGIOCTDcQMi4qbqvHYZSSHp67ScsyXdJalxMa5lVsYJwgCaAB8D+wNbAv8PeExSp1IGtR6OjIjmwB5AL5L41yKpSY1HVf16pOXcHzgJOKfE8VSrevI7qlecIIyI+DoiboiImRGxJiKeBj4A9qzsNZK2k/S4pHmSvpD0u0qO+42kjyUtljRe0n45+3pLGpfu+1TSXen2ZpL+nJ53oaTXJbXLUI7ZwLPAbul5QtIlkqYB09Jt/dMa0kJJ/5LUvVCZcpvNlPi1pM/SuN+SVHa9ByXdnHO+8yVNlzRf0jBJ2+bsC0kDJE1LY7lHkgqVMS3ndOCfQM+c821IubpIGpVu+1zSI5K2yhJDRZKOTq+/WNL7kg5Lt5c3U6Xr5U1Vkjql78O5kj4CRkl6VtKlFc49UdJx6fKukkam7+l7kk7ckHgtGycIW0f6YbwLMKWS/Y2Bp4EPgU5AB+CvlZzudZIPsm8BfwH+JqlZuu83wG8ioiXQBXgs3f5jkprMdkArYACwNEPc2wFHAG/mbD4G+B7QVdJ3gUHAhel57wOGpU1sWct0CPADkvdnS+BE4Is8sRwA3JLub5+et+L5+gN7Ad3T4w4tVMb03LsC+wHT0/UNLZfSGLcFvkPyft+QJYYK8fQGHgJ+CmxF8v7MXI9T7J9e/1BgCHBKzrm7AjsAz0jaAhhJ8nfUFjgZ+H16jBWBE4StRVJT4BFgcES8W8lhvUk+VH6a1j6WRUTejumI+HNEfBERqyLiTmBT4Nvp7pXATpJaR8RXEfFqzvZWwE4RsToixkfE4irCfkLSQuAV4CXglzn7bomI+RGxFLgAuC8i/pOedzCwHNh7Pcq0EmgB7AooIt6JiLl5jjsNGBQRb0TEcuAaYJ8KzXa3RsTCiPgIGE1OjaASb0j6GngHGAP8Pt2+QeWKiOkRMTIilkfEPOAukg/r9XVuWtaRaQ10dhV/O/nckMa2FPgH0FPSDum+04DH0/ewPzAzIv6U/j29Cfwd+NEGxGwZOEFYOUmNgIeBFcClOdufVdI5+pWk00i+aX4YEasynPMqSe9IWpR+iG8JtE53n0vyTfzdtBmpf7r9YWAE8FdJcyT9Kk1clTkmIraKiB0i4uL0g6bMxznLOwD/lTbDLEzj2Y7kAzRTmSJiFPA74B7gM0n3S2qZ59BtSb61l73uK5KaRoecYz7JWV4CNAeQNCXn/d4v55g90mNOIqkVbbEx5ZLUTtJflXR6Lwb+zDe/m/WxHfD+BryuTPnvKCK+BJ4hqR1AUpt4JF3eAfhehXKeBmyzEde2KjhBGJC0rQMPAO2A4yNiZdm+iDg8IpqnP4+Q/IfeXgU6FdMPt5+RNJ9sHRFbAYtImjaIiGkRcQpJc8FtwFBJW0TEyoi4MSK6An1IvjmeuYFFyx2u+GPgF2kyKfvZPCKGZC1TGvdvI2JPoCtJgvtpnsPmkHygAZA2j7QCZmc4f7ec93tshX0REY8B/wau28hy/ZLk/dk9beY7nfR3s54+JmkizOdrYPOc9Xwf5hWHlB4CnCJpH6AZSe2q7DovVShn84i4aANitgycIKzMH0jagY+s8A08n9eAucCtkrZQ0qm8b57jWgCrgHlAE0nXAeXftiWdLqlNRKwBFqab10j6oaTd0/bzxSTNOms2pnCpPwIDJH1PiS0k9ZPUImuZJO2Vvr4pyYffskpiGwKcLamnpE1JPoz/ExEzq6EcALcC50vaZiPK1QL4ClgkqQP5E10WD5CU9UBJjSR1SPtJACYAJ0tqKqkXcEKG8w0nSa4DgUfTvw9I+lJ2kXRGer6m6e/jOxsYtxXgBGGk7b0XkrSBf1KhOWkdEbEaOBLYCfgImEXS7FHRCOA5YCpJc8sy1m7yOQyYIukrkg7rk9PktA0wlCQ5vEPSr/DwRhaTiBgHnE/SRLSApJP3rPUsU0uSD+QFaZm+AG7Pc60XgGtJ2sjnknzDPrnicRtRlreAl0n6Fja0XDeSNFstImnWeXwDY3kNOBv4dXqul/im9nQtSdkXpNf7S4bzLU9jOSj3+LT56RCS93EOSRPdbST9WlYE8oRBZmaWj2sQZmaWlxOEmZnl5QRhZmZ5OUGYmVle9WZwrNatW0enTp1KHYaZWZ0yfvz4zyOiTb599SZBdOrUiXHjxpU6DDOzOkXSh5XtcxOTmZnl5QRhZmZ5OUGYmVleThBmZpaXE4SZmeVVtAQhaZCSaRknV7Jfkn6rZErGSZL2yNn3YyVTMU6T9ONixWhmZpUrZg3iQZLROitzOLBz+nMByXDTSPoWcD3JhCi9geslbV3EOM3MLI+iPQcRES9XmF6xoqOBhyIZTvZVSVtJag/0BUZGxHwASSNJEs2QYsS5ZMUq7h2zMZNhmZll9+1tWtKve/tSh5FJKR+U68DacwPMSrdVtn0dki4gqX2w/fbbb1AQS1es5u7R0zfotWZm6yMCWjRr4gRREyLifuB+gF69em3QxBatmm/KB7f0q9a4zMzyufnptxny2kelDiOzUt7FNJtksvMyHdNtlW03M6vTPvj8a75esZpfDn+Hu0ZOZemK1aUOqUqlTBDDgDPTu5n2BhZFxFySaSoPkbR12jl9SLrNzKxOW5PO4Pngv2by2xen8eZHC0ocUdWK1sQkaQhJh3NrSbNI7kxqChAR95JMTH4Eyfy5S0jmtCUi5ku6CXg9PdXAsg5rM7O67E9n9wbg9Znz+dG9/2ZNLZ/xuZh3MZ1SYH8Al1SybxAwqBhxmZmVWlnT0pWPTeB7O7aiSSPx34ftyjZbNitxZGvzk9RmZjVs262SRPDZl8t548MF/OPN2fzngy9KHNW66vRdTGZmddFObVsw89bk7skZ877igDtfKnFE+bkGYWZmeTlBmJlZXk4QZmaWlxOEmVkJLV2Z3NF096jaN+SPE4SZWQl9u10LAFatXlPiSNblu5jMzEqoSeNG7NVpa5o2rn3f12tfRGZmVis4QZiZWV5OEGZmlpcThJmZ5eUEYWZmeTlBmJlZXk4QZmaWlxOEmZnl5QRhZlZi733yJf96/wv+Of3zUoeyFicIM7MSW7xsFQA/GzqpxJGszQnCzKzEpt58OP27t2fVmto1HpMThJlZiW3SpBFvz13Mp4uXs2jJylKHU84JwsysFli+Mqk9vPnxghJH8g0nCDOzWuDuU79b6hDW4QRhZmZ5OUGYmVleThBmZpaXE4SZmeXlBGFmZnk5QZiZWV5NshwkqS2wL7AtsBSYDIyLiNr12J+ZmVWbKhOEpB8CVwPfAt4EPgOaAccAXSQNBe6MiMVFjtPMzGpYoRrEEcD5EfFRxR2SmgD9gYOBvxchNjMzK6EqE0RE/LSKfauAJ6o7IDMzqx02uJNa0tnVGYiZmdUuG3MX043VFoWZmdU6hTqpK5u9QkC76g/HzMxqi0Kd1O2AQ4GK488K+Fehk0s6DPgN0Bj434i4tcL+HYBBQBtgPnB6RMxK9/0K6EdSyxkJXBERUeiaZmZWPQo1MT0NNI+IDyv8zATGVPVCSY2Be4DDga7AKZK6VjjsDuChiOgODARuSV/bh+S5i+7AbsBewP7rUzAzM9s4VSaIiDg3Il6pZN+pBc7dG5geETMiYgXwV+DoCsd0BUaly6Nz9gfJ8xabAJsCTYFPC1zPzMyqUTGH2ugAfJyzPivdlmsicFy6fCzQQlKriPg3ScKYm/6MiIh3Kl5A0gWSxkkaN2/evGovgJlZQ1bqsZiuAvaX9CZJE9JsYLWknYDvAB1JksoBkvar+OKIuD8iekVErzZt2tRk3GZm1Wrel8sBuH3EeyWO5BuZxmLaQLOB7XLWO6bbykXEHNIahKTmwPERsVDS+cCrEfFVuu9ZYB9gbBHjNTMrmQN2bQtA6+abljiSbxSzBvE6sLOkzpI2AU4GhuUeIKm1pLIYriG5owngI5KaRRNJTUlqF+s0MZmZ1RdNGzdi57bN2WLTxqUOpVzmBCHp/qrWK0qH4rgUGEHy4f5YREyRNFDSUelhfYH3JE0luaX2F+n2ocD7wFsk/RQTI+KprLGamdnGW58mpvsKrK8jIoYDwytsuy5neShJMqj4utXAhesRm5mZVbPMNYiIGF/VupmZ1S+Fhtp4iuSZhLwi4qjK9pmZWd1WqInpjhqJwszMap1C80G8VLYsaTNg+4ioPTfpmplZ0WTqg5B0JDABeC5d7ylpWJUvMjOzOi1rJ/UNJGMrLQSIiAlA56JEZGZmtULWBLEyIhZV2Oaht83M6rGsz0FMkXQq0FjSzsDlZJgPwszM6q6sNYjLgG7AcmAIsBj4P0WKyczMaoFMNYiIWAL8XNJtyWp8WdywzMys1LLexbSXpLeAScBbkiZK2rO4oZmZWSll7YN4ALg4IsYCSPo+8CeSKUHNzKweytoHsbosOQCk05CuKk5IZmZWGxQai2mPdPElSfeRdFAHcBIwprihmZlZKRVqYrqzwvr1Oct+DsLMrBpN++wrpn32Fa99MJ/enb9V6nAKjsX0w5oKxMysoWskWBPw1uxFtT9B5JLUj+RZiGZl2yJiYDGCMjNriN687hB63Ph8qcMol/U213tJ+h0uAwT8CNihiHGZmVmJZb2LqU9EnAksiIgbgX2AXYoXlpmZlVrWBLE0/XeJpG2BlUD74oRkZma1QdY+iKclbQXcDrxBcgfT/xYrKDMzK72sYzHdlC7+XdLTQLM8w3+bmVk9UuhBueOq2EdEPF79IZmZWW1QqAZxZBX7AnCCMDOrpwo9KHd2TQViZma1S9a7mMzMrIFxgjAzq4UigojSDnnnBGFmVsvc9PTbdL5mON2uH8HiZStLFkfWoTY2l3StpD+m6ztL6l/c0MzMGpYmjbTW+pIVq1nw9YoSRZO9BvEnYDnJEBsAs4GbixKRmVkDtcWmTRj1X/vz7k2HcdPR3QDY//Yx9Lp5JC9NnVfj8WRNEF0i4lckQ2wQEUtIBu0zM7NqtGOb5jRr2pjPvlxevu3zr1bw3ieLazyWrENtrJC0GekkQZK6kNQozMysCK48eBdO33sHmm/ahG7XjyhJDFkTxA3Ac8B2kh4B9gXOKlJMZmYNniTatWzG18tXlSyGrGMxPS9pPLA3SdPSFRHxeVEjMzOzksqUICQ9BfwFGBYRXxc3JDMzqw2ydlLfAewHvC1pqKQTJDUr9CJJh0l6T9J0SVfn2b+DpBclTZI0RlLHnH3bS3pe0juS3pbUKWuhzMxs42VKEBHxUkRcDOwI3AecCHxW1WskNQbuAQ4HugKnSOpa4bA7gIciojswELglZ99DwO0R8R2gd6HrmZlZ9cr8JHV6F9PxwABgL2BwgZf0BqZHxIyIWAH8FTi6wjFdgVHp8uiy/WkiaRIRIwEi4qv01lozM6shWZ+kfgx4BzgA+B3JcxGXFXhZB+DjnPVZ6bZcE4GyOSeOBVpIakUy3/VCSY9LelPS7WmNpGJcF0gaJ2ncvHk1/xCJmVl9lrUG8QBJUhgQEaMjYk01Xf8qYH9JbwL7kzyhvZqk83y/dP9eJE1bZ1V8cUTcHxG9IqJXmzZtqikkM7PaZ9S7Nd/KXmhGuQMiYhSwBXC0tPbD0wVmlJsNbJez3jHdlvv6OaQ1CEnNgeMjYqGkWcCEiJiR7nuC5BbbBzKUycys3mjSOPncfXXG/Jq/doH9+5P0EeSbWa7QjHKvAztL6kySGE4GTs09QFJrYH5aI7kGGJTz2q0ktYmIeSRNW+MKxGpmVu9s2qQx/bq35925tWyojYi4Pl0cGBEf5O5LP/ireu0qSZcCI4DGwKCImCJpIDAuIoYBfYFbJAXwMnBJ+trVkq4CXlRSbRkP/HG9S2dmZhss61Abfwf2qLBtKLBnVS+KiOHA8ArbrstZHpqeJ99rRwLdM8ZnZmbVrFAfxK5AN2BLScfl7GoJFHxQzszM6q5CNYhvA/2BrVi7H+JL4PwixWRmZrVAoT6IJ4EnJe0TEf+uoZjMzCzHZ4uX8f68rzny7leQoFnTxtx1Yg86br15Ua9bqInpZ+lEQadKOqXi/oi4vGiRmZkZAK/PXADAnIVL6dx6C177YD5vz1lc2gRB8vQ0+BZTM7OS6de9Pc9MmsvY//4hM+Z9Tf+7X6mR6xZqYnoq/bd83CVJjYDmEVHzN+WamTVA95y6B/ecWvi46pZ1LKa/SGopaQtgMsmw3z8tbmhmZlZKWcdi6prWGI4BngU6A2cUKygzMyu9rAmiqaSmJAliWESsJBlqw8zM6qmsCeI+YCbJoH0vS9oBcB+EmVmJLF62qujXyDqj3G8jokNEHBGJD4EfFjk2MzOrYPGylQBc/fdJRb9W1k7qLSXdVTY5j6Q7SWoTZmZWg/p0aQ3A93duXfRrZW1iGkQyvMaJ6c9i4E/FCsrMzCrXbduWNGmkwgdupKyjuXaJiONz1m+UNKEI8ZiZWS2RtQaxVNL3y1Yk7QssLU5IZmZWldkLlzJp1qKiXydrDWIA8JCkLdP1BcCPixOSmZlVZeGSpKP6y2UradGsadGuUzBBSOoJ7EQyZehsAA+zYWZWOv27t+fpSXNZsWpNUa9TZROTpOuAx4DjgWeAk5wczMxKq3fnb9XIdQrVIE4CekbEEkmtgOfw3NBmZg1CoU7q5RGxBCAivshwvJmZ1ROFahA7ShqWLgvokrNORBxVtMjMzKykCiWIoyus31GsQMzMrHYpNGHQSzUViJmZ1S6F7mJ6StKR6VDfFfftKGmgpHOKF56ZmZVKoSam84Ergf+RNB+YBzQDOgHvA7+LiCeLGqGZmZVEoSamT4CfAT+T1AloTzLExtSyu5vMzKx+yjrUBhExk2TSIDMzawD8XIOZmeXlBGFmZnk5QZiZWV6Z+iDS+R9uAHZIXyMgImLH4oVmZmallLWT+gHgJ8B4YHXxwjEzs9oia4JYFBHPFjUSMzOrVbImiNGSbgceB5aXbYyIN4oSlZmZlVzWBPG99N9eOdsCOKB6wzEzs9oiU4KIiB9uyMklHQb8BmgM/G9E3Fph/w7AIKANMB84PSJm5exvCbwNPBERl25IDGZmtmEy3eYqaUtJd0kal/7cKWnLAq9pDNwDHA50BU6R1LXCYXcAD0VEd2AgcEuF/TcBL2eJ0czMqlfW5yAGAV8CJ6Y/i4E/FXhNb2B6RMyIiBXAX1l3fomuwKh0eXTufkl7Au2A5zPGaGZm1ShrgugSEdenH/YzIuJGoNAzEB2Aj3PWZ6Xbck0EjkuXjwVaSGolqRFwJ3BVVReQdEFZrWbevHkZi2JmZllkTRBLJX2/bCV9cG5pNVz/KmB/SW8C+wOzSZ6zuBgYntsfkU9E3B8RvSKiV5s2baohHDMzK5P1LqaLgMFpv4NIOpTPKvCa2cB2Oesd023lImIOaQ1CUnPg+IhYKGkfYD9JFwPNgU0kfRURV2eM18zMNlLWu5gmAD3Su4qIiMUZXvY6sLOkziSJ4WTg1NwDJLUG5kfEGuAakr4OIuK0nGPOAno5OZiZ1awqE4Sk0yPiz5KurLAdgIi4q7LXRsQqSZcCI0hucx0UEVMkDQTGRcQwoC9wi6QguVvpko0pjJmZVZ9CNYgt0n9bbMjJI2I4MLzCtutylocCQwuc40HgwQ25vpmZbbhCU47el/57Y82EY2ZmtUXWB+V+JamlpKaSXpQ0T9LpxQ7OzMxKJ+ttroekHdP9Seal3gn4abGCMjOz0suaIMqaovoBf4uIRUWKx8zMaomsz0E8LeldkofjLpLUBlhWvLDMzKzUMtUg0mcQ+pA8j7AS+Jp1x1UyM7N6pNBzEAdExChJx+Vsyz3k8WIFZmZmpVWoiWl/ktFWj8yzL3CCMDOrtwo9B3F9+u/ZNROOmZnVFlmfg/ilpK1y1reWdHPRojIzs5LLepvr4RGxsGwlIhYARxQlIjMzqxWyJojGkjYtW5G0GbBpFcebmVkdl/U5iEeAFyWVTTN6NjC4OCGZmVltkHU+iNskTQQOSjfdFBEjiheWmZmVWtYaBMA7wKqIeEHS5pJaRMSXxQrMzMyqNuHjhRywa9uKz6dVm6x3MZ1PMm/DfemmDsATRYnIzMyq9OqMLwA4d/A4pszJMsHnhsnaSX0JsC+wGCAipgFtixWUmZlVrn/3bcuXl6xYXbTrZE0QyyNiRdmKpCYkT1KbmVkNO2L39jxy3veKfp2sCeIlSf8X2EzSwcDfgKeKF5aZmZVa1gTx38A84C3gQpJ5pv9fsYIyM7PSK3gXk6TGwJSI2BX4Y/FDMjOz2qBgDSIiVgPvSdq+BuIxM7NaIutzEFsDUyS9RjJZEAARcVRRojIzs5LLmiCuLWoUZmZW6xSaUa4ZMADYiaSD+oGIWFUTgZmZWWkV6oMYDPQiSQ6HA3cWPSIzM6sVCjUxdY2I3QEkPQC8VvyQzMysNihUg1hZtuCmJTOzhqVQDaKHpLKRoETyJPXidDkiomVRozMzs5KpMkFEROOaCsTMzGqXrENtmJlZA+MEYWZmeTlBmJlZXk4QZmaWV1EThKTDJL0nabqkq/Ps30HSi5ImSRojqWO6vaekf0uaku47qZhxmpnZuoqWINJhwu8heQK7K3CKpK4VDrsDeCgiugMDgVvS7UuAMyOiG3AY8D+StipWrGZmtq5i1iB6A9MjYkY6XelfgaMrHNMVGJUujy7bHxFT03mviYg5wGdAmyLGamZWp3z4xRIA7nvp/aJdo5gJogPwcc76rHRbronAcenysUALSa1yD5DUG9gEKN67YGZWx/xgl9YArFi9pmjXKHUn9VXA/pLeBPYHZgOry3ZKag88DJwdEeu8C5IukDRO0rh58+bVVMxmZiXXcevN6bDVZrRt0axo1yhmgpgNbJez3jHdVi4i5kTEcRHxXeDn6baFAJJaAs8AP4+IV/NdICLuj4heEdGrTRu3QJmZVadiJojXgZ0ldZa0CXAyMCz3AEmtJZXFcA0wKN2+CfAPkg7soUWM0czMKlG0BJGO/nopMAJ4B3gsIqZIGiipbKrSviTzXU8F2gG/SLefCPwAOEvShPSnZ7FiNTOzdWWdcnSDRMRwYHiFbdflLA8F1qkhRMSfgT8XMzYzM6taqTupzcyslnKCMDOzvJwgzMwsLycIMzPLq6id1KW2cuVKZs2axbJly0oditUhzZo1o2PHjjRt2rTUoZhVafbCpYx+77Oinb9eJ4hZs2bRokULOnXqhKRSh2N1QETwxRdfMGvWLDp37lzqcMwKmv/1ClavCRo3qv7PuHrdxLRs2TJatWrl5GCZSaJVq1audVqdcMCubYt6/nqdIAAnB1tv/puxuqLndlsV9fz1PkGYmdmGcYIosk8++YSTTz6ZLl26sOeee3LEEUcwdepUZs6cyW677VZt17nuuut44YUXABg7dizdunWjZ8+ezJ49mxNOOGGjzh0RHHDAASxevLh82xNPPIEk3n333fJtM2fOZLPNNqNnz5507dqVAQMGsGbNxg1F/PLLL7PHHnvQpEkThg6tfFiu8ePHs/vuu7PTTjtx+eWXExEAzJ8/n4MPPpidd96Zgw8+mAULFgDw9NNPc91111V6PjNzgiiqiODYY4+lb9++vP/++4wfP55bbrmFTz/9tNqvNXDgQA466CAAHnnkEa655homTJhAhw4dqvxgrWjVqlXrbBs+fDg9evSgZcuW5duGDBnC97//fYYMGbLWsV26dGHChAlMmjSJt99+myeeeGLDCpTafvvtefDBBzn11FOrPO6iiy7ij3/8I9OmTWPatGk899xzANx6660ceOCBTJs2jQMPPJBbb70VgH79+vHUU0+xZMmSjYrPrD6r13cx5brxqSm8PWdx4QPXQ9dtW3L9kd0q3T969GiaNm3KgAEDyrf16NEDSL5tl5k5cyZnnHEGX3/9NQC/+93v6NOnD3PnzuWkk05i8eLFrFq1ij/84Q/06dOHc889l3HjxiGJc845h5/85CecddZZ9O/fn4ULF/LYY48xYsQInn32WX7xi1/Qv39/Jk+ezOrVq7n66qsZM2YMy5cv55JLLuHCCy9kzJgxXHvttWy99da8++67TJ06da1yPPLII1xwwQXl61999RWvvPIKo0eP5sgjj+TGG29cp+xNmjShT58+TJ8+fYPe2zKdOnUCoFGjyr/LzJ07l8WLF7P33nsDcOaZZ/LEE09w+OGH8+STTzJmzBgAfvzjH9O3b19uu+02JNG3b1+efvppTjzxxI2K0ay+ajAJohQmT57MnnvuWfC4tm3bMnLkSJo1a8a0adM45ZRTGDduHH/5y1849NBD+fnPf87q1atZsmQJEyZMYPbs2UyePBmAhQsXrnWu8847j1deeYX+/ftzwgknrJWIHnjgAbbccktef/11li9fzr777sshhxwCwBtvvMHkyZPz3tr5z3/+k/vuu698/cknn+Swww5jl112oVWrVowfP36dci5ZsoQXX3yRgQMHrnO+/fbbjy+//HKd7XfccUd5LWh9zJ49m44dO5avd+zYkdmzk6lHPv30U9q3bw/ANttss1btrVevXowdO9YJwqwSDSZBVPVNv9RWrlzJpZdeyoQJE2jcuHH5N/i99tqLc845h5UrV3LMMcfQs2dPdtxxR2bMmMFll11Gv379yj/gs3j++eeZNGlSeZPTokWLmDZtGptssgm9e/eu9L7/+fPn06JFi/L1IUOGcMUVVwBw8sknM2TIkPIE8f7779OzZ08kcfTRR3P44Yevc76xY8dmjrk6SVrrDqW2bdsyZ86cksRiVhc0mARRCt26dcvU/v/rX/+adu3aMXHiRNasWUOzZskUgj/4wQ94+eWXeeaZZzjrrLO48sorOfPMM5k4cSIjRozg3nvv5bHHHmPQoEGZ4okI7r77bg499NC1to8ZM4Ytttii0tc1adKENWvW0KhRI+bPn8+oUaN46623kMTq1auRxO233w580wdRlequQXTo0IFZs2aVr8+aNYsOHZLpz9u1a8fcuXNp3749c+fOpW3bb+4bX7ZsGZttttl6X8+soXAndREdcMABLF++nPvvv79826RJk9b5Br1o0SLat29Po0aNePjhh1m9OpmW+8MPP6Rdu3acf/75nHfeebzxxht8/vnnrFmzhuOPP56bb76ZN954I3M8hx56KH/4wx9YuXIlAFOnTi3v96jKt7/9bWbMmAHA0KFDOeOMM/jwww+ZOXMmH3/8MZ07d16vWsHYsWOZMGHCOj8bkhwA2rdvT8uWLXn11VeJCB566CGOPvpoAI466igGDx4MwODBg8u3Q1L+6ryTzKxUxn+4oCjndYIoIkn84x//4IUXXqBLly5069aNa665hm222Wat4y6++GIGDx5Mjx49ePfdd8u/zY8ZM4YePXrw3e9+l0cffZQrrriC2bNn07dvX3r27Mnpp5/OLbfckjme8847j65du7LHHnuw2267ceGFF+a9a6mifv36lXf0DhkyhGOPPXat/ccff/w6dzNVl9dff52OHTvyt7/9jQsvvJBu3b5pKuzZs2f58u9//3vOO+88dtppJ7p06VLetHX11VczcuRIdt55Z1544QWuvvrq8teMHj2afv36FSVus5ow5LWPALj4kfFFOb/K7hev63r16hXjxo1ba9s777zDd77znRJFVH/MnTuXM888k5EjR5Y6lGrz6aefcuqpp/Liiy/m3e+/HasLnpv8CQP+PJ6/X9SHPXfYeoPOIWl8RPTKt899EFZQ+/btOf/881m8ePFaz0LUZR999BF33nlnqcMw2yiH7bYNM28tXi3YCcIyqW+3gu61116lDsGs1qv3fRD1pQnNao7/ZswS9TpBNGvWjC+++ML/4S2zsvkgym41NmvI6nUTU8eOHZk1axbz5s0rdShWh5TNKGfW0NXrBNG0aVPPCmZmtoHqdROTmZltOCcIMzPLywnCzMzyqjdPUkuaB3y4EadoDXxeTeHUFQ2tzA2tvOAyNxQbU+YdIqJNvh31JkFsLEnjKnvcvL5qaGVuaOUFl7mhKFaZ3cRkZmZ5OUGYmVleThDfuL/wIfVOQytzQysvuMwNRVHK7D4IMzPLyzUIMzPLywnCzMzyalAJQtJhkt6TNF3S1Xn2byrp0XT/fyR1KkGY1SpDma+U9LakSZJelLRDKeKsToXKnHPc8ZJCUp2/JTJLmSWdmP6up0j6S03HWN0y/G1vL2m0pDfTv+8jShFndZE0SNJnkiZXsl+Sfpu+H5Mk7bHRF42IBvEDNAbeB3YENgEmAl0rHHMxcG+6fDLwaKnjroEy/xDYPF2+qCGUOT2uBfAy8CrQq9Rx18DveWfgTWDrdL1tqeOugTLfD1yULncFZpY67o0s8w+APYDJlew/AngWELA38J+NvWZDqkH0BqZHxIyIWAH8FTi6wjFHA4PT5aHAgZJUgzFWt4JljojREbEkXX0VqOvjXGf5PQPcBNwGLKvJ4IokS5nPB+6JiAUAEfFZDcdY3bKUOYCyOXK3BObUYHzVLiJeBuZXccjRwEOReBXYSlL7jblmQ0oQHYCPc9ZnpdvyHhMRq4BFQKsaia44spQ517kk30DqsoJlTqve20XEMzUZWBFl+T3vAuwi6Z+SXpV0WI1FVxxZynwDcLqkWcBw4LKaCa1k1vf/e0H1ej4Iy07S6UAvYP9Sx1JMkhoBdwFnlTiUmtaEpJmpL0kt8WVJu0fEwlIGVWSnAA9GxJ2S9gEelrRbRKwpdWB1RUOqQcwGtstZ75huy3uMpCYk1dIvaiS64shSZiQdBPwcOCoiltdQbMVSqMwtgN2AMZJmkrTVDqvjHdVZfs+zgGERsTIiPgCmkiSMuipLmc8FHgOIiH8DzUgGtauvMv1/Xx8NKUG8DuwsqbOkTUg6oYdVOGYY8ON0+QRgVKS9P3VUwTJL+i5wH0lyqOvt0lCgzBGxKCJaR0SniOhE0u9yVESMK0241SLL3/YTJLUHJLUmaXKaUYMxVrcsZf4IOBBA0ndIEkR9nn94GHBmejfT3sCiiJi7MSdsME1MEbFK0qXACJI7IAZFxBRJA4FxETEMeICkGjqdpDPo5NJFvPEylvl2oDnwt7Q//qOIOKpkQW+kjGWuVzKWeQRwiKS3gdXATyOiztaOM5b5v4A/SvoJSYf1WXX5C5+kISRJvnXar3I90BQgIu4l6Wc5ApgOLAHO3uhr1uH3y8zMiqghNTGZmdl6cIIwM7O8nCDMzCwvJwgzM8vLCcLMzPJygrCSkbRa0gRJkyU9JWmraj7/zPSefyR9Vckxm0l6SVJjSZ0kLU1jelvSvemT1+tzzV6Sfpsu95XUJ2ffAElnbkyZ0vPcIOmqAsc8KOmE9Thnp8pGCa1w3C8kfVzx/ZR0qaRzsl7P6gYnCCulpRHRMyJ2I3nu5JISxHAO8HhErE7X34+InkB3khFAj1mfk0XEuIi4PF3tC/TJ2XdvRDy0sQGX2FMkA+VVNIj6P9ZRg+MEYbXFv0kHFpPURdJzksZLGitp13R7O0n/kDQx/emTbn8iPXaKpAvW87qnAU9W3JgO1vgvYKf02/UofTNnxvbpdX+U1n4mSno53dZX0tNK5hIZAPwkrZHsV/bNX9Kukl4ru1Z6/rfS5T3TGs14SSNUYDROSedLej2N4e+SNs/ZfZCkcZKmSuqfHt9Y0u3payZJunB93qyIeDXf07npiMAzJeVLHlZHOUFYyUlqTDIkQtlTzvcDl0XEnsBVwO/T7b8FXoqIHiTj4k9Jt5+THtsLuFxSphF40yEadoyImXn2bZ7G9BZwNzA4IroDj6RxAFwHHJrGs9bT5+k57wV+ndaSxubsexfYRFLndNNJwKOSmqbXOiEtzyDgFwWK8XhE7JXG8A7J+ENlOpF82+8H3CupWbp/UUTsBewFnJ8TR1nZt5U0vMB18xkH7LcBr7NaqsEMtWG10maSJpDUHN4BRkpqTtIsUzb0B8Cm6b8HAGcCpE1Ci9Ltl0s6Nl3ejmQQuizDSLQGFlbY1iWNKYAnI+JZSQ8Dx6X7HwZ+lS7/E3hQ0mPA4xmul+sxksRwa/rvScC3SQYSHJmWvTFQaCyd3STdDGxFMmTKiNxrpCOXTpM0A9gVOATontM/sSXJ+zW17EURMYdkyIb19Vl6DasnnCCslJZGRM/02/oIkj6IB4GFaT9AQZL6AgcB+0TEEkljSAZly3T9PMe+n/XaETFA0vdIvqGPl7RnxusCPEqSBB9PThXTJO0OTImIfdbjPA8Cx0TERElnkQ7IVxZixZBJZhu7LCJyEwmqnul1m5G8p1ZPuInJSi5tv76cZHC1JcAHkn4E5fPs9kgPfZFkWtSytvQtSb4BL0iTw64kw3dnve4CoHHa9FKVf/HNwI2nAWPTGLpExH8i4jqSUUK3q/C6L0mGF8937fdJBs27liRZALwHtFEydwGSmkrqViC2FsDctHnqtAr7fiSpkaQuJFNzvkeSiC9Kj0fSLpK2KHCNrHYBCt4JZXWHE4TVChHxJjCJZJKX04BzJU0k6Wcom0ryCuCHaYfueJK7jJ4Dmkh6h6S55tX1vPTzwPcLHHMZcLakScAZaRwAt0t6K7099F8k8yLnego4tqyTOs95HwVO55s5C1aQDDN/W1r2CeTcBVWJa4H/kDR3vVth30fAaySzBA6IiGXA/wJvA2+kcd9HhZaEqvogJP1KyUiim0uaJemGnN37AiMLxGt1iEdztQZNyfSjP4mIM0odS12mZF6RK/0+1i+uQViDFhFvAKPTO6lsw7Umqc1YPeIahJmZ5eUahJmZ5eUEYWZmeTlBmJlZXk4QZmaWlxOEmZnl9f8BeMz9mdn27OUAAAAASUVORK5CYII=",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"evaluate_emeddings_approach(labels=['An Amazon review with a negative sentiment.', 'An Amazon review with a positive sentiment.'], engine='text-similarity-babbage-001')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As shown above, zero-shot classification with embeddings can lead to great results, especially when the labels are more descriptive than just simple words."
]
}
],
"metadata": {
"interpreter": {
"hash": "be4b5d5b73a21c599de40d6deb1129796d12dc1cc33a738f7bac13269cfcafe8"
},
"kernelspec": {
"display_name": "Python 3.7.3 64-bit ('base': conda)",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}