openai-cookbook/examples/Clustering_for_transaction_classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Clustering for Transaction Classification\n",
    "\n",
    "This notebook covers use cases where your data is unlabelled but has features that can be used to cluster them into meaningful categories. The challenge with clustering is making the features that make those clusters stand out human-readable, and that is where we'll look to use GPT-3 to generate meaningful cluster descriptions for us. We can then use these to apply labels to a previously unlabelled dataset.\n",
    "\n",
    "To feed the model we use embeddings created using the approach displayed in the notebook [Multiclass classification for transactions Notebook](Multiclass_classification_for_transactions.ipynb), applied to the full 359 transactions in the dataset to give us a bigger pool for learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports\n",
    "import openai\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.cluster import KMeans\n",
    "from sklearn.manifold import TSNE\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import os\n",
    "\n",
    "openai.api_key = os.getenv(\"OPENAI_API_KEY\")\n",
    "COMPLETIONS_MODEL = \"text-davinci-003\"\n",
    "\n",
    "# This path leads to a file with data and precomputed embeddings\n",
    "embedding_path = \"data/library_transactions_with_embeddings_359.csv\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clustering\n",
    "\n",
    "We'll reuse the approach from the [Clustering Notebook](Clustering.ipynb), using K-Means to cluster our dataset using the feature embeddings we created previously. We'll then use the Completions endpoint to generate cluster descriptions for us and judge their effectiveness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(embedding_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(359, 1536)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "embedding_df = pd.read_csv(embedding_path)\n",
    "embedding_df[\"embedding\"] = embedding_df.embedding.apply(eval).apply(np.array)\n",
    "matrix = np.vstack(embedding_df.embedding.values)\n",
    "matrix.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/ted/.virtualenvs/openai/lib/python3.9/site-packages/sklearn/cluster/_kmeans.py:870: FutureWarning: The default value of `n_init` will change from 10 to 'auto' in 1.4. Set the value of `n_init` explicitly to suppress the warning\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "n_clusters = 5\n",
    "\n",
    "kmeans = KMeans(n_clusters=n_clusters, init=\"k-means++\", random_state=42)\n",
    "kmeans.fit(matrix)\n",
    "labels = kmeans.labels_\n",
    "embedding_df[\"Cluster\"] = labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Clusters identified visualized in language 2d using t-SNE')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEICAYAAABcVE8dAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABazElEQVR4nO29eXxcV3n//z6zaWY00kgaLZY03uIttrM4sSeOSUKAAFkwNQGXL22BJA1JaQuFtIFAaUnoThuaGtp+afozhLZQFoNr8BdnbVZInHEWx3Ecx1tsj2RrH0mj2aSZ8/vjuSONZMmSrW0knbdees3MvXfufe6dmc957nOe8xyltcZgMBgMsx/bdBtgMBgMhqnBCL7BYDDMEYzgGwwGwxzBCL7BYDDMEYzgGwwGwxzBCL7BYDDMEWaE4Cul7lNK/dd023EuKKV2KaVuGWHdIqWUVko5JunYMaXUBdZzj1LqF0qpTqXUT5RSv6OUevQ893urUuq58do0GQy9pme7/uM4xojfQ6XUNUqpg+e53/O+rnOBybg+SqkF1nfSPpH7LXQKRvCVUr+tlNpjfQinrB/s1RO4/0kV2aForW/UWn9vso+jlHpKKfWpIcf2aa2PWi83AzVAQGv9m1rr72ut3z/Zdg1liE1Tcbwpuf55x3tWa71iqo43k1FKXamUekwp1a6UarEckdqptEFrfcL6TmYmet9KqYeUUn81yjZlSqnvKKVOK6W6lVJvKaW+lLdeK6X2KaVsecv+Sin1kPU8p2exIf//52zHLQjBV0r9MfBPwN8g4rQA+Fdg0zSaNYipaigmgYXAW1rrvuk2xGCwKAceBBYh389u4LvTadA08ADgA1YCfuA3gMNDtqkDPjbKfsqshiv3/6Ozbq21ntZ/62RjwG+eZZv7gP+ynr8LiAxZ/zbwXuv5FcAeoAtoAv7RWn4C0NaxYsAGa/nvAgeADuARYGHefjXwh8Ah4BigrA+q2dr/PuCiEWx+CviU9dwO3A+0AketfWrAkXcNtgKngAbgrwC7te5W4Dnr/R2WHTda6/4ayABJ65z+Oc/upcDXgDTQa62/Pbe/PDsvBB4D2oGDwEfz1gWAn1vn+iLwl/nvHXK+u4DPDFm2F/hwvk3W85uAN5AfegNwd/65DtlH/vs+ALxi2XMSuC9vu0VDrmn+9d+b97nHrO3eZa27Evg1ELW2e1fePhcDT1t2Pgb8M9b3cJjzfxd530vkO3k38BrQCfwIcI/w3qGfyRbr/LqAl4BrhvwWfgz8h2XXfmBd3vrLrWvUDfzEOu5fjff6Wus/CRwH2oA/Z/DvzgZ8CThirf8xUDFGDbgc6D7P792g634OejDc9+UvgV9Z1+5RoHIs5z7k2Hciv7c08l37xQh2vw586CzXRAP3INqTs/GvgIeGs3+s/4Ug+DcAfWcznHMT/OeBT1jPfcCVI10g5A7iMNLKOoA/A3495KI/BlQAHuB65AdYhoj/SqB2BJufYkBwPg28Ccy39vXkkC/bduDfgGKg2vqS/17ej7QXuANpOH4faATU0OOM8CPuv3ZDf/TW8U4Ct1nnfxnSKK2y1v8Q+eEWAxch4jzSD++TwK/yXq9CRLRoGJtOYYkY4u1dPtS2Ec7lXcDFiLhcgvyAP3SWH/CnhrHzTuuzKAXqkR/wTdY+32e9rsr7Lv0jUAS8ExGCcxH8FxEvrQJxKj49wnsHnTfwcUT0HMCfAKexGgvr80xaNtuBvwVesNa5EFH6HOAEPowIz1gF/2zXdxUiYFdbx7kf+V7mfnefA14Agtb1+jfgv8eoAZ/PncN5fO8GXffz1QPr+3IEWI781p8C/m4s5z6MTQ/lrvlZzvn/Qxrr24Blw6zXwDJEb3I6Mm7BL4SQTgBo1RMXcugFliqlKrXWMa31C2fZ9tPA32qtD1jH/xtgjVJqYd42f6u1btdaJ6x9lyBesbLed2oMNn0U+Cet9UmtdTvyIwVAKVWD/Hg/r7Xu0Vo3I3cR+bdyx7XW/64l3vg9oBYJfY2XjcDbWuvvaq37tNavAD8FftPqzPoI8FXLrtetY4/EdgZfu98Bfqa1Tg2zbS+wSilVqrXu0Fq/PBZjtdZPaa33aa2zWuvXgP8Grh3bqYLVJ/RXwG9orbsQYf2l1vqX1j4fQ7zBm5RSC4AQ8Oda65TW+hngF2M9lsU3tdaN1mf+C2DNWN6ktf4vrXWb9Zl8AxHQ/P6B5yybM8B/Apday69EGolvaq17tdY/QxqdMTHK9d2MeKvPaa3TwFcRwcnxaeArWuuI9ZnfB2weLRSqlLrE2tcXrNfn+r0bjXPRg+9qrd+yfus/ZuDzGu3cz4fPAt8HPgO8oZQ6rJS6ccg2Grmb+HOllGuE/bQqpaJ5/yvPdtBCEPw2oHICY+S3I630m0qpsFJq41m2XQhsyV0sJKyhEM8vx8ncE631/yK39f8CNCulHlRKlY7Bprr8/SBeWL4NTuBUnh3/hnj6OU7n2RC3nvrGcNzRWAisz//CIEI9D6hCxGMkuwehte4G/h8DDdVvIV/o4fgI0sgdV0o9rZTaMBZjlVLrlVJPWh19nYjIVI7xvfORH/EtWuu3rMULkcYt//yvRhrUOqBDa92Tt5sRz38ETuc9jzPGz0wpdbdS6oCVWRVFQn755zl0v27r91MHNGjLBbTI//xGO+7Zru+g77D1PWzLe/tCYHvedTyAhBtHdEyUUkuRUODntNbPWovP6Xs3Bs5FD0b6vEY797NiZcblOlV3WftIaK3/Rmu9FnF6fwz8RClVkf9erfUvgQjweyPsvlJrXZb3f+BsthSC4D8PpIAPjXH7HsCbe2F5BFW511rrQ1rr30IE8+vANqVUMcO3yCeR0En+BfNorX+dt82g92mtv2l9SKuQL9IXxmDzKSSck2PBEBtSDP7gSrXWq8ew3zPsO0dOAk8POX+f1vr3gRYk1DaS3cPx38BvWQLuRkJXZxqsdVhrvQn5jP4H+bLDmZ/tvCFv/QES252vtfYD30Ya6LOilPJYx/knrfWuvFUngf8ccv7FWuu/Qz6zcuu7k2O08x83SqlrgC8id4XlWusypA9g1PNEbK5XSuVvm//5jef6nkLCNbn3ehChynES6VvKv5ZurXXDCOe5EHgc+Eut9X/mrTrX79356sG5MNq5D2WoZnxfD3SqDvXise42/wYJYS0eZn9fAf6UvPM8X6Zd8LXWncgt0r8opT6klPIqpZxKqRuVUn8/zFveQjyaDyilnEjcvSi3Uin1caVUldY6i8SQAbLIFykL5OeCfxv4slJqtfVev1LqN0eyVSkVsrwgJ/JFS1r7HI0fA3+klAoqpcqRzq3c+Z9COoi+oZQqVUrZlFJLlFJjDVU0DTmnc2EnsFwp9Qnrmjutc1xphQt+BtxnfSargFtG2d8vEU/vL4AfWZ/BIJRSLsvj8Wute5HOtNx2e4HVSqk1Sik3EhbIpwRo11onlVJXAL89xvP8DvCm1nro9+m/gA8qpa5XStmVUm6l1LuUUkGt9XEkvPM1y+argQ+O8XjjoQQRvBbAoZT6KtLfMBaeR7zqzyilHEqpTUinZY7xXN9tyLV6hxVeuI/BjdC3gb/OhfSUUlXW8c9AKVUP/C+SZPDt/HXn8b07Xz04F0Y796GM+ptUSv259VtzWZ/F5yz7zhjLobV+CunkHe33NyrTLvgAVpzyj5EPqwXxFj6DeGVDt+0E/gDp9GhAhDeSt8kNwH6lVAzJdviYdfsUR7JafmXddl6ptd6OtPo/VEp1IRf1jBY4j1Lg35FsmVyP/T+M4RT/HckA2gu8jHyh8/kk0hn0hrXvbUhYYSxsQWKlHUqpb47xPUB/GOb9SBimEbml/ToDP5jPILe1p5GOqO+Osr8Ucm7vRbzFkfgE8LZ1zT+NhJGwQi1/gXh+h5DspHz+APgLpVQ34iT8mLHxMeBmNThf+Rqt9Umk4/5PGfjefYGB38VvA+uRUN+9SGbMZPMI8DAiZMcRp2JMYRkrvvxhJIwRRfoodiJ3kOO6vlrr/Ujc+YeIxxtDstVyfTRbkLuDR633v4Bcu+H4FCKI9+V/Jnnrx/y9O18
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "tsne = TSNE(\n",
    "    n_components=2, perplexity=15, random_state=42, init=\"random\", learning_rate=200\n",
    ")\n",
    "vis_dims2 = tsne.fit_transform(matrix)\n",
    "\n",
    "x = [x for x, y in vis_dims2]\n",
    "y = [y for x, y in vis_dims2]\n",
    "\n",
    "for category, color in enumerate([\"purple\", \"green\", \"red\", \"blue\",\"yellow\"]):\n",
    "    xs = np.array(x)[embedding_df.Cluster == category]\n",
    "    ys = np.array(y)[embedding_df.Cluster == category]\n",
    "    plt.scatter(xs, ys, color=color, alpha=0.3)\n",
    "\n",
    "    avg_x = xs.mean()\n",
    "    avg_y = ys.mean()\n",
    "\n",
    "    plt.scatter(avg_x, avg_y, marker=\"x\", color=color, s=100)\n",
    "plt.title(\"Clusters identified visualized in language 2d using t-SNE\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cluster 0 Theme:  All of the transactions have to do with legal deposit services from the Agency for Legal Deposit Libraries (ALDL).\n",
      "ALDL, ALDL Charges\n",
      "   Agency for Legal Deposit Libaries, Agency services\n",
      "   ALDL, Legal Deposit Services\n",
      "   ALDL, ALDL Charges\n",
      "   ALDL, Legal deposit services\n",
      "   British Library, Legal Deposit Services\n",
      "   ALDL, Legal deposit materials April to July 22\n",
      "   ALDL, Jan20-Mar 20 charge from Agency for Legal Deposit Libraries\n",
      "   ALDL, ALDL Charges\n",
      "   ALDL, Legal Deposit Services\n",
      "   ----------------------------------------------------------------------------------------------------\n",
      "Cluster 1 Theme:  These transactions all involve the purchase of goods or services from various vendors.\n",
      "GARDINER AND THEOBALD LLP, GB Feasibility Study\n",
      "   BSI, Annual subscription\n",
      "   EX LIBRIS, IT equipment\n",
      "   Private Sale, Literary & Archival Items\n",
      "   XMA SCOTLAND LTD, 2 invoices one for the replacement of obsolete hardware and the other for a new laptop\n",
      "   JISC SERVICES LTD SUBSCRIPTION ACCOUNT, Annual Subscription\n",
      "   EDF Energy, Utilities\n",
      "   JISC Services Ltd, Managed router service charge annual subscription 01/04/22 to 31/03/23\n",
      "   RICK GEKOSKI, Papers 1970's to 2019 Alisdair Gray\n",
      "   Trustmarque, Miscrosoft Software Licenses\n",
      "   ----------------------------------------------------------------------------------------------------\n",
      "Cluster 2 Theme:  All of these transactions are related to ECG Facilities Service, and involve payments for facility management fees, inspections and maintenance, maintenance contracts, electrical and mechanical works, boiler replacements, USP batteries, gutter works, and cleaning of pigeon fouling.\n",
      "ECG FACILITIES SERVICE, This payment covers multiple invoices for facility management fees\n",
      "   ECG FACILITIES SERVICE, Facilities Management Charge\n",
      "   ECG FACILITIES SERVICE, Inspection and Maintenance of all Library properties\n",
      "   ECG Facilities Service, Facilities Management Charge\n",
      "   ECG FACILITIES SERVICE, Maintenance contract - October\n",
      "   ECG FACILITIES SERVICE, Electrical and mechanical works\n",
      "   ECG FACILITIES SERVICE, This payment covers multiple invoices for facility management fees\n",
      "   ECG FACILITIES SERVICE, CB Bolier Replacement (1),USP Batteries,Gutter Works & Cleaning of pigeon fouling\n",
      "   ECG Facilities Service, Facilities Management Charge\n",
      "   ECG Facilities Service, Facilities Management Charge\n",
      "   ----------------------------------------------------------------------------------------------------\n",
      "Cluster 3 Theme:  All of these transactions are related to Kelvin Hall and involve spending money on services or goods.\n",
      "CBRE, Kelvin Hall\n",
      "   GLASGOW CITY COUNCIL, Kelvin Hall\n",
      "   University Of Glasgow, Kelvin Hall\n",
      "   GLASGOW LIFE, Oct 20 to Dec 20 service charge - Kelvin Hall\n",
      "   Computacenter Uk, Kelvin Hall\n",
      "   XMA Scotland Ltd, Kelvin Hall\n",
      "   GLASGOW LIFE, Service Charges Kelvin Hall 01/07/19-30/09/19\n",
      "   Glasgow Life, Kelvin Hall Service Charges\n",
      "   Glasgow City Council, Kelvin Hall\n",
      "   GLASGOW LIFE, Quarterly service charge KH\n",
      "   ----------------------------------------------------------------------------------------------------\n",
      "Cluster 4 Theme:  These transactions all involve construction and refurbishment work.\n",
      "M & J Ballantyne Ltd, George IV Bridge Work\n",
      "   John Graham Construction Ltd, Causewayside Refurbishment\n",
      "   John Graham Construction Ltd, Causewayside Refurbishment\n",
      "   John Graham Construction Ltd, Causewayside Refurbishment\n",
      "   John Graham Construction Ltd, Causewayside Refurbishment\n",
      "   ARTHUR MCKAY BUILDING SERVICES, Causewayside Work\n",
      "   John Graham Construction Ltd, Causewayside Refurbishment\n",
      "   Morris & Spottiswood Ltd, George IV Bridge Work\n",
      "   ECG FACILITIES SERVICE, Causewayside IT Work\n",
      "   John Graham Construction Ltd, Causewayside Refurbishment\n",
      "   ----------------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "# We'll read 10 transactions per cluster as we're expecting some variation\n",
    "transactions_per_cluster = 10\n",
    "\n",
    "for i in range(n_clusters):\n",
    "    print(f\"Cluster {i} Theme:\", end=\" \")\n",
    "\n",
    "    transactions = \"\\n\".join(\n",
    "        embedding_df[embedding_df.Cluster == i]\n",
    "        .combined.str.replace(\"Supplier: \", \"\")\n",
    "        .str.replace(\"Description: \", \":  \")\n",
    "        .str.replace(\"Value: \", \":  \")\n",
    "        .sample(transactions_per_cluster, random_state=42)\n",
    "        .values\n",
    "    )\n",
    "    response = openai.Completion.create(\n",
    "        model=COMPLETIONS_MODEL,\n",
    "        # We'll include a prompt to instruct the model what sort of description we're looking for\n",
    "        prompt=f'''We want to group these transactions into meaningful clusters so we can target the areas we are spending the most money. \n",
    "                What do the following transactions have in common?\\n\\nTransactions:\\n\"\"\"\\n{transactions}\\n\"\"\"\\n\\nTheme:''',\n",
    "        temperature=0,\n",
    "        max_tokens=100,\n",
    "        top_p=1,\n",
    "        frequency_penalty=0,\n",
    "        presence_penalty=0,\n",
    "    )\n",
    "    print(response[\"choices\"][0][\"text\"].replace(\"\\n\", \"\"))\n",
    "\n",
    "    sample_cluster_rows = embedding_df[embedding_df.Cluster == i].sample(transactions_per_cluster, random_state=42)\n",
    "    for j in range(transactions_per_cluster):\n",
    "        print(sample_cluster_rows.Supplier.values[j], end=\", \")\n",
    "        print(sample_cluster_rows.Description.values[j], end=\"\\n   \")\n",
    "        #print(str(sample_cluster_rows.Value).values[j])\n",
    "\n",
    "    print(\"-\" * 100)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conclusion\n",
    "\n",
    "We now have five new clusters that we can use to describe our data. Looking at the visualisation some of our clusters have some overlap and we'll need some tuning to get to the right place, but already we can see that GPT-3 has made some effective inferences. In particular, it picked up that items including legal deposits were related to literature archival, which is true but the model was given no clues on. Very cool, and with some tuning we can create a base set of clusters that we can then use with a multiclass classifier to generalise to other transactional datasets we might use."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "openai",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.9"
  },
  "vscode": {
   "interpreter": {
    "hash": "365536dcbde60510dc9073d6b991cd35db2d9bac356a11f5b64279a5e6708b97"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}