Archives
/
openai-cookbook
mirror of https://github.com/openai/openai-cookbook
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

adds data download from CDN with precomputed embeddings

Browse Source
pull/5/head
Ted Sanders 2 years ago
parent 350b9a7333
commit bd061ad2d4
1 changed files with 37 additions and 24 deletions
Show Stats Download Patch File Download Diff File

61
examples/Semantic_text_search_using_embeddings.ipynb
Unescape Escape View File

@ -18,9 +18,9 @@
"import pandas as pd\n",
"import numpy as np\n",
"\n",
"\n",
"df = pd.read_csv('output/embedded_1k_reviews.csv')\n",
"df['babbage_search'] = df.babbage_search.apply(eval).apply(np.array)"
"datafile_path = \"https://cdn.openai.com/API/examples/data/fine_food_reviews_with_embeddings_1k.csv\" # for your convenience, we precomputed the embeddings\n",
"df = pd.read_csv(datafile_path)\n",
"df[\"babbage_search\"] = df.babbage_search.apply(eval).apply(np.array)\n"
]
},
{
@ -39,11 +39,11 @@
"name": "stdout",
"output_type": "stream",
"text": [
"Jamaican Blue beans: Excellent coffee bean for roasting. Our family just purchased another 5 pounds for more roasting. Plenty of flavor and mild on acidity when roasted to a dark brown bean and befor\n",
"Fantastic Instant Refried beans: Fantastic Instant Refried Beans have been a staple for my family now for nearly 20 years. All 7 of us love it and my grown kids are passing on the tradition.\n",
"\n",
"Good Buy: I liked the beans. They were vacuum sealed, plump and moist. Would recommend them for any use. I personally split and stuck them in some vodka to make vanilla extract. Yum!\n",
"Jamaican Blue beans: Excellent coffee bean for roasting. Our family just purchased another 5 pounds for more roasting. Plenty of flavor and mild on acidity when roasted to a dark brown bean and befor\n",
"\n",
"Fantastic Instant Refried beans: Fantastic Instant Refried Beans have been a staple for my family now for nearly 20 years. All 7 of us love it and my grown kids are passing on the tradition.\n",
"Delicious!: I enjoy this white beans seasoning, it gives a rich flavor to the beans I just love it, my mother in law didn't know about this Zatarain's brand and now she is traying different seasoning\n",
"\n"
]
}
@ -53,16 +53,26 @@
"\n",
"# search through the reviews for a specific product\n",
"def search_reviews(df, product_description, n=3, pprint=True):\n",
" embedding = get_embedding(product_description, engine='text-search-babbage-query-001')\n",
" df['similarities'] = df.babbage_search.apply(lambda x: cosine_similarity(x, embedding))\n",
" embedding = get_embedding(\n",
" product_description,\n",
" engine=\"text-search-babbage-query-001\"\n",
" )\n",
" df[\"similarities\"] = df.babbage_search.apply(lambda x: cosine_similarity(x, embedding))\n",
"\n",
" res = df.sort_values('similarities', ascending=False).head(n).combined.str.replace('Title: ','').str.replace('; Content:', ': ')\n",
" res = (\n",
" df.sort_values(\"similarities\", ascending=False)\n",
" .head(n)\n",
" .combined.str.replace(\"Title: \", \"\")\n",
" .str.replace(\"; Content:\", \": \")\n",
" )\n",
" if pprint:\n",
" for r in res:\n",
" print(r[:200])\n",
" print()\n",
" return res\n",
"res = search_reviews(df, 'delicious beans', n=3)\n"
"\n",
"\n",
"res = search_reviews(df, \"delicious beans\", n=3)\n"
]
},
{
@ -74,17 +84,17 @@
"name": "stdout",
"output_type": "stream",
"text": [
"Rustichella ROCKS!: Anything this company makes is worthwhile eating! My favorite is their Trenne.<br />Their whole wheat pasta is the best I have ever had.\n",
"\n",
"sooo good: tastes so good. Worth the money. My boyfriend hates wheat pasta and LOVES this. cooks fast tastes great.I love this brand and started buying more of their pastas. Bulk is best.\n",
"\n",
"Wonderful: Came quickly. Was plentiful and delicious and cheaper than in the store. You will enjoy it if you like thick pasta.\n",
"Tasty and Quick Pasta: Barilla Whole Grain Fusilli with Vegetable Marinara is tasty and has an excellent chunky vegetable marinara. I just wish there was more of it. If you aren't starving or on a \n",
"\n",
"Rustichella ROCKS!: Anything this company makes is worthwhile eating! My favorite is their Trenne.<br />Their whole wheat pasta is the best I have ever had.\n",
"\n"
]
}
],
"source": [
"res = search_reviews(df, 'whole wheat pasta', n=3)"
"res = search_reviews(df, \"whole wheat pasta\", n=3)\n"
]
},
{
@ -109,7 +119,7 @@
}
],
"source": [
"res = search_reviews(df, 'bad delivery', n=1)"
"res = search_reviews(df, \"bad delivery\", n=1)\n"
]
},
{
@ -134,7 +144,7 @@
}
],
"source": [
"res = search_reviews(df, 'spoilt', n=1)"
"res = search_reviews(df, \"spoilt\", n=1)\n"
]
},
{
@ -148,22 +158,20 @@
"text": [
"Good food: The only dry food my queen cat will eat. Helps prevent hair balls. Good packaging. Arrives promptly. Recommended by a friend who sells pet food.\n",
"\n",
"A great deal on Greenies: Paid only $22 with free shipping for 96 teenies compared to about $35 at the pet store. How can you go wrong with a deal like that? The dog begs for his daily Greenie. Got \n",
"Good product: I like that this is a better product for my pets but really for the price of it I couldn't afford to buy this all the time. My cat isn't very picky usually and she ate this, we usually \n",
"\n"
]
}
],
"source": [
"res = search_reviews(df, 'pet food', n=2)"
"res = search_reviews(df, \"pet food\", n=2)\n"
]
}
],
"metadata": {
"interpreter": {
"hash": "be4b5d5b73a21c599de40d6deb1129796d12dc1cc33a738f7bac13269cfcafe8"
},
"kernelspec": {
"display_name": "Python 3.7.3 64-bit ('base': conda)",
"display_name": "Python 3.9.9 ('openai')",
"language": "python",
"name": "python3"
},
"language_info": {
@ -176,9 +184,14 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
"version": "3.9.9"
},
"orig_nbformat": 4
"orig_nbformat": 4,
"vscode": {
"interpreter": {
"hash": "365536dcbde60510dc9073d6b991cd35db2d9bac356a11f5b64279a5e6708b97"
}
}
},
"nbformat": 4,
"nbformat_minor": 2

Write Preview
Loading…
Cancel
Save