Harrison/add to search (#778)

Co-authored-by: Enrico Shippole <enricoship@gmail.com>
ankush/async-llmchain
Harrison Chase 1 year ago committed by GitHub
parent 309d86e339
commit c658f0aed3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -13,24 +13,25 @@
"source": [
"This notebook goes over how to use the bing search component.\n",
"\n",
"First, you need to set up the proper API keys and environment variables. To set it up, follow the instructions found here.\n",
"First, you need to set up the proper API keys and environment variables. To set it up, follow the instructions found [here](https://levelup.gitconnected.com/api-tutorial-how-to-use-bing-web-search-api-in-python-4165d5592a7e).\n",
"\n",
"Then we will need to set some environment variables."
]
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"os.environ[\"BING_SUBSCRIPTION_KEY\"] = \"\""
"os.environ[\"BING_SUBSCRIPTION_KEY\"] = \"\"\n",
"os.environ[\"BING_SEARCH_URL\"] = \"\""
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
@ -39,7 +40,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
@ -48,16 +49,16 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Thanks to the flexibility of <b>Python</b> and the powerful ecosystem of packages, the Azure CLI supports features such as autocompletion (in shells that support it), persistent credentials, JMESPath result parsing, lazy initialization, network-less unit tests, and more. Building an open-source and cross-platform Azure CLI with <b>Python</b> by Dan Taylor. <b>Python</b> Brochure. <b>Python</b> is a programming language that lets you work more quickly and integrate your systems more effectively. You can learn to use <b>Python</b> and see almost immediate gains in productivity and lower maintenance costs. Learn more about <b>Python</b> . Learning. Before getting started, you may want to find out which IDEs and text editors are tailored to make <b>Python</b> editing easy, browse the list of introductory books, or look at code samples that you might find helpful.. There is a list of tutorials suitable for experienced programmers on the BeginnersGuide/Tutorials page. There is also a list of resources in other languages which might be ... <b>Python</b> is a popular programming language. <b>Python</b> can be used on a server to create web applications. Start learning <b>Python</b> now ». With <b>Python</b>, you can use while loops to run the same task multiple times and for loops to loop once over list data. In this module, you&#39;ll learn about the two loop types and when to apply each. Manage data with <b>Python</b> dictionaries. <b>Python</b> dictionaries allow you to model complex data. This module explores common scenarios where you could use ... This module is part of these learning paths. Build real world applications with <b>Python</b>. Introduction 1 min. What is <b>Python</b>? 3 min. Use the REPL 2 min. Variables and basic data types in <b>Python</b> 4 min. Exercise - output 1 min. Reading keyboard input 3 min. Exercise - Build a calculator 1 min. <b>Python</b>&#39;s source code is freely available to the public, and its usage and distribution are unrestricted, including for commercial purposes. It is widely used for web development, and using it, practically anything can be created, including mobile apps, online apps, tools, data analytics, machine learning, and so on. ... <b>Python</b> is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation. <b>Python</b> is dynamically-typed and garbage-collected. It supports multiple programming paradigms, including structured (particularly procedural), object-oriented and functional programming.'"
"'Thanks to the flexibility of <b>Python</b> and the powerful ecosystem of packages, the Azure CLI supports features such as autocompletion (in shells that support it), persistent credentials, JMESPath result parsing, lazy initialization, network-less unit tests, and more. Building an open-source and cross-platform Azure CLI with <b>Python</b> by Dan Taylor. <b>Python</b> releases by version number: Release version Release date Click for more. <b>Python</b> 3.11.1 Dec. 6, 2022 Download Release Notes. <b>Python</b> 3.10.9 Dec. 6, 2022 Download Release Notes. <b>Python</b> 3.9.16 Dec. 6, 2022 Download Release Notes. <b>Python</b> 3.8.16 Dec. 6, 2022 Download Release Notes. <b>Python</b> 3.7.16 Dec. 6, 2022 Download Release Notes. In this lesson, we will look at the += operator in <b>Python</b> and see how it works with several simple examples.. The operator += is a shorthand for the addition assignment operator.It adds two values and assigns the sum to a variable (left operand). W3Schools offers free online tutorials, references and exercises in all the major languages of the web. Covering popular subjects like HTML, CSS, JavaScript, <b>Python</b>, SQL, Java, and many, many more. This tutorial introduces the reader informally to the basic concepts and features of the <b>Python</b> language and system. It helps to have a <b>Python</b> interpreter handy for hands-on experience, but all examples are self-contained, so the tutorial can be read off-line as well. For a description of standard objects and modules, see The <b>Python</b> Standard ... <b>Python</b> is a general-purpose, versatile, and powerful programming language. It&#39;s a great first language because <b>Python</b> code is concise and easy to read. Whatever you want to do, <b>python</b> can do it. From web development to machine learning to data science, <b>Python</b> is the language for you. To install <b>Python</b> using the Microsoft Store: Go to your Start menu (lower left Windows icon), type &quot;Microsoft Store&quot;, select the link to open the store. Once the store is open, select Search from the upper-right menu and enter &quot;<b>Python</b>&quot;. Select which version of <b>Python</b> you would like to use from the results under Apps. Under the “<b>Python</b> Releases for Mac OS X” heading, click the link for the Latest <b>Python</b> 3 Release - <b>Python</b> 3.x.x. As of this writing, the latest version was <b>Python</b> 3.8.4. Scroll to the bottom and click macOS 64-bit installer to start the download. When the installer is finished downloading, move on to the next step. Step 2: Run the Installer'"
]
},
"execution_count": 4,
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
@ -76,7 +77,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
@ -85,7 +86,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 25,
"metadata": {},
"outputs": [
{
@ -94,7 +95,7 @@
"'Thanks to the flexibility of <b>Python</b> and the powerful ecosystem of packages, the Azure CLI supports features such as autocompletion (in shells that support it), persistent credentials, JMESPath result parsing, lazy initialization, network-less unit tests, and more. Building an open-source and cross-platform Azure CLI with <b>Python</b> by Dan Taylor.'"
]
},
"execution_count": 6,
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
@ -103,12 +104,63 @@
"search.run(\"python\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Metadata Results"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Run query through BingSearch and return snippet, title, and link metadata.\n",
"\n",
"- Snippet: The description of the result.\n",
"- Title: The title of the result.\n",
"- Link: The link to the result."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": []
"source": [
"search = BingSearchAPIWrapper()"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'snippet': 'Lady Alice. Pink Lady <b>apples</b> arent the only lady in the apple family. Lady Alice <b>apples</b> were discovered growing, thanks to bees pollinating, in Washington. They are smaller and slightly more stout in appearance than other varieties. Their skin color appears to have red and yellow stripes running from stem to butt.',\n",
" 'title': '25 Types of Apples - Jessica Gavin',\n",
" 'link': 'https://www.jessicagavin.com/types-of-apples/'},\n",
" {'snippet': '<b>Apples</b> can do a lot for you, thanks to plant chemicals called flavonoids. And they have pectin, a fiber that breaks down in your gut. If you take off the apples skin before eating it, you won ...',\n",
" 'title': 'Apples: Nutrition &amp; Health Benefits - WebMD',\n",
" 'link': 'https://www.webmd.com/food-recipes/benefits-apples'},\n",
" {'snippet': '<b>Apples</b> boast many vitamins and minerals, though not in high amounts. However, <b>apples</b> are usually a good source of vitamin C. Vitamin C. Also called ascorbic acid, this vitamin is a common ...',\n",
" 'title': 'Apples 101: Nutrition Facts and Health Benefits',\n",
" 'link': 'https://www.healthline.com/nutrition/foods/apples'},\n",
" {'snippet': 'Weight management. The fibers in <b>apples</b> can slow digestion, helping one to feel greater satisfaction after eating. After following three large prospective cohorts of 133,468 men and women for 24 years, researchers found that higher intakes of fiber-rich fruits with a low glycemic load, particularly <b>apples</b> and pears, were associated with the least amount of weight gain over time.',\n",
" 'title': 'Apples | The Nutrition Source | Harvard T.H. Chan School of Public Health',\n",
" 'link': 'https://www.hsph.harvard.edu/nutritionsource/food-features/apples/'}]"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"search.results(\"apples\", 5)"
]
}
],
"metadata": {

@ -16,19 +16,19 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 1,
"id": "34bb5968",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"os.environ[\"GOOGLE_CSE_ID\"] = \n",
"os.environ[\"GOOGLE_API_KEY\"] = "
"os.environ[\"GOOGLE_CSE_ID\"] = \"\"\n",
"os.environ[\"GOOGLE_API_KEY\"] = \"\""
]
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 2,
"id": "ac4910f8",
"metadata": {},
"outputs": [],
@ -38,7 +38,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 3,
"id": "84b8f773",
"metadata": {},
"outputs": [],
@ -48,17 +48,17 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 4,
"id": "068991a6",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'STATE OF HAWAII. 1 Child\\'s First Name. (Type or print). 2. Sex. BARACK. 3. This Birth. CERTIFICATE OF LIVE BIRTH. FILE. NUMBER 151 le. lb. Middle Name. Barack Hussein Obama II is an American politician who served as the 44th president of the United States from 2009 to 2017. A member of the Democratic Party,\\xa0... First Lady Michelle LaVaughn Robinson Obama is a lawyer, writer, and the wife of the 44th President, Barack Obama. She is the first African-American First\\xa0... Barack Obama, in full Barack Hussein Obama II, (born August 4, 1961, Honolulu, Hawaii, U.S.), 44th president of the United States (200917) and the first\\xa0... Aug 18, 2017 ... It took him several seconds and multiple clues to remember former President Barack Obama\\'s first name. Miller knew that every answer had to\\xa0... Feb 9, 2015 ... Michael Jordan misspelled Barack Obama\\'s first name on 50th-birthday gift ... Knowing Obama is a Chicagoan and huge basketball fan,\\xa0... His full name is Barack Hussein Obama II. Since the “II” is simply because he was named for his father, his last name is Obama. Jan 16, 2007 ... 4, 1961, in Honolulu. His first name means \"one who is blessed\" in Swahili. While Obama\\'s father, Barack Hussein Obama Sr., was from Kenya, his\\xa0... Jan 19, 2017 ... Hopeful parents named their sons for the first Black president, whose name is a variation of the Hebrew name Baruch, which means “blessed”\\xa0... Feb 27, 2020 ... President Barack Obama was born Barack Hussein Obama, II, as shown here on his birth certificate here . As reported by Reuters here , his\\xa0...'"
"'1 Child\\'s First Name. 2. 6. 7d. Street Address. 71. (Type or print). BARACK. Sex. 3. This Birth. 4. If Twin or Triplet,. Was Child Born. Barack Hussein Obama II is an American retired politician who served as the 44th president of the United States from 2009 to 2017. His full name is Barack Hussein Obama II. Since the “II” is simply because he was named for his father, his last name is Obama. Feb 9, 2015 ... Michael Jordan misspelled Barack Obama\\'s first name on 50th-birthday gift ... Knowing Obama is a Chicagoan and huge basketball fan,\\xa0... Aug 18, 2017 ... It took him several seconds and multiple clues to remember former President Barack Obama\\'s first name. Miller knew that every answer had to end\\xa0... First Lady Michelle LaVaughn Robinson Obama is a lawyer, writer, and the wife of the 44th President, Barack Obama. She is the first African-American First\\xa0... Barack Obama, in full Barack Hussein Obama II, (born August 4, 1961, Honolulu, Hawaii, U.S.), 44th president of the United States (200917) and the first\\xa0... When Barack Obama was elected president in 2008, he became the first African American to hold ... The Middle East remained a key foreign policy challenge. Feb 27, 2020 ... President Barack Obama was born Barack Hussein Obama, II, as shown here on his birth certificate here . As reported by Reuters here , his\\xa0... Jan 16, 2007 ... 4, 1961, in Honolulu. His first name means \"one who is blessed\" in Swahili. While Obama\\'s father, Barack Hussein Obama Sr., was from Kenya, his\\xa0...'"
]
},
"execution_count": 7,
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
@ -67,13 +67,118 @@
"search.run(\"Obama's first name?\")"
]
},
{
"cell_type": "markdown",
"id": "074b7f07",
"metadata": {},
"source": [
"## Number of Results\n",
"You can use the `k` parameter to set the number of results"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "5083fbdd",
"metadata": {},
"outputs": [],
"source": [
"search = GoogleSearchAPIWrapper(k=1)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "77aaa857",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'The official home of the Python Programming Language.'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"search.run(\"python\")"
]
},
{
"cell_type": "markdown",
"id": "11c8d94f",
"metadata": {},
"source": [
"'The official home of the Python Programming Language.'"
]
},
{
"cell_type": "markdown",
"id": "73473110",
"metadata": {},
"source": [
"## Metadata Results"
]
},
{
"cell_type": "markdown",
"id": "109fe796",
"metadata": {},
"source": [
"Run query through GoogleSearch and return snippet, title, and link metadata.\n",
"\n",
"- Snippet: The description of the result.\n",
"- Title: The title of the result.\n",
"- Link: The link to the result."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 7,
"id": "028f4cba",
"metadata": {},
"outputs": [],
"source": []
"source": [
"search = GoogleSearchAPIWrapper()"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "4d8f734f",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'snippet': 'Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment,\\xa0...',\n",
" 'title': 'Apple',\n",
" 'link': 'https://www.apple.com/'},\n",
" {'snippet': \"Jul 10, 2022 ... Whether or not you're up on your apple trivia, no doubt you know how delicious this popular fruit is, and how nutritious. Apples are rich in\\xa0...\",\n",
" 'title': '25 Types of Apples and What to Make With Them - Parade ...',\n",
" 'link': 'https://parade.com/1330308/bethlipton/types-of-apples/'},\n",
" {'snippet': 'An apple is an edible fruit produced by an apple tree (Malus domestica). Apple trees are cultivated worldwide and are the most widely grown species in the\\xa0...',\n",
" 'title': 'Apple - Wikipedia',\n",
" 'link': 'https://en.wikipedia.org/wiki/Apple'},\n",
" {'snippet': 'Apples are a popular fruit. They contain antioxidants, vitamins, dietary fiber, and a range of other nutrients. Due to their varied nutrient content,\\xa0...',\n",
" 'title': 'Apples: Benefits, nutrition, and tips',\n",
" 'link': 'https://www.medicalnewstoday.com/articles/267290'},\n",
" {'snippet': \"An apple is a crunchy, bright-colored fruit, one of the most popular in the United States. You've probably heard the age-old saying, “An apple a day keeps\\xa0...\",\n",
" 'title': 'Apples: Nutrition & Health Benefits',\n",
" 'link': 'https://www.webmd.com/food-recipes/benefits-apples'}]"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"search.results(\"apples\", 5)"
]
}
],
"metadata": {
@ -93,6 +198,11 @@
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
},
"vscode": {
"interpreter": {
"hash": "a0a0263b650d907a3bfe41c0f8d6a63a071b884df3cfdc1579f00cdc1aed6b03"
}
}
},
"nbformat": 4,

@ -54,7 +54,7 @@ class BingSearchAPIWrapper(BaseModel):
values,
"bing_search_url",
"BING_SEARCH_URL",
default="https://api.bing.microsoft.com/v7.0/search",
# default="https://api.bing.microsoft.com/v7.0/search",
)
values["bing_search_url"] = bing_search_url
@ -71,3 +71,30 @@ class BingSearchAPIWrapper(BaseModel):
snippets.append(result["snippet"])
return " ".join(snippets)
def results(self, query: str, num_results: int) -> List[Dict]:
"""Run query through BingSearch and return metadata.
Args:
query: The query to search for.
num_results: The number of results to return.
Returns:
A list of dictionaries with the following keys:
snippet - The description of the result.
title - The title of the result.
link - The link to the result.
"""
metadata_results = []
results = self._bing_search_results(query, count=num_results)
if len(results) == 0:
return [{"Result": "No good Bing Search Result was found"}]
for result in results:
metadata_result = {
"snippet": result["snippet"],
"title": result["name"],
"link": result["url"],
}
metadata_results.append(metadata_result)
return metadata_results

@ -48,6 +48,7 @@ class GoogleSearchAPIWrapper(BaseModel):
search_engine: Any #: :meta private:
google_api_key: Optional[str] = None
google_cse_id: Optional[str] = None
k: int = 10
class Config:
"""Configuration for this pydantic object."""
@ -91,10 +92,37 @@ class GoogleSearchAPIWrapper(BaseModel):
def run(self, query: str) -> str:
"""Run query through GoogleSearch and parse result."""
snippets = []
results = self._google_search_results(query, num=10)
results = self._google_search_results(query, num=self.k)
if len(results) == 0:
return "No good Google Search Result was found"
for result in results:
snippets.append(result["snippet"])
return " ".join(snippets)
def results(self, query: str, num_results: int) -> List[Dict]:
"""Run query through GoogleSearch and return metadata.
Args:
query: The query to search for.
num_results: The number of results to return.
Returns:
A list of dictionaries with the following keys:
snippet - The description of the result.
title - The title of the result.
link - The link to the result.
"""
metadata_results = []
results = self._google_search_results(query, num=num_results)
if len(results) == 0:
return [{"Result": "No good Google Search Result was found"}]
for result in results:
metadata_result = {
"snippet": result["snippet"],
"title": result["title"],
"link": result["link"],
}
metadata_results.append(metadata_result)
return metadata_results

Loading…
Cancel
Save