searx: add install instructions, update doc and notebooks (#1420)

- Added instructions on setting up self hosted searx
- Add notebook example with agent
- Use `localhost:8888` as example url to stay consistent since public
instances are not really usable.

Co-authored-by: blob42 <spike@w530>
fix-searx
blob42 1 year ago committed by GitHub
parent bca0935d90
commit 3d54b05863
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -5,21 +5,44 @@ It is broken into two parts: installation and setup, and then references to the
## Installation and Setup
- You can find a list of public SearxNG instances [here](https://searx.space/).
- It recommended to use a self-hosted instance to avoid abuse on the public instances. Also note that public instances often have a limit on the number of requests.
- To run a self-hosted instance see [this page](https://searxng.github.io/searxng/admin/installation.html) for more information.
- To use the tool you need to provide the searx host url by:
1. passing the named parameter `searx_host` when creating the instance.
2. exporting the environment variable `SEARXNG_HOST`.
While it is possible to utilize the wrapper in conjunction with [public searx
instances](https://searx.space/) these instances frequently do not permit API
access (see note on output format below) and have limitations on the frequency
of requests. It is recommended to opt for a self-hosted instance instead.
### Self Hosted Instance:
See [this page](https://searxng.github.io/searxng/admin/installation.html) for installation instructions.
When you install SearxNG, the only active output format by default is the HTML format.
You need to activate the `json` format to use the API. This can be done by adding the following line to the `settings.yml` file:
```yaml
search:
formats:
- html
- json
```
You can make sure that the API is working by issuing a curl request to the API endpoint:
`curl -kLX GET --data-urlencode q='langchain' -d format=json http://localhost:8888`
This should return a JSON object with the results.
## Wrappers
### Utility
To use the wrapper we need to pass the host of the SearxNG instance to the wrapper with:
1. the named parameter `searx_host` when creating the instance.
2. exporting the environment variable `SEARXNG_HOST`.
You can use the wrapper to get results from a SearxNG instance.
```python
from langchain.utilities import SearxSearchWrapper
s = SearxSearchWrapper(searx_host="http://localhost:8888")
s.run("what is a large language model?")
```
### Tool
@ -29,7 +52,7 @@ You can do this with:
```python
from langchain.agents import load_tools
tools = load_tools(["searx-search"], searx_host="https://searx.example.com")
tools = load_tools(["searx-search"], searx_host="http://localhost:8888")
```
For more information on this, see [this page](../modules/agents/tools.md)
For more information on tools, see [this page](../modules/agents/tools.md)

@ -12,7 +12,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 2,
"id": "e6860c2d",
"metadata": {
"pycharm": {
@ -28,7 +28,7 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 3,
"id": "dadbcfcd",
"metadata": {},
"outputs": [],
@ -238,6 +238,92 @@
"source": [
"agent.run(\"What is the weather in Pomfret?\")"
]
},
{
"cell_type": "markdown",
"id": "eabad3af",
"metadata": {},
"source": [
"## SearxNG Meta Search Engine\n",
"\n",
"Here we will be using a self hosted SearxNG meta search engine."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "b196c704",
"metadata": {},
"outputs": [],
"source": [
"tools = load_tools([\"searx-search\"], searx_host=\"http://localhost:8888\", llm=llm)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "9023eeaa",
"metadata": {},
"outputs": [],
"source": [
"agent = initialize_agent(tools, llm, agent=\"zero-shot-react-description\", verbose=True)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "3aad92c1",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I should look up the current weather\n",
"Action: SearX Search\n",
"Action Input: \"weather in Pomfret\"\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mMainly cloudy with snow showers around in the morning. High around 40F. Winds NNW at 5 to 10 mph. Chance of snow 40%. Snow accumulations less than one inch.\n",
"\n",
"10 Day Weather - Pomfret, MD As of 1:37 pm EST Today 49°/ 41° 52% Mon 27 | Day 49° 52% SE 14 mph Cloudy with occasional rain showers. High 49F. Winds SE at 10 to 20 mph. Chance of rain 50%....\n",
"\n",
"10 Day Weather - Pomfret, VT As of 3:51 am EST Special Weather Statement Today 39°/ 32° 37% Wed 01 | Day 39° 37% NE 4 mph Cloudy with snow showers developing for the afternoon. High 39F....\n",
"\n",
"Pomfret, CT ; Current Weather. 1:06 AM. 35°F · RealFeel® 32° ; TODAY'S WEATHER FORECAST. 3/3. 44°Hi. RealFeel® 50° ; TONIGHT'S WEATHER FORECAST. 3/3. 32°Lo.\n",
"\n",
"Pomfret, MD Forecast Today Hourly Daily Morning 41° 1% Afternoon 43° 0% Evening 35° 3% Overnight 34° 2% Don't Miss Finally, Heres Why We Get More Colds and Flu When Its Cold Coast-To-Coast...\n",
"\n",
"Pomfret, MD Weather Forecast | AccuWeather Current Weather 5:35 PM 35° F RealFeel® 36° RealFeel Shade™ 36° Air Quality Excellent Wind E 3 mph Wind Gusts 5 mph Cloudy More Details WinterCast...\n",
"\n",
"Pomfret, VT Weather Forecast | AccuWeather Current Weather 11:21 AM 23° F RealFeel® 27° RealFeel Shade™ 25° Air Quality Fair Wind ESE 3 mph Wind Gusts 7 mph Cloudy More Details WinterCast...\n",
"\n",
"Pomfret Center, CT Weather Forecast | AccuWeather Daily Current Weather 6:50 PM 39° F RealFeel® 36° Air Quality Fair Wind NW 6 mph Wind Gusts 16 mph Mostly clear More Details WinterCast...\n",
"\n",
"12:00 pm · Feels Like36° · WindN 5 mph · Humidity43% · UV Index3 of 10 · Cloud Cover65% · Rain Amount0 in ...\n",
"\n",
"Pomfret Center, CT Weather Conditions | Weather Underground star Popular Cities San Francisco, CA 49 °F Clear Manhattan, NY 37 °F Fair Schiller Park, IL (60176) warning39 °F Mostly Cloudy...\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The current weather in Pomfret is mainly cloudy with snow showers around in the morning. The temperature is around 40F with winds NNW at 5 to 10 mph. Chance of snow is 40%.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The current weather in Pomfret is mainly cloudy with snow showers around in the morning. The temperature is around 40F with winds NNW at 5 to 10 mph. Chance of snow is 40%.'"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"What is the weather in Pomfret\")"
]
}
],
"metadata": {
@ -256,7 +342,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
"version": "3.9.11"
},
"vscode": {
"interpreter": {

@ -1,7 +1,13 @@
"""Chain that calls SearxNG meta search API.
SearxNG is a privacy-friendly free metasearch engine that aggregates results from
multiple search engines and databases.
`multiple search engines
<https://docs.searxng.org/admin/engines/configured_engines.html>`_ and databases and
supports the `OpenSearch
<https://github.com/dewitt/opensearch/blob/master/opensearch-1-1-draft-6.md>`_
specification.
More detailes on the installtion instructions `here. <../../ecosystem/searx.html>`_
For the search API refer to https://docs.searxng.org/dev/search_api.html
@ -176,7 +182,7 @@ class SearxSearchWrapper(BaseModel):
.. code-block:: python
from langchain.utilities import SearxSearchWrapper
searx = SearxSearchWrapper(searx_host="https://searx.example.com")
searx = SearxSearchWrapper(searx_host="http://localhost:8888")
Example with SSL disabled:
.. code-block:: python
@ -184,7 +190,7 @@ class SearxSearchWrapper(BaseModel):
from langchain.utilities import SearxSearchWrapper
# note the unsecure parameter is not needed if you pass the url scheme as
# http
searx = SearxSearchWrapper(searx_host="http://searx.example.com",
searx = SearxSearchWrapper(searx_host="http://localhost:8888",
unsecure=True)

Loading…
Cancel
Save