searx: implement async + helper tool providing json results (#2129 )

- implemented `arun` and `aresults`. Reuses aiosession if available.
- helper tools `SearxSearchRun` and `SearxSearchResults`
- update doc

Co-authored-by: blob42 <spike@w530>

2023-03-28 22:49:02 -07:00

2.3 KiB

Raw Blame History

SearxNG Search API

This page covers how to use the SearxNG search API within LangChain. It is broken into two parts: installation and setup, and then references to the specific SearxNG API wrapper.

Installation and Setup

While it is possible to utilize the wrapper in conjunction with public searx instances these instances frequently do not permit API access (see note on output format below) and have limitations on the frequency of requests. It is recommended to opt for a self-hosted instance instead.

Self Hosted Instance:

See this page for installation instructions.

When you install SearxNG, the only active output format by default is the HTML format. You need to activate the json format to use the API. This can be done by adding the following line to the settings.yml file:

search:
    formats:
        - html
        - json

You can make sure that the API is working by issuing a curl request to the API endpoint:

curl -kLX GET --data-urlencode q='langchain' -d format=json http://localhost:8888

This should return a JSON object with the results.

Wrappers

Utility

To use the wrapper we need to pass the host of the SearxNG instance to the wrapper with: 1. the named parameter searx_host when creating the instance. 2. exporting the environment variable SEARXNG_HOST.

You can use the wrapper to get results from a SearxNG instance.

from langchain.utilities import SearxSearchWrapper
s = SearxSearchWrapper(searx_host="http://localhost:8888")
s.run("what is a large language model?")

Tool

You can also load this wrapper as a Tool (to use with an Agent).

You can do this with:

from langchain.agents import load_tools
tools = load_tools(["searx-search"],
                    searx_host="http://localhost:8888",
                    engines=["github"])

Note that we could optionally pass custom engines to use.

If you want to obtain results with metadata as json you can use:

tools = load_tools(["searx-search-results-json"],
                    searx_host="http://localhost:8888",
                    num_results=5)

For more information on tools, see this page

2.3 KiB Raw Blame History