You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
searxng/searx
Markus Heiser 14323d683f [fix] ddg-lite & ddg-extra: don't send empty vqd value
DDG's bot detection is sensitive to the vqd value.  For some search terms (such
as extremely long search terms that are often sent by bots), no vqd value can be
determined.

If SearXNG cannot determine a vqd value, then no request should go out to
DDG (WEB): a request with a wrong vqd value leads to DDG temporarily putting
SearXNG's IP on a block list.

Requests from IPs in this block list run into timeouts.

Not sure, but it seems the block list is a sliding window: to get my IP rid from
the bot list I had to cool down my IP for 1h (send no requests from that IP to
DDG).

Since such issues can't reproduce in a local instance I tested this patch 24h on
my public SearXNG instance: There are still errors (rare), but the reliability
is still 100%.

Related:

- https://github.com/searxng/searxng/pull/2922
- https://github.com/searxng/searxng/pull/2923

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
11 months ago
..
answerers
botdetection [fix] limiter / botdetection: remove http_connection method 12 months ago
data [feat] duckduckgo: support for videos and news 12 months ago
enginelib
engines [fix] ddg-lite & ddg-extra: don't send empty vqd value 11 months ago
infopage
metrics
network
plugins
search
static
templates/simple [fix] categories.html: can't select social media category using search on category select 12 months ago
tools
translations [translations] update from Weblate 12 months ago
__init__.py add warning about enabling public_instance setting 1 year ago
autocomplete.py
babel_extract.py
compat.py
exceptions.py
external_bang.py
external_urls.py
flaskfix.py
locales.py
preferences.py [mod] add hotkeys option to settings.yml 12 months ago
query.py
redisdb.py
redislib.py
results.py
searxng.msg
settings.yml [mod] yacy: use official instance by default and fix crashes 12 months ago
settings_defaults.py [mod] add hotkeys option to settings.yml 12 months ago
settings_loader.py
sxng_locales.py Update searx.data - update_engine_traits.py 12 months ago
unixthreadname.py
utils.py
version.py
webadapter.py
webapp.py
webutils.py