Commit Graph

13 Commits (9b482e8fccfc5e5504e4c8abdb60ce5c11678018)

Author SHA1 Message Date
Alexandre Flament c27fef1cde [mod] metrics: add secondary parameter
Some error won't stop the engine:
* additional HTTP redirects for example
* some invalid results

secondary=True allows to flag these errors as not important.
3 years ago
Alexandre Flament 7acd7ffc02 [enh] rewrite and enhance metrics 3 years ago
Alexandre Flament aae7830d14 [mod] refactoring: processors
Report to the user suspended engines.

searx.search.processor.abstract:
* manages suspend time (per network).
* reports suspended time to the ResultContainer (method extend_container_if_suspended)
* adds the results to the ResultContainer (method extend_container)
* handles exceptions (method handle_exception)
3 years ago
Alexandre Flament d14994dc73 [httpx] replace searx.poolrequests by searx.network
settings.yml:

* outgoing.networks:
   * can contains network definition
   * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
     keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
   * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
   * local_addresses can be "192.168.0.1/24" (it supports IPv6)
   * support_ipv4 & support_ipv6: both True by default
     see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
   * either a full network description
   * either reference an existing network

* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
3 years ago
Alexandre Flament eaa694fb7d [enh] replace requests by httpx 3 years ago
Alexandre Flament 99e0651cea [mod] by default allow only HTTPS, not HTTP
Related to https://github.com/searx/searx/pull/2373
3 years ago
Alexandre Flament c22d4c764c [fix] duckduckgo engine: "!ddg !g" do not redirect to google
* searx understand "!ddg !g time" as : send "!g time" to DDG
* !g a DDG bang for Google: DDG return a HTTP redirect to Google

This commit adds a the allows_redirect param not to follow HTTP redirect.

The DDG engine returns a empty result as before without HTTP redirect.
3 years ago
Alexandre Flament aedf03c0f7 Fix: activate raise_for_error by default
Fix commit d703119d3a :
Some engines need to parse the HTTP error but
raise_for_error is always set to False in the "request" function.
3 years ago
Alexandre Flament 3b7b852aa8 [fix] checker: minor fix about language detection 3 years ago
Alexandre Flament f3e1bd308f [mod] checker: minor adjustements on the default tests
the query "time" is convinient because most of the search engine will return some results,
but some engines in the general category will return documentation about the HTML tags <time> or <input type="time">
3 years ago
Alexandre Flament 8cbc9f2d58 [enh] add checker 3 years ago
Alexandre Flament 5c6a5407a0 [fix] fix of PR #2225 4 years ago
Alexandre Flament 7ec8bc3ea7 [mod] split searx.search into different processors
see searx.search.processors.abstract.EngineProcessor

First the method searx call the get_params method.

If the return value is not None, then the searx call the method search.
4 years ago