blob42
/
searxng
forked from Archives/searxng
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,439 Commits
10 Branches
17 Tags
41 MiB
whitelist-ratelimiter
mine
gh-pages
master
dependabot/pip/master/flask-babel-3.0.0
translations
dependabot/pip/master/mock-5.0.1
dependabot/pip/master/yamllint-1.29.0
dependabot/pip/master/pylint-2.15.10
dependabot/pip/master/sphinx-6.1.3
v1.0.0
v0.18.0
v0.17.0
v0.16.0
v0.15.0
v0.14.0
v0.13.1
v0.13.0
v0.12.0
v0.11.0
v0.10.0
v0.9.0
v0.8.1
v0.8.0
v0.7.0
v0.6.0
1.0.0
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '3d96a9839a'
${ noResults }
Commit Graph

81 Commits (3d96a9839a12649874b6d4cf9466bd3616b0a03c)

Author SHA1 Message Date
Markus Heiser 3d96a9839a [format.python] initial formatting of the python code 
This patch was generated by black [1]::

    make format.python

[1] https://github.com/psf/black

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Markus Heiser 488ace1da9 [fix] google engine - suggestion 
BTW: google no longer offers *spelling suggestions*

Closes: https://github.com/searxng/searxng/issues/442
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Markus Heiser f0059b80ed [pylint] engines: drop no longer needed 'missing-function-docstring' 
Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914168470
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Markus Heiser cd033b5416 [fix] drop useless pylint: disable=undefined-variable 
Since 7b235a1 (see line 591) it is no longer needed to disable
'undefined-variable' for names defined in::

   PYLINT_ADDITIONAL_BUILTINS_FOR_ENGINES

Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914068609
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Markus Heiser aecfb2300d [mod] one logger per engine - drop obsolete logger.getChild 
Remove the no longer needed `logger = logger.getChild(...)` from engines.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Noémi Ványi 3d5e6e0abb [enh] google: add filter=0 to Google engine for more results 
backport from searx ( 23b3b56a06ef831af0a1b30a12c26ebd50e329bb )
 3 years ago
Émilien Devos 6c9f276571
Add missing parameter for mobile UI search 3 years ago
Markus Heiser 0ef6aa5126 [docs] add documentation from the sources of the google engines 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Markus Heiser 05e90f2e57 [fix] google answers: normalize space of the answers. 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Markus Heiser f096d68ec6 [mod] google engine: reduce mobile UI parameters to what is needed 
Reverse engineering shows that not all of the parameters used by google's mobile
UI (aka "more results" button) are needed [1].

[1] https://github.com/searxng/searxng/pull/160#issuecomment-865013625

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Alexandre Flament 7a5c36408a [mod] google: add "use_mobile_ui" parameter to use mobile endpoint. 
disable by default, it has to be enabled in settings.yml

related to  #159
 3 years ago
Markus Heiser 1ac3961336 [mod] google - get_lang_info add documentataion & comments 
BTW: remove obsolete log messages from google engine

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Alexandre Flament 1c67b6aece [enh] google engine: supports "default language" 
Same behaviour behaviour than Whoogle [1].  Only the google engine with the
"Default language" choice "(all)"" is changed by this patch.

When searching for a locate place, the result are in the expect language,
without missing results [2]:

  > When a language is not specified, the language interpretation is left up to
  > Google to decide how the search results should be delivered.

The query parameters are copied from Whoogle.  With the ``all`` language:

- add parameter ``source=lnt``
- don't use parameter ``lr``
- don't add a ``Accept-Language`` HTTP header.

The new signature of function ``get_lang_info()`` is:

    lang_info = get_lang_info(params, lang_list, custom_aliases, supported_any_language)

Argument ``supported_any_language`` is True for google.py and False for the other
google engines.  With this patch the function now returns:

- query parameters: ``lang_info['params']``
- HTTP headers: ``lang_info['headers']``
- and as before this patch:
  - ``lang_info['subdomain']``
  - ``lang_info['country']``
  - ``lang_info['language']``

[1] https://github.com/benbusby/whoogle-search
[2] https://github.com/benbusby/whoogle-search/releases/tag/v0.5.4
 3 years ago
Markus Heiser dc29f1d826 [pylint] tag PYLINT_FILES by comment `# lint: pylint` 
These py files are linted by `test.pylint`, all other files are linted by
`test.pep8`.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Alexandre Flament 48720e20a8 Merge remote-tracking branch 'searx/master' 3 years ago
Robin Schneider dfc66ff0f0
Fix grammar mistake in debug log output 3 years ago
Alexandre Flament eaa694fb7d [enh] replace requests by httpx 3 years ago
Alexandre Flament ca93a01844 [mod] dynamically set language_support variable 
The language_support variable is set to True by default,
and set to False in only 5 engines.

Except the documentation and the /config URL, this variable is not used.

This commit remove the variable definition in the engines, and
set value according to supported_languages length: False when the length is 0,
True otherwise.

Close #2485
 3 years ago
Markus Heiser 7f505bdc6f [fix] google: avoid unnecessary SearxEngineXPathException errors 
Avoid SearxEngineXPathException errors when parsing non valid results::

    .//div[@class="yuRUbf"]//a/@href index 0 not found
    Traceback (most recent call last):
      File "./searx/engines/google.py", line 274, in response
        url = eval_xpath_getindex(result, href_xpath, 0)
      File "./searx/searx/utils.py", line 608, in eval_xpath_getindex
        raise SearxEngineXPathException(xpath_spec, 'index ' + str(index) + ' not found')
    searx.exceptions.SearxEngineXPathException: .//div[@class="yuRUbf"]//a/@href index 0 not found

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Markus Heiser b1fefec40d [fix] normalize the language & region aspects of all google engines 
BTW: make the engines ready for search.checker:

- replace eval_xpath by eval_xpath_getindex and eval_xpath_list
- google_images: remove outer try/except block

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Markus Heiser baec54c492 [fix] revise of the google-news engine 
This revise is based on the methods developed in the revise of the google engine
(see commit 410c2f9).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Alexandre Flament a4dcfa025c [enh] engines: add about variable 
move meta information from comment to the about variable
so the preferences, the documentation can show these information
 3 years ago
Alexandre Flament 64cccae99e [mod] various engines: use eval_xpath* functions and searx.exceptions.* 
Engine list: ahmia, duckduckgo_images, elasticsearch, google, google_images, google_videos, youtube_api
 4 years ago
Alexandre Flament 2006eb4680 [mod] move extract_text, extract_url to searx.utils 4 years ago
Markus Heiser 8162d7aff4 [fix] google engine - div classes has been renamed in HTML reult 
Since 1. October 2020 google has changed the 'class' attribute of the HTML
result page.

Fix the xpath expressions and ignore <div class="g" ../> sections which do not
match to title's xpath expression.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 4 years ago
Marc Abonce Seguin ecf5899153 fetch google's search langs rather than ui langs 4 years ago
Dalf 1022228d95 Drop Python 2 (1/n): remove unicode string and url_utils 4 years ago
Adam Tauber 52eba0c721 [fix] pep8 4 years ago
Markus Heiser 410c2f903d [fix] revise google engine 
this commit is picked from #1985
 4 years ago
Marc Abonce Seguin ccaf6ca02c [fix] update xpaths for new google results page 5 years ago
Adam Tauber 731e34299d
Merge pull request #1744 from dalf/optimizations 
[mod] speed optimization
 5 years ago
Emilien Devos 8f51430f5c [fix] Force Google old UI with a new user agent 5 years ago
Dalf 85b3723345 [mod] speed optimization 
compile XPath only once
avoid redundant call to urlparse
get_locale(webapp.py): avoid useless call to request.accept_languages.best_match
 5 years ago
Emilien Devos cbd1ebdce8 [fix] Force Google old UI (#1597) 5 years ago
Noémi Ványi b63d645a52 Revert "remove 'all' option from search languages" 
This reverts commit 4d1770398a.
 5 years ago
Marc Abonce Seguin 0169b63e84 [fix] fetch google's supported languages 5 years ago
Marc Abonce Seguin 5568f24d6c [fix] check language aliases when setting search language 5 years ago
Marc Abonce Seguin f7f9c50393 [fix] force English results in Google when using en-US 6 years ago
Marc Abonce Seguin 772c048d01 refactor engine's search language handling 
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.

Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
 6 years ago
Marc Abonce Seguin d1eae9359f fix fetch_langauges to be more accurate 
Add languages supported by either all default general engines or 10 engines.
 6 years ago
Noémi Ványi 2d5eed9b59 send constant cookie with query to Google 7 years ago
marc 4d1770398a remove 'all' option from search languages 7 years ago
Adam Tauber 1613c6319e [fix] handle /sorry redirects 7 years ago
Adam Tauber 6eb9503896 [fix] use english in google engine if no language was set - this prevents guessing the language by the IP of the instance 7 years ago
Adam Tauber 6fdb6640d9 [fix] revert language changes to prevent CAPTCHAs 7 years ago
Adam Tauber 9ab8536479 [fix] fix language support of google 7 years ago
Adam Tauber 52e615dede [enh] py3 compatibility 7 years ago
Adam Tauber 52d1087202 [enh] add result number parsing to google engine 7 years ago
David A Roberts 1d30141c20 [enh] show spelling corrections 7 years ago
Adam Tauber 0d4da30c7f [enh] add instant answers to google engine 8 years ago