blob42
/
searxng
forked from Archives/searxng
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,900 Commits
10 Branches
17 Tags
41 MiB
whitelist-ratelimiter
mine
gh-pages
master
dependabot/pip/master/flask-babel-3.0.0
translations
dependabot/pip/master/mock-5.0.1
dependabot/pip/master/yamllint-1.29.0
dependabot/pip/master/pylint-2.15.10
dependabot/pip/master/sphinx-6.1.3
v1.0.0
v0.18.0
v0.17.0
v0.16.0
v0.15.0
v0.14.0
v0.13.1
v0.13.0
v0.12.0
v0.11.0
v0.10.0
v0.9.0
v0.8.1
v0.8.0
v0.7.0
v0.6.0
1.0.0
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'afd71a6c0f'
${ noResults }
Commit Graph

56 Commits (afd71a6c0fb2dad424ec184517e0425e457b85cf)

Author SHA1 Message Date
Ahmad Alkadri 7fc8d72889 [fix] bing: parsing result; check to see if the element contains links 
This patch is to hardening the parsing of the bing response:

1. To fix [2087] check if the selected result item contains a link, otherwise
   skip result item and continue in the result loop.  Increment the result
   pointer when a result has been added / the enumerate that counts for skipped
   items is no longer valid when result items are skipped.

   To test the bugfix use:   ``!bi :all cerbot``

2. Limit the XPath selection of result items to direct children nodes (list
   items ``li``) of the ordered list (``ol``).

   To test the selector use: ``!bi :en pontiac aztek wiki``

   .. in the result list you should find the wikipedia entry on top,
   compare [2068]

[2087] https://github.com/searxng/searxng/issues/2087
[2068] https://github.com/searxng/searxng/issues/2068
 1 year ago
ahmad-alkadri 9ee99423fe [fix] Bing-Web engine: XPath to get the wikipedia result 
Modify the XPath selector to get the wikipedia result plus small fixes.

About result content: especially with the Wikipedia result, we'd get several
paragraph elements, only the first paragraph would be taken and displayed on the
search result
 1 year ago
Markus Heiser 8df1f0c47e [mod] add 'Accept-Language' HTTP header to online processores 
Most engines that support languages (and regions) use the Accept-Language from
the WEB browser to build a response that fits to the language (and region).

- add new engine option: send_accept_language_header

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Alexandre Flament a1e8af0796 bing.py: resolve bing.com/ck/a redirections 
add a new function searx.network.multi_requests to send multiple HTTP requests at once
 2 years ago
Alexandre FLAMENT f00cdb5e51 bing engine: _fetch_supported_languages: don't use the language code as a country 
ref #1029
 2 years ago
Martin Fischer b02f762687 [enh] add more categories 2 years ago
Markus Heiser 61ce0c2244 [fix] bing engines: fetch_supported_languages 
The Request to and the Response from https://www.bing.com/account/general has
been changed.

[1] https://github.com/searxng/searxng/pull/672#discussion_r777104919

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Markus Heiser 3d96a9839a [format.python] initial formatting of the python code 
This patch was generated by black [1]::

    make format.python

[1] https://github.com/psf/black

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Markus Heiser 6b85607274 [fix] bing engine: fix paging support, show inital page. 
Follow up queries for the pages needed to be fixed.

- Split search-term in one for initial query and one for following queries.
- Set some headers in HTTP requests, bing needs for paging support.
- IMO //div[@class="sa_cc"] does no longer match in a bing response.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Markus Heiser b2177e5916 [pylint] Bing (Web) engine 
Fix remarks from pylint and improved code-style.  In preparation for a bug-fix
of the Bing (Web) engine I add this engine to the pylint-list.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Markus Heiser aecfb2300d [mod] one logger per engine - drop obsolete logger.getChild 
Remove the no longer needed `logger = logger.getChild(...)` from engines.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Alexandre Flament ca93a01844 [mod] dynamically set language_support variable 
The language_support variable is set to True by default,
and set to False in only 5 engines.

Except the documentation and the /config URL, this variable is not used.

This commit remove the variable definition in the engines, and
set value according to supported_languages length: False when the length is 0,
True otherwise.

Close #2485
 3 years ago
Alexandre Flament a4dcfa025c [enh] engines: add about variable 
move meta information from comment to the about variable
so the preferences, the documentation can show these information
 3 years ago
Alexandre Flament 3038052c79 [mod] remove unused import 
use
from searx.engines.duckduckgo import _fetch_supported_languages, supported_languages_url  # NOQA
so it is possible to easily remove all unused import using autoflake:
autoflake --in-place --recursive --remove-all-unused-imports searx tests
 4 years ago
Alexandre Flament 2006eb4680 [mod] move extract_text, extract_url to searx.utils 4 years ago
Dalf c225db45c8 Drop Python 2 (4/n): SearchQuery.query is a str instead of bytes 4 years ago
Dalf 1022228d95 Drop Python 2 (1/n): remove unicode string and url_utils 4 years ago
Markus Heiser 1c853f9573 bing_news: parital rollback of c89c05bc 
The bing_news bug (discussed in #1838) was caused by wrong language tags, which
was fixed e0c99d9d / no need to change the bing_news search string.

closes: https://github.com/asciimoo/searx/issues/1838

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 4 years ago
Markus Heiser e0c99d9dcb bugfix: fetch_supported_languages bing, -news, -videos, -images 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 4 years ago
Adam Tauber 17b6faa4c3 [fix] pep8 4 years ago
Adam Tauber 2292e6e130 [fix] handle missing result size 4 years ago
Dalf 85b3723345 [mod] speed optimization 
compile XPath only once
avoid redundant call to urlparse
get_locale(webapp.py): avoid useless call to request.accept_languages.best_match
 5 years ago
Léo Bourrel 88261e111c Fix bing engine results count (#1387) 
This PR fixes the result count from bing which was throwing an (hidden) error and add a validation to avoid reading more results than avalaible.

For example :
If there is 100 results from some search and we try to get results from 120 to 130, Bing will send back the results from 0 to 10 and no error. If we compare results count with the first parameter of the request we can avoid this "invalid" results.
 5 years ago
Dalf 1cee2c1796 [fix] bing engine 
before this commit, sometimes there are no results
use a generic user-agent instead of one with the OS "Windows NT 6.3; WOW64"
 5 years ago
Noémi Ványi b63d645a52 Revert "remove 'all' option from search languages" 
This reverts commit 4d1770398a.
 5 years ago
Marc Abonce Seguin 75b276f408 fix bing "garbage" results (issue #1275) 6 years ago
Marc Abonce Seguin 772c048d01 refactor engine's search language handling 
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.

Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
 6 years ago
marc 4d1770398a remove 'all' option from search languages 7 years ago
Adam Tauber 52e615dede [enh] py3 compatibility 7 years ago
marc fd65c12921 make search language handling less strict 
languages.py can change, so users may query on a language that is not
on the list anymore, even if it is still recognized by a few engines.

also made no and nb the same because they seem to return the same,
though most engines will only support one or the other.
 7 years ago
Adam Tauber 6bf9c398a7 [fix] use english as default language in bing 
If no language is specified, bing returns results with multiple languages
for one query which isn't really useful. Setting english as default
insted if nothing.
 8 years ago
marc af35eee10b tests for _fetch_supported_languages in engines 
and refactor method to make it testable without making requests
 8 years ago
marc f62ce21f50 [mod] fetch supported languages for several engines 
utils/fetch_languages.py gets languages supported by each engine and
generates engines_languages.json with each engine's supported language.
 8 years ago
marc 149802c569 [enh] add supported_languages on engines and auto-generate languages.py 8 years ago
Adam Tauber 16bdc0baf4 [mod] do not escape html content in engines 8 years ago
Adam Tauber 43ddbc60da [fix] pep8 8 years ago
Adam Tauber 16f2e346b3 [fix] bing unicode issue part III. 8 years ago
Adam Tauber 1176505fa4 [fix] bing character encoding - closes #760 8 years ago
Adam Tauber 17b08d096c [fix] unicode search expression for bing 8 years ago
Adam Tauber 16ff8d06c7 [fix] bing paging and language support 
see https://msdn.microsoft.com/en-us/library/ff795620.aspx for bing
specific search operators

closes #755
 8 years ago
Adam Tauber 2f7752b410 [enh] display number of results 8 years ago
Adam Tauber 604f32f672 [fix] bing unicode encode error - fixes #408 9 years ago
Alexandre Flament 4689fe341c update versions.cfg to use the current up-to-date packages 9 years ago
Cqoicebordel 525af2a031 Add bing in the test units 9 years ago
dalf 7c13d630e4 [fix] pep8 : engines (errors E121, E127, E128 and E501 still exist) 10 years ago
Thomas Pointhuber 3d61d9b930 little refactoring 10 years ago
Thomas Pointhuber 4b1e0423a0 update bing engines and fix bing_news 10 years ago
Adam Tauber 66f02dc2b6 [fix] unused globals removed 10 years ago
Gabor Nagy 22fd0746d9 Improves PEP8 compatibility. 10 years ago
asciimoo 2a788c8f29 [enh] search language support init 10 years ago