blob42
/
searxng
forked from Archives/searxng
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,670 Commits
10 Branches
17 Tags
41 MiB
whitelist-ratelimiter
mine
gh-pages
master
dependabot/pip/master/flask-babel-3.0.0
translations
dependabot/pip/master/mock-5.0.1
dependabot/pip/master/yamllint-1.29.0
dependabot/pip/master/pylint-2.15.10
dependabot/pip/master/sphinx-6.1.3
v1.0.0
v0.18.0
v0.17.0
v0.16.0
v0.15.0
v0.14.0
v0.13.1
v0.13.0
v0.12.0
v0.11.0
v0.10.0
v0.9.0
v0.8.1
v0.8.0
v0.7.0
v0.6.0
1.0.0
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'ba8959ad7c'
${ noResults }
Commit Graph

55 Commits (ba8959ad7c18ce4165d29b7a472d845bd96f4735)

Author SHA1 Message Date
Markus Heiser ba8959ad7c [fix] typos / reported by @kianmeng in searx PR-3366 
[PR-3366] https://github.com/searx/searx/pull/3366

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Alexandre Flament cd2dd5dd55 Wikidata engine: ignore dummy entities 
Close #641
 2 years ago
Alexandre Flament d068b67a71 Wikidata engine: minor change of the SPARQL request 
The engine can be slow especially when the query won't return any answer.
See https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual/MWAPI#Find_articles_in_Wikipedia_speaking_about_cheese_and_see_which_Wikibase_items_they_correspond_to

Related to #1290
 2 years ago
Markus Heiser 2de007138c [fix] prepare for pylint 2.14.0 
Remove issue reported by Pylint 2.14.0:

- no-self-use: has been moved to optional extension [1]
- The refactoring checker now also raises 'consider-using-generator' messages
  for max(), min() and sum(). [2]

.pylintrc:
  - <option name>-hint has been removed since long, Pylint 2.14.0 raises an
    error on invalid options
  - bad-continuation and bad-whitespace have been removed [3]

[1] https://pylint.pycqa.org/en/latest/whatsnew/2/2.14/summary.html#removed-checkers
[2] https://pylint.pycqa.org/en/latest/whatsnew/2/2.14/full.html#what-s-new-in-pylint-2-14-0
[2] https://pylint.pycqa.org/en/latest/whatsnew/2/2.6/summary.html#summary-release-highlights

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Markus Heiser a967e59590 [pylint] searx/engines/wikidata.py (no functional change) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Léon Tiekötter 1c151ae92b
[fix] wikidata: URL decoding and file extension handling 
Add '.png' to the second img_src_name if it has the extension '.svg'.
Use urllib.parse.unquote for URL decoding.
 2 years ago
Markus Heiser a13c5d70c7 [fix] wikidata engine: select image with higher (not lower) priority 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Léon Tiekötter a50f32bcfc
wikidata: load thumbnail instead of full image 2 years ago
Léon Tiekötter 560a14e77b
[fix] wikidata info box images 
Wikidata info box images are now loaded from uploads.wikimedia.org instead of commons.wikimedia.org to prevent redirects

Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Markus Heiser d84226bf63 [fix] issues reported by pylint 
Fix pylint issues from commit (3d96a983)

    [format.python] initial formatting of the python code

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Markus Heiser 3d96a9839a [format.python] initial formatting of the python code 
This patch was generated by black [1]::

    make format.python

[1] https://github.com/psf/black

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Markus Heiser fcdc2c2cd2 [format.python] disable py code formatting for some hunks of code 
Disable the python code formatting from python-black, where the readability of
code suffers by formatting.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2 years ago
Markus Heiser aecfb2300d [mod] one logger per engine - drop obsolete logger.getChild 
Remove the no longer needed `logger = logger.getChild(...)` from engines.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 3 years ago
Alexandre Flament d14994dc73 [httpx] replace searx.poolrequests by searx.network 
settings.yml:

* outgoing.networks:
   * can contains network definition
   * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
     keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
   * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
   * local_addresses can be "192.168.0.1/24" (it supports IPv6)
   * support_ipv4 & support_ipv6: both True by default
     see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
   * either a full network description
   * either reference an existing network

* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
 3 years ago
Alexandre Flament a4dcfa025c [enh] engines: add about variable 
move meta information from comment to the about variable
so the preferences, the documentation can show these information
 3 years ago
Alexandre Flament d703119d3a [enh] add raise_for_httperror 
check HTTP response:
* detect some comme CAPTCHA challenge (no solving). In this case the engine is suspended for long a time.
* otherwise raise HTTPError as before

the check is done in poolrequests.py (was before in search.py).

update qwant, wikipedia, wikidata to use raise_for_httperror instead of raise_for_status
 3 years ago
Alexandre Flament bef2f2efa8 [fix] wikidata: fix crash when the item has no description at all and at least one URL. 3 years ago
Alexandre Flament 9ed3ee2beb [mod] wikidata: WDGeoAttribute class: doesn't change the method signature of get_str 3 years ago
Alexandre Flament 3038052c79 [mod] remove unused import 
use
from searx.engines.duckduckgo import _fetch_supported_languages, supported_languages_url  # NOQA
so it is possible to easily remove all unused import using autoflake:
autoflake --in-place --recursive --remove-all-unused-imports searx tests
 4 years ago
Alexandre Flament 95bd6033fa [mod] wikidata engine: use one SPARQL request instead of 2 HTTP requests. 4 years ago
Alexandre Flament 2006eb4680 [mod] move extract_text, extract_url to searx.utils 4 years ago
Dalf 1022228d95 Drop Python 2 (1/n): remove unicode string and url_utils 4 years ago
Marc Abonce Seguin 0d8970c8f2
only return one url per "type" in Wikidata (#2151) 
i.e. only one official website, one Twitter, etc.
 4 years ago
Adam Tauber 29960aa1d9 [enh] add official site link to the top of the infobox - closes #1644 4 years ago
Dalf 85b3723345 [mod] speed optimization 
compile XPath only once
avoid redundant call to urlparse
get_locale(webapp.py): avoid useless call to request.accept_languages.best_match
 5 years ago
Dalf 6e0285b2db [fix] wikidata engine: faster processing, remove one HTTP redirection. 
* Search URL is https://www.wikidata.org/w/index.php?{query}&ns0=1 (with ns0=1 at the end to avoid an HTTP redirection)
* url_detail: remove the disabletidy=1 deprecated parameter
* Add eval_xpath function: compile once for all xpath.
* Add get_id_cache: retrieve all HTML with an id, avoid the slow to procress dynamic xpath '//div[@id="{propertyid}"]'.replace('{propertyid}')
* Create an etree.HTMLParser() instead of using the global one (see #1575)
 5 years ago
Noémi Ványi b63d645a52 Revert "remove 'all' option from search languages" 
This reverts commit 4d1770398a.
 5 years ago
Marc Abonce Seguin 5568f24d6c [fix] check language aliases when setting search language 5 years ago
Léo Bourrel 7a474db61b Fix formatting 6 years ago
Léo Bourrel acaef6600e Update path to wikidata image 6 years ago
Marc Abonce Seguin b12857a70d [fix] make search requests on wikidata more accurate 6 years ago
Marc Abonce Seguin 772c048d01 refactor engine's search language handling 
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.

Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
 6 years ago
marc 4d1770398a remove 'all' option from search languages 7 years ago
Adam Tauber 52e615dede [enh] py3 compatibility 7 years ago
marc af35eee10b tests for _fetch_supported_languages in engines 
and refactor method to make it testable without making requests
 8 years ago
marc f62ce21f50 [mod] fetch supported languages for several engines 
utils/fetch_languages.py gets languages supported by each engine and
generates engines_languages.json with each engine's supported language.
 8 years ago
marc 149802c569 [enh] add supported_languages on engines and auto-generate languages.py 8 years ago
marc ad58b14be7 [fix] merge infoboxes based on weight 
also minor changes in attributes and images from wikidata
 8 years ago
marc a0a1284998 wikidata refactor and more attributes (see issue #560) 8 years ago
a01200356 93ef11adc0 [enh] multilingual wikidata 
disambiguation and tags are in local language

TOFIX:
    needs to query the api every time to know each label's name
 8 years ago
a01200356 8d335dbdae [enh] wikipedia infobox 
creates simple multilingual infobox using wikipedia's api
 8 years ago
Adam Tauber bd22e9a336 [fix] pep8 compatibilty 8 years ago
Adam Tauber 362c849797 [fix][mod] wikidata date handling refactor - fixes #387 9 years ago
dalf d07cfd9089 [enh] use one single http connection pool : improve response time. close #100 9 years ago
Adam Tauber d7ea44ab8d [fix] dates before 1900 9 years ago
Adam Tauber cc4e17b668 [fix] pep8 9 years ago
Cqoicebordel 7937218be6 Use human readable date 
For DoB and DoD, wikipedia use a non standard ISO format, not easily readable.
Now the date is displayed in an human readable form, using the language setting as locale if available. If not, it uses the default locale.
 10 years ago
dalf ffcec383b7 [fix] pep8 : duckduckgo_definitions and wikidata engines 10 years ago
Adam Tauber b0bb94fd37 [fix] wikidata: using only the first url 10 years ago
dalf cac1761a54 [enh] infoboxes : if the result doesn't contain anything except one link, use the normal result template 10 years ago