Commit Graph

2767 Commits (f251a9718e16d74714082af12f3edcc47d810d73)

Author SHA1 Message Date
Adam Tauber ae122ea943 [enh] release v1.0.0 4 years ago
Adam Tauber 15f2385a7b [enh] update translations from transifex 4 years ago
Alexandre Flament bb316d2698
Merge pull request #2682 from dalf/fix-checker
[fix] checker: various bug fixes
4 years ago
Alexandre Flament 725a69616b
Merge pull request #2681 from dalf/fix-wikipedia-title
[fix] wikipedia: remove HTML from the title
4 years ago
Noémi Ványi 9bb312c505 Remove duplicated key from dict in Semantic Scholar 4 years ago
Noémi Ványi f596f5767b fix Semantic Scholar engine 4 years ago
Adam Tauber 28286cf3f2 [fix] update seznam engine to be compatible with the new website 4 years ago
Alexandre Flament 0b45afd4d7 [fix] checker: various bug fixes
* initialize engine_data (youtube engine)
* don't crash if an engine don't set result['url']
4 years ago
Alexandre Flament fcfcf662ff [fix] wikipedia: remove HTML from the title
fr.wikipedia.org (and it seems not other wikipedia websites),
adds HTML to api_result['displayTitle'].
(Search for '!wp :fr Braid' for example)

The commit uses api_result['title']
4 years ago
Adam Tauber 06b754ad67 [mod] increase lobste.rs engine timeout to avoid timeouts most of the time 4 years ago
Adam Tauber 0ba71c3644 [fix] make ina engine compatible with the new response json 4 years ago
Adam Tauber 6255b33c9d [fix] rewrite hoogle to use html/xpath instead of json
the json response has been changed and it contains html chunks which is
not compatible with our json engine, so we have to switch to html/xpath
parsing
4 years ago
Adam Tauber 45f0e1a859 [fix] update geektimes.ru url - it redirects to habr.com 4 years ago
Adam Tauber 50ba2b9e87 [fix] update google play movies xpath 4 years ago
Adam Tauber 88657fe9c2 [fix] update google play apps xpath 4 years ago
Adam Tauber 5f450fda74 [enh] add year filter to duckduckgo 4 years ago
Adam Tauber fd737dc9d8 [fix] remove debug code 4 years ago
Alexandre Flament d648001688 [mod] preferences: a tooltip is shown when the mouse is over the engine names 4 years ago
Alexandre Flament 6bd01bf81f [mod] oscar: fix the sourcemap URL in *.min.css
Close https://github.com/searx/searx/issues/2670

Note: clean-css contains a bug:
* a multiline comment or URL adds "$stdin" to the sourcemap (see src/less/logicodev/search.less)
* in this case when the user opens the devtools, the browser fails to load this "https://.../$stdin" URL
* it is not a problem and the error appears only when the user actively tries to debug the CSS.
* seems related to https://github.com/jakubpawlowicz/clean-css/issues/593
4 years ago
Alexandre Flament a48ec0b4bd
Merge pull request #2671 from searx/update-soundcloud
[mod] soundcloud: faster initialization
4 years ago
Alexandre Flament 30c950a2c7
Merge pull request #2660 from dalf/upd-translations
[mod] replace /translations.js with an embedded JSON
4 years ago
Alexandre Flament 38c210d746
[mod] soundcloud: faster initialization
The get_cliend_id() function:
* fetches https://soundcloud.com
* then fetches each referenced javascript URL to get the client id.

This commit fetches the javascript URLs in the reverse order: the client id is in the last javascript URL.
4 years ago
James Higginbotham ce6eb81a71
Update settings.yml to enable HTTP for yacy
Added a line to the yacy entry to enable HTTP if the local yacy instance isn't using HTTPS. Otherwise, an error will be thrown in the logs: "No connection adapters were found for 'http://localhost:8090/yacysearch.json...'". This is likely related to ticket #2641 that forces HTTPS by default.
4 years ago
Alexandre Flament 2b0dd96bd3 [mod] oscar: remove space
* reduce by 15% the uncompressed output (on average)
* dos2unix searx/templates/oscar/result_templates/files.html
4 years ago
Dr. Rolf Jansen 7a9dc63d74
Merge branch 'master' into conditional-sigusr1 4 years ago
Alexandre Flament 6553c79029 [mod] replace /translations.js by embedded JSON
In webapp.py, there is a new function "get_translations" lists available translations

Close #2064
4 years ago
Alexandre Flament 32cd0d31b3 [mod] upgrade pygments
add searx_extra/update/update_pygments.py to update the css style of the oscar and simple themes.
4 years ago
Dr. Rolf Jansen 2a6dbeb6a5
Merge branch 'master' into conditional-sigusr1 4 years ago
Adam Tauber 4c631ac6d0 [fix] remove debug code 4 years ago
Dr. Rolf Jansen 4a27dabcf7
Merge branch 'master' into conditional-sigusr1 4 years ago
Noémi Ványi 8158d8654a fix Microsoft Academic engine 4 years ago
Adam Tauber f97b4ff7b6 [fix] update youtube_noapi paging 4 years ago
Adam Tauber dd34ac396c
Merge pull request #2652 from kvch/solr-engine
Add Apache Solr engine
4 years ago
Alexandre Flament 1664258061
Merge pull request #2655 from return42/fix-imports
[fix] remove unused import from yahoo-news engine
4 years ago
Alexandre Flament 5b176b3496
Merge pull request #2659 from MarcAbonce/onions-http-fix
Fix HTTP error in onion engines
4 years ago
Marc Abonce Seguin f4a0a4d756 fix HTTP error in onion engines
regression from https://github.com/searx/searx/pull/2641
most onion websites only serve HTTP, so it must be enabled
4 years ago
Rolf 80025c3244 Windows does not support SIGUSR1, so don't use it unconditionally. 4 years ago
Markus Heiser 6e1f1085ef [fix] remove unused import from yahoo-news engine
Signed-off-by: Markus Heiser <markus@darmarit.de>
4 years ago
Markus Heiser 3703ebb22a [drop] Acgsou engine - www.acgsou.com no longer exists
- https://www.acgsou.com/ acgsou.com is redirected to 36dm.club
- @rinpatch do not plan on maintaining the engine [1]

[1] https://github.com/searx/searx/pull/1283#issuecomment-798783585

Signed-off-by: Markus Heiser <markus@darmarit.de>
4 years ago
Noémi Ványi ff527e2681 Add Solr engine 4 years ago
Alexandre Flament 9292571304
Merge pull request #2346 from dalf/upgrade-oscar
[mod] oscar: upgrade dependencies
4 years ago
Alexandre Flament 92dd5e245e
Merge pull request #2626 from mikeri/solidtorrents
Add Solid Torrents engine
4 years ago
Alexandre Flament a1a492baed
Merge pull request #2641 from dalf/disable_http_by_default
[mod] by default allow only HTTPS, not HTTP
4 years ago
Alexandre Flament cb04d42806 [mod] oscar: update README.rst 4 years ago
Alexandre Flament 86912e2272 [mod] oscar: get bootstrap and typeahead from NPM 4 years ago
Alexandre Flament 44407353ef [mod] oscar: get leaflet and jquery from NPM
easy to upgrade (package.json)
4 years ago
Alexandre Flament c7133efb12 [mod] oscar: move compiled files to the src directory 4 years ago
Alexandre Flament eda3b513ac [mod] oscar: remove polyfills for Internet Explorer 4 years ago
Alexandre Flament 1268910274 [mod] oscar: remove unused images 4 years ago
Alexandre Flament bdb41bea7b [mod] theme: remove require-2.1.15.min.js
See https://github.com/requirejs/requirejs/issues/1816

requirejs loads one file: leaflet.

This commit:
* removes requirejs
* load leaflet using <script src...> HTML tag in searx/templates/oscar/base.html
4 years ago
Alexandre Flament 2f3d5ec2af [mod] oscar: upgrade npm dependencies 4 years ago
Markus Heiser 96422e5c9f [fix] APKMirror engine - update xpath selectors and fix img_src
BTW: make the code slightly more readable

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser d2faea423a [fix] rewrite Yahoo-News engine
Many things have been changed since last review of this engine.  This patch fix
xpath selectors, implements suggestion and is a complete review / rewrite of the
engine.

Signed-off-by: Markus Heiser <markus@darmarit.de>
4 years ago
Alexandre Flament 99e0651cea [mod] by default allow only HTTPS, not HTTP
Related to https://github.com/searx/searx/pull/2373
4 years ago
Michael Ilsaas 5549d58de3 Add Solid Torrents engine 4 years ago
Adam Tauber 44f4a9d49a [enh] add ability to send engine data to subsequent requests 4 years ago
Alexandre Flament 87f4cc4a9e
Merge pull request #2631 from searx/update_data_update_languages.py
Update searx.data - update_languages.py
4 years ago
Markus Heiser 4845183128 [mod] don't dump traceback of SearxEngineResponseException on init
When initing engines a "SearxEngineResponseException" is logged very verbose,
including full traceback information:

    ERROR:searx.engines:yggtorrent engine: Fail to initialize
    Traceback (most recent call last):
      File "share/searx/searx/engines/__init__.py", line 293, in engine_init
        init_fn(get_engine_from_settings(engine_name))
      File "share/searx/searx/engines/yggtorrent.py", line 42, in init
        resp = http_get(url, allow_redirects=False)
      File "share/searx/searx/poolrequests.py", line 197, in get
        return request('get', url, **kwargs)
      File "share/searx/searx/poolrequests.py", line 190, in request
        raise_for_httperror(response)
      File "share/searx/searx/raise_for_httperror.py", line 60, in raise_for_httperror
        raise_for_captcha(resp)
      File "share/searx/searx/raise_for_httperror.py", line 43, in raise_for_captcha
        raise_for_cloudflare_captcha(resp)
      File "share/searx/searx/raise_for_httperror.py", line 30, in raise_for_cloudflare_captcha
        raise SearxEngineCaptchaException(message='Cloudflare CAPTCHA', suspended_time=3600 * 24 * 15)
    searx.exceptions.SearxEngineCaptchaException: Cloudflare CAPTCHA, suspended_time=1296000

For SearxEngineResponseException this is not needed.  Those types of exceptions
can be a normal use case.  E.g. for CAPTCHA errors like shown in the example
above. It should be enough to log a warning for such issues:

    WARNING:searx.engines:yggtorrent engine: Fail to initialize // Cloudflare CAPTCHA, suspended_time=1296000

closes: #2612

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Alexandre Flament 0165e14a7f
Merge pull request #2632 from searx/update_data_update_wikidata_units.py
Update searx.data - update_wikidata_units.py
4 years ago
Alexandre Flament 152f6fc1da
Merge pull request #2630 from searx/update_data_update_ahmia_blacklist.py
Update searx.data - update_ahmia_blacklist.py
4 years ago
dalf 1e8b846954 Update searx.data - update_currencies.py 4 years ago
dalf 2f8a708481 Update searx.data - update_wikidata_units.py 4 years ago
dalf d9dc3376d0 Update searx.data - update_languages.py 4 years ago
dalf 2857473553 Update searx.data - update_ahmia_blacklist.py 4 years ago
Alexandre Flament aac37f288f
Merge pull request #2593 from dalf/update-autocomplete
Update autocomplete
4 years ago
Alexandre Flament 63f17d2e4c [enh] autocomplete refactoring, autocomplete on external bangs 4 years ago
Markus Heiser d48e2e7b0b [enh] google scholar - python implementation of the engine
The old xpath configuration for google scholar did not work and is replaced by a
python implementation.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Alexandre Flament 4fa1290c11 [fix] answers: don't crash when the query is an empty string 4 years ago
Alexandre Flament e2fb500892
Merge pull request #2608 from return42/unittest2
[py2to3] use unittest from py3, remove unittest2 from py2
4 years ago
Alexandre Flament 0c663e25fc
Merge pull request #2604 from searx/update_data_firefox_version
Update searx.data - firefox_version
4 years ago
Alexandre Flament f77983e174
Merge pull request #2602 from MarcAbonce/fix-bing-fetch-languages
Fix fetch_languages for Bing
4 years ago
GazoilKerozen 5f6ac3afa2
Add Freesound engine (#2596)
Add freesound engine with player.

Co-authored-by: Gazoil <maildeguzel@gmail.com>
4 years ago
Markus Heiser 3bae35940a [py2to3] use unittest from py3, remove unittest2 from py2
- unittest2 is a backport of the new features added to the unittest testing
  framework in Python 2.7

- unittest2 was only needed in py2 and can be dropped now

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Alexandre Flament b05f4d0664
Merge pull request #2605 from searx/update_data_currencies
Update searx.data - currencies
4 years ago
Alexandre Flament aec5188b51
Merge pull request #2606 from searx/update_data_wikidata_units
Update searx.data - wikidata_units
4 years ago
dalf 893b6e4901 Update searx.data - ahmia_blacklist 4 years ago
dalf 7b9005df31 Update searx.data - wikidata_units 4 years ago
dalf 4c8ae5b7ed Update searx.data - firefox_version 4 years ago
dalf d2778b5efe Update searx.data - currencies 4 years ago
Marc Abonce Seguin d6681fd33b remove articles number from engines_languages.json 4 years ago
Marc Abonce Seguin 9b6ffed061 fix fetch_languages for bing
Bing has a list of regions that it supports and some of these regions
may have more than one possible language.

In some cases, like Switzerland, these languages are always shown as
options, so there is no issue. But in other cases, like Andorra, Bing
will only show one language at the time, either the region's default or
the request's language if the latter is supported by that region.

For example, if the HTTP request is in French, Andorra will appear as
fr-AD but if the same page is requested in any other language Andorra
will appear as ca-AD.

This is specially a problem when Bing assumes that the request is in
English because it overrides enough language codes to make several major
languages like Arabic dissappear from the languages.py file.

To avoid that issue, I set the Accept-Language header to a language
that's only supported in one region to hopefully avoid these overrides.
4 years ago
Alexandre Flament 7c1847d5f2 [mod] add utils/fetch_external_bangs.py
Based on duckduckgo bangs
Store bangs on a trie to allow autocomplete (not in this commit)
4 years ago
Alexandre Flament 5f4a085fc4
Merge pull request #2595 from dalf/update-wikidata-units
[mod] update wikidata_units.json and fetch_wikidata_units.py
4 years ago
Alexandre Flament 46ca32c3cc [mod] update currencies.json and fetch_currencies.py
use a sparql request on wikidata to get the list of currencies.

currencies.json contains the translation for all supported searx languages.

Supersede #993
4 years ago
Alexandre Flament 93d1da4906 [mod] update wikidata_units.json and fetch_wikidata_units.py
The fetch_wikidata_units.py result won't change randomly.
See comments in the script.
4 years ago
Noémi Ványi 1be6ab2a91 Fix paging of Bing Images 4 years ago
datagram1 1d0a32a2c5 Added rumble.com video search engine. TODO video embedding.
Update rumble.py

some lines too long.

Disable Rumble engine

disabled : True

PEP8 fix

change line spacing
4 years ago
Alexandre Flament 44a6593c13
Merge pull request #2573 from unixfox/yggtorrent
update yggtorrent url + add it back
4 years ago
Emilien Devos 4b37e10dd9 fix yggtorrent url + add it back 4 years ago
Thorben Günther fbbd4cc21f
Improve peertube searching
At the moment videos without a description are not shown - setting
default content to "" fixes this.
Another current bug is that thumbnails are not displayed. This is caused
by a double slash in the url. For this every trailing slash is now
stripped (for backwards compatibility) and the API response is correctly
parsed.
4 years ago
Alexandre Flament 45027765e3
Merge pull request #2566 from dalf/remove-yandex
[remove] yandex engine
4 years ago
Alexandre Flament c22d4c764c [fix] duckduckgo engine: "!ddg !g" do not redirect to google
* searx understand "!ddg !g time" as : send "!g time" to DDG
* !g a DDG bang for Google: DDG return a HTTP redirect to Google

This commit adds a the allows_redirect param not to follow HTTP redirect.

The DDG engine returns a empty result as before without HTTP redirect.
4 years ago
Alexandre Flament d76660463b
Merge pull request #2562 from dalf/mod-json-engine
[mod] json_engine: add content_html_to_text and title_html_to_text
4 years ago
Alexandre Flament 7dcf67a47a
Merge pull request #2565 from dalf/upd-wikipedia
[upd] wikipedia engine: return an empty result on query with illegal characters
4 years ago
Alexandre Flament 2b60d0d243
Merge pull request #2564 from dalf/fix-seznam
[fix] fix seznam engine
4 years ago
Alexandre Flament 7e83818879
Merge pull request #2560 from dalf/fix-duckduckgo
Fix duckduckgo
4 years ago
Alexandre Flament 63d6ccfbc2
Merge pull request #2557 from dalf/fix-raise_for_httperror
Fix: activate raise_for_error by default
4 years ago
Alexandre Flament 74c8b5606f
Merge pull request #2541 from return42/mediathekviewweb
[enh] add engine MediathekViewWeb (API)
4 years ago
Alexandre Flament 5d9db6c2f7 [remove] yandex engine 4 years ago
Alexandre Flament 35dd069402 [fix] fix seznam engine
no paging support
4 years ago