Commit Graph

374 Commits (640d6ce77877551361b0acd116a9cd7faf47c611)

Author SHA1 Message Date
Emilien Devos 4b37e10dd9 fix yggtorrent url + add it back 3 years ago
Alexandre Flament 45027765e3
Merge pull request #2566 from dalf/remove-yandex
[remove] yandex engine
3 years ago
Alexandre Flament d76660463b
Merge pull request #2562 from dalf/mod-json-engine
[mod] json_engine: add content_html_to_text and title_html_to_text
3 years ago
Alexandre Flament 2b60d0d243
Merge pull request #2564 from dalf/fix-seznam
[fix] fix seznam engine
3 years ago
Alexandre Flament 74c8b5606f
Merge pull request #2541 from return42/mediathekviewweb
[enh] add engine MediathekViewWeb (API)
3 years ago
Alexandre Flament 5d9db6c2f7 [remove] yandex engine 3 years ago
Alexandre Flament 35dd069402 [fix] fix seznam engine
no paging support
3 years ago
Alexandre Flament ff84a1af35 [mod] json_engine: add content_html_to_text and title_html_to_text
Some JSON API returns HTML in either in the HTML or the content.
This commit adds two new parameters to the json_engine:
content_html_to_text and title_html_to_text, False by default.

If True, then the searx.utils.html_to_text removes the HTML tags.

Update crossref, openairedatasets and openairepublications engines
3 years ago
Alexandre Flament 436d366448
Merge pull request #2544 from mrwormo/congresslibrary
[Engine] Add Library of Congress engine
3 years ago
Markus Heiser bc1be3f0e9 [enh] add engine MediathekViewWeb (API)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
mrwormo 051da88328 Add Library of Congress engine 3 years ago
Alexandre Flament 9211cdfe9b [upd] remove google_play_music engine
Google Play Music has been replaced by Youtube music.
3 years ago
Noémi Ványi ac309f5b8d Fix naver engine
Closes #2540
3 years ago
Markus Heiser 41c03cf011 [drop] metager - xpath engine won't work anymore
The new version of MetaGer needs to reload the reults (into a iframe) with a
unique tag (see HTML response below).

Implementing a dedicated metager-engine for searx makes no sense to me. The
great days of MetaGer seems to be ended.  I remember the good old days this
project started in the 90's of the last century.  But in the last few years it
becomes more and more crap.  As the name suggested, MetaGer was made for
germans in the first place.  They have added a english and spain translation but
the i18n is very poor compared to what searx offers.

It's a pity, lets drop MetaGer.

This is the first response, the id (b82679980656899ba5a17ffd02a56846) is unique
for each query:

    $ curl "https://metager.org/meta/meta.ger3?eingabe=foo&submit-query=&focus=web"
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <link rel="stylesheet" href="/index.css?id=b82679980656899ba5a17ffd02a56846">
        <script src="/index.js?id=b82679980656899ba5a17ffd02a56846"></script>
    <title>foo - MetaGer</title>
    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1" />
    </head>
    <body>
        <iframe id="mg-framed" src="https://metager.org/meta/meta.ger3?eingabe=foo&amp;submit-query=&amp;focus=web&amp;mgv=b82679980656899ba5a17ffd02a56846" autofocus="true" onload="this.contentWindow.focus();"></iframe>
     </body>

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
mrwormo c4c1636b18 Add Creative Commons search engine 3 years ago
Markus Heiser e436287385 [mod] checker: add some additional tests
BTW: fix indentation by 2 spaces

The additional tests has been commented out in the google engines to not release
any CAPTCHA issues.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament 73c86f9bf2 [mod] checker: disable by default 3 years ago
Alexandre Flament 80d7411f2c
Merge pull request #2452 from kvch/add-wilby-engine
Add wiby.me engine
3 years ago
Alexandre Flament b405646749
Merge pull request #2451 from mrwormo/invidious-engine
[Fix] Invidious Engine
3 years ago
Alexandre Flament a4dcfa025c [enh] engines: add about variable
move meta information from comment to the about variable
so the preferences, the documentation can show these information
3 years ago
mrwormo 2dff3887f0 [fix] Invidious engine by enabling requests by randomly picking amongst working instances 3 years ago
Alexandre Flament 3a9f513521 [enh] checker: background check
See settings.yml for the options
SIGUSR1 signal starts the checker.
The result is available at /stats/checker
3 years ago
Alexandre Flament ca0889d488 [enh] checker: wikidata & ddd: add specific tests 3 years ago
Alexandre Flament 16a889dd8f [enh] checker: add rosebud test 3 years ago
Alexandre Flament 424e6abc7e [mod] settings.yml: move brand settings to a dedicated section 3 years ago
Markus Heiser 9485179064 [mod] move brand options from Makefile to settings.yml
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Noémi Ványi a6dd1de4a8 Add wiby.me engine
Closes #2339
3 years ago
Markus Heiser b0bb0a3a0f [fix] Library Genesis links shifted by 1 #1998
Fixes: #1998
Suggested-by: @linuxmue
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Émilien Devos fc6cfc3b58
Remove voat due to its shutdown
Voat shutted down on December 25th, 2020 at 12 noon PST: https://voat.co/host/voat/static/inactive.min.html?ReturnUrl=/
3 years ago
Alexandre Flament 36600118fb
Merge pull request #2372 from dalf/remove-broken-engines
[remove] remove searchcode_doc and twitter
4 years ago
Alexandre Flament d703119d3a [enh] add raise_for_httperror
check HTTP response:
* detect some comme CAPTCHA challenge (no solving). In this case the engine is suspended for long a time.
* otherwise raise HTTPError as before

the check is done in poolrequests.py (was before in search.py).

update qwant, wikipedia, wikidata to use raise_for_httperror instead of raise_for_status
4 years ago
Alexandre Flament 1c9e7cef50 [remove] remove searchcode_doc and twitter
* twitter: the API has changed. the engine needs to rewritten.
* searchcode_doc: the API about documentation doesn't exist anymore.
4 years ago
Alexandre Flament 42a194898b
Merge pull request #2360 from dalf/update-libgen
[mod] libgen: update the URL to http://libgen.rs/
4 years ago
Alexandre Flament bd4869ecd0
Merge pull request #2366 from dalf/remove-seedpeer
[remove] seedpeer engine
4 years ago
Alexandre Flament 56c64d6b64 [remove] seedpeer engine
the website is offline.
4 years ago
Alexandre Flament 4ec810749b [fix] 1x engine 4 years ago
Alexandre Flament b3032cf953 [mod] libgen: update the URL to http://libgen.rs/
https://libgen.is actually redirect to http://libgen.rs/

It seems there is no HTTPS version:
* https://www.wikidata.org/wiki/Q22017206
* https://librarygenesis.net/
4 years ago
mrwormo 2b153db74c Disable Invidious engine by default 4 years ago
Noémi Ványi 4a36a3044d
Add recoll engine (#2325)
recoll is a local search engine based on Xapian:
http://www.lesbonscomptes.com/recoll/

By itself recoll does not offer web or API access,
this can be achieved using recoll-webui:
https://framagit.org/medoc92/recollwebui.git

This engine uses a custom 'files' result template

set `base_url` to the location where recoll-webui can be reached
set `dl_prefix` to a location where the file hierarchy as indexed by recoll can be reached
set `search_dir` to the part of the indexed file hierarchy to be searched, use an empty string to search the entire search domain
4 years ago
Alexandre Flament 3786920df9 [enh] Add multiple outgoing proxies
credits go to @bauruine see https://github.com/searx/searx/pull/1958
4 years ago
Noémi Ványi 43e697681e New engine: Elasticsearch 4 years ago
Adam Tauber 063260d090 [enh] add default http headers - closes #715 4 years ago
a01200356 c3daa08537 [enh] Add onions category with Ahmia, Not Evil and Torch
Xpath engine and results template changed to account for the fact that
archive.org doesn't cache .onions, though some onion engines migth have
their own cache.

Disabled by default. Can be enabled by setting the SOCKS proxies to
wherever Tor is listening and setting using_tor_proxy as True.

Requires Tor and updating packages.

To avoid manually adding the timeout on each engine, you can set
extra_proxy_timeout to account for Tor's (or whatever proxy used) extra
time.
4 years ago
Noémi Ványi 33e139cae6 Let admins lock user preferences 4 years ago
Nicholas Kegler 8e15d3e4c1 Open Semantic Search Engine 4 years ago
Adam Tauber a05c660e30 [enh] add ability to set enabled plugins from settings - closes #1613 #778 4 years ago
Noémi Ványi ce000a9fef Fix XPATH of lobste.rs engine && add timeout 4 years ago
Noémi Ványi f0278d41fc add ebay enginte to shopping category 4 years ago
Finn 53c8d945b4
[enh] Add SepiaSearch engine (#2227)
supported_languages values: see https://framagit.org/framasoft/peertube/search-index/-/blob/master/client/src/views/Search.vue#L618-641
4 years ago
Alexandre Flament f204e4903d [fix] migration from github.com/asciimoo/searx to github.com/searx/searx : fix URLs 4 years ago