Commit Graph

4186 Commits (d01741c9a2072025e578169114a68a6712581c7d)
 

Author SHA1 Message Date
Alexandre Flament d01741c9a2
Merge pull request #15 from return42/add-springer
Add a search engine for Springer Nature
3 years ago
Alexandre Flament cdcfd74f39
Merge pull request #14 from return42/add-artic
[enh] add engine using API from "The Art Institute of Chicago"
3 years ago
Markus Heiser 213041adc7 [enh] themes - add result.metadata to result sub header
Signed-off-by: Markus Heiser <markus@darmarit.de>
3 years ago
Pierre Chevalier a80bf1ba97 [enh] Add Springer Nature engine
Springer Nature is a global publisher dedicated to providing service to research
community [1] with official API [2].

To test this PR, first get your API key following this page:

   https://dev.springernature.com/signup

In searx/engines/springer.py at line 24, add this API key.  I left my own key,
commented out in the line aboce.  Feel free to use it, if needed.

[1] https://www.springernature.com/
[2] https://dev.springernature.com/
3 years ago
habsinn 41a2e3785e [enh] add engine using API from "The Art Institute of Chicago" 3 years ago
Alexandre Flament 981cdb7acf
Merge pull request #10 from return42/fix-youtube
[fix] youtube - send CONSENT Cookie to not be redirected
3 years ago
Alexandre Flament e9508cc897
Merge pull request #13 from return42/fix-morty-doc
[fix] Fix typo in morty installation instruction
3 years ago
Alexandre Flament a0e55a587a
Merge pull request #11 from return42/add-doi
[enh] add DOI resolver from sci-hub / replace default DOI
3 years ago
Zackptg5 bed7c3c614 [fix] Fix typo in morty installation instruction 3 years ago
Markus Heiser 9a05ad4c73 [enh] add DOI resolver from sci-hub / replace default DOI
The new sci-hub URLs are comming from @aurora-vasiliev [1].

[1] https://github.com/searx/searx/pull/2706

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser e9a6ab4015 [fix] youtube - send CONSENT Cookie to not be redirected
In the EU there exists a "General Data Protection Regulation" [1] aka GDPR (BTW:
very user friendly!) which requires consent to tracking.  To get the consent
from the user, youtube requests are redirected to confirm and get a CONSENT
Cookie from https://consent.youtube.com

This patch adds a CONSENT Cookie to the youtube request to avoid redirection.

[1] https://en.wikipedia.org/wiki/General_Data_Protection_Regulation

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Reported-by: https://github.com/searx/searx/issues/2774
3 years ago
Alexandre Flament f251a9718e
Merge pull request #9 from return42/drop-https-rewrite
[drop] plugin HTTPS rewrite - rule base is to old (update is to big)
3 years ago
Markus Heiser 247c46c6b0 [drop] plugin HTTPS rewrite - rule base is to old (update is to big)
The usefulness of the _HTTPS rewrite_ plugin is questionable:

- the 36 rule files have not been updated since 2015 [1]
- actual there are 23760 rule files in the https-everywhere repo [2]

For the first, we can remove this plugin.  For a complete new implementation, it
might be good to know that there is a project "https-everywhere : Privacy for
Pythons" [3]

related:  https://github.com/return42/searx-next/issues/8

[1] https://github.com/return42/searx-next/tree/d187a1d/searx/plugins/https_rules
[2] https://github.com/EFForg/https-everywhere/tree/master/src/chrome/content/rules
[3] https://github.com/jayvdb/https-everywhere-py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament c6d5605d27
Merge pull request #7 from searxng/metrics
Metrics
3 years ago
Alexandre Flament baff1cbbab fix issues from review 3 years ago
Alexandre Flament b7848e3422 [fix] searxng fix: sjp engine 3 years ago
Alexandre Flament 7cfd8d900a [mod] oscar: /preferences , engines tab: report engine times
* display the median time instead of the average.
* add a "Reliability" column (sum up the metrics and the checker results).
* the "selected language", "SafeSearch", "Time range" values are displayed as "broken" when the checker tests fail.
3 years ago
Alexandre Flament c27fef1cde [mod] metrics: add secondary parameter
Some error won't stop the engine:
* additional HTTP redirects for example
* some invalid results

secondary=True allows to flag these errors as not important.
3 years ago
Alexandre Flament 7acd7ffc02 [enh] rewrite and enhance metrics 3 years ago
Alexandre Flament aae7830d14 [mod] refactoring: processors
Report to the user suspended engines.

searx.search.processor.abstract:
* manages suspend time (per network).
* reports suspended time to the ResultContainer (method extend_container_if_suspended)
* adds the results to the ResultContainer (method extend_container)
* handles exceptions (method handle_exception)
3 years ago
Alexandre Flament ae5954f2da Merge remote-tracking branch 'return42/fix-preference-save' 3 years ago
Alexandre Flament 624d3ec6db Merge remote-tracking branch 'return42/fix-url-bar-suggestion' 3 years ago
Alexandre Flament 92db0227b1 Merge remote-tracking branch 'dalf/oscar-images' 3 years ago
Alexandre Flament 48720e20a8 Merge remote-tracking branch 'searx/master' 3 years ago
Noémi Ványi 8362257b9a
Merge pull request #2736 from plague-doctor/sjp
Add new engine: SJP - Słownik języka polskiego
3 years ago
Noémi Ványi e56323d3c8
Merge pull request #2759 from ypid/fix/typo
Fix grammar mistake in debug log output
3 years ago
Noémi Ványi 312a51566c
Merge pull request #2764 from mikamp116/patch-1
Fix bug for 'FileNotFoundError' in 'standalone_searx.py'
3 years ago
Noémi Ványi 59df3bec28
Merge pull request #2763 from return42/add-bandcamp
Add Bandcamp search engine
3 years ago
Plague Doctor d275d7a35e Code refactoring. 3 years ago
Markus Heiser f637bfc635 [mod] oscar's "default" template should make use of result.thumbnail
Some engine do have set result.img_src, other return a result.thumbnail.  If
result.img_src is unset and a result.thumbnail is given, show it to the UI.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 062d589f86 [fix] xpath expressions to grap all items from bandcamp's response
I also found some items missing a thumbnail and I used text_extract for content
and title, to remove unneeded whitespaces.

BTW: added bandcamp's favicon

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Kyle Anthony Williams 4d3c399ee9 [feat] add bandcamp engine 3 years ago
Mikayel Mardanyan Petrosyan 4652ef0f06
Update standalone_searx.py
Fix bug for 'FileNotFoundError: [Errno 2] No such file or directory: 'utils/standalone_searx.py' ' in example to run standalone_searx.py from python
3 years ago
Alexandre Flament 01cefffbf6
Merge pull request #1 from metasearch-lab/httpx_networks
Httpx networks
3 years ago
Alexandre Flament d14994dc73 [httpx] replace searx.poolrequests by searx.network
settings.yml:

* outgoing.networks:
   * can contains network definition
   * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
     keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
   * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
   * local_addresses can be "192.168.0.1/24" (it supports IPv6)
   * support_ipv4 & support_ipv6: both True by default
     see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
   * either a full network description
   * either reference an existing network

* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
3 years ago
Markus Heiser c4793afadc [fix] https-scheme missing in preferences-page
This patch is an addition to PR #2656 which removed all usage of `base_url` from
the templates, except one was forgotten in the cookie URL of the preferences.

closes: 2740

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 2bf297b19f [fix] redirect when saving preferences
Erroneously commit 87e4c4762 droped the 302 redirect.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Robin Schneider dfc66ff0f0
Fix grammar mistake in debug log output 3 years ago
Noémi Ványi 6c0114567e
Merge pull request #2744 from ColonisationCaptain/patch-1
Correct typo/grammatical mistake in documentation
3 years ago
Alexandre Flament eaa694fb7d [enh] replace requests by httpx 3 years ago
Alexandre Flament 111180705b [fix] test: avoid HTTP requests
patch engine initialization to skip HTTP request
(engine_init function in searx.engines.initialize_engines)
3 years ago
ColonisationCaptain b2093a8bc0
correct typo/grammatical mistake 3 years ago
Noémi Ványi 5d5ecdb745
Merge pull request #2717 from 3nprob/configure-config-paths
Allow overriding env vars SEARX_SETTINGS_PATH,UWSGI_SETTINGS_PATH
3 years ago
Plague Doctor 599ff39ddf Fix conflicts 3 years ago
Noémi Ványi cc359345a8
Merge pull request #2735 from plague-doctor/wordnik
Add new engine: Wordnik.com
3 years ago
Noémi Ványi a9a51ceb48
Merge pull request #2733 from dalf/fix-2656
SCRIPT_NAME remove trailing slash to avoid infinite redirect
3 years ago
Noémi Ványi 52e08ed777
Merge pull request #2741 from return42/fix-sphinx-theme
[fix] docutils v0.17 incompatibility to previeous v0.16
3 years ago
Markus Heiser 4557d58919 [fix] docutils v0.17 incompatibility to previeous v0.16
With docutils v0.17 a lot of html markup has been changed (see below) what cause
a lot of problems in CSS from Sphinx and other Sphinx extensions & customizing.

For the first this fix pins to previous v0.16. In sphinx 4.0 these problems will
be addressed [2] and we can relax (drop) in the requirements-dev.

HTML5 writer [1]:

  Use the new semantic tags <main>, <section>, <header>, <footer>, <aside>,
  <figure>, and <figcaption>.  See minimal.css and plain.css for styling rule
  examples.

  Change the initial_header_level setting's default to "2", as browsers use the
  same style for <h1> and <h2> when nested in a section.

  Use HTML text-level tags <small>, <s>, <q>, <dfn>, <var>, <samp>, <kbd>, <i>,
  <b>, <u>, <mark>, and <bdi> if a matching class value is found in inline and
  literal elements.  Use <ins> and <del> if a matching class value is found in
  inline, literal, or container elements.

  New optional style responsive.css, adapts to different screen sizes.

  New option embed_images.

[1] https://docutils.sourceforge.io/RELEASE-NOTES.html
[2] https://github.com/sphinx-doc/sphinx/issues/9056

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
3nprob 0fb423ea59 Allow overriding env vars SEARX_SETTINGS_PATH,UWSGI_SETTINGS_PATH 3 years ago
Plague Doctor 6631f11305 Add new engine: SJP 3 years ago