Commit Graph

3361 Commits (bd285de48a7db9f67c9e5dd50aba8742f552a245)

Author SHA1 Message Date
dependabot[bot] 94de9ee61a
Bump grunt-contrib-jshint in /searx/static/themes/oscar
Bumps [grunt-contrib-jshint](https://github.com/gruntjs/grunt-contrib-jshint) from 3.0.0 to 3.1.1.
- [Release notes](https://github.com/gruntjs/grunt-contrib-jshint/releases)
- [Changelog](https://github.com/gruntjs/grunt-contrib-jshint/blob/main/CHANGELOG)
- [Commits](https://github.com/gruntjs/grunt-contrib-jshint/commits/v3.1.1)

---
updated-dependencies:
- dependency-name: grunt-contrib-jshint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
3 years ago
Alexandre Flament 6b643e961b
Merge pull request #397 from searxng/dependabot/npm_and_yarn/searx/static/themes/simple/master/grunt-contrib-jshint-3.1.1
Bump grunt-contrib-jshint from 3.0.0 to 3.1.1 in /searx/static/themes/simple
3 years ago
searxng-bot 91206a5682 [translations] update
b2709671 - 2021-10-04 - Markus Heiser <markus.heiser@darmarit.de>
3 years ago
dependabot[bot] a88ee63aca
Bump grunt-contrib-jshint in /searx/static/themes/simple
Bumps [grunt-contrib-jshint](https://github.com/gruntjs/grunt-contrib-jshint) from 3.0.0 to 3.1.1.
- [Release notes](https://github.com/gruntjs/grunt-contrib-jshint/releases)
- [Changelog](https://github.com/gruntjs/grunt-contrib-jshint/blob/main/CHANGELOG)
- [Commits](https://github.com/gruntjs/grunt-contrib-jshint/commits/v3.1.1)

---
updated-dependencies:
- dependency-name: grunt-contrib-jshint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
3 years ago
Markus Heiser 5731b6b700 [mod] searx.plugins.prepare_package_resources() - use generators
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser aa5a5147b2 [fix] searx.plugins.initialize() - don't miss module & module-name
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 3abbe6d25b [fix] engine torznab - categories, before join convert int to str
BTW add init() function and replace SearxEngineAPIException by ValueError.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 9fb77065bd [fix] engine torznab - marginal issues reported from linters
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Paolo Basso d803df8d89 [mod] engines - add torznab WebAPI 3 years ago
Alexandre Flament 7246d62770
Merge pull request #386 from return42/fix-384
[mod] set 'engine.supported_languages' from the origin python module
3 years ago
Markus Heiser 19e41c137e [mod] set 'engine.supported_languages' from the origin python module
The key of the dictionary 'searx.data.ENGINES_LANGUAGES' is the *engine name*
configured in settings.xml.  When multiple engines are configured to use the
same origin engine (e.g. `engine: google`)::

    - name: google
      engine: google
      use_mobile_ui: false
      ...

    - name: google italian
      engine: google
      use_mobile_ui: false
      language: it
      ...

    - name: google mobile ui
      engine: google
      shortcut: gomui
      use_mobile_ui: true

There exists no entry for ENGINES_LANGUAGES[engine.name] (e.g. `name: google
mobile ui` or `name: google italian`).  This issue can be solved by recreate the
ENGINES_LANGUAGES::

    make data.languages

But this is nothing an SearXNG admin would like to do when just configuring
additional engines, since this just doubles entries in ENGINES_LANGUAGES and
BTW: `make data.languages` has various external requirements which might be not
installed or not available, on a production host.

With this patch, if engine.name fails, ENGINES_LANGUAGES[engine.engine] is used
to get the engine.supported_languages (e.g. `google` for the engine named
`google mobile`).

For an engine, when there is `language: ...` in the YAML settings, the engine
supports only one language, in this case engine.supported_languages should
contains this value defined in settings.yml (e.g. `it` for the engine named
`google italian`).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Closes: https://github.com/searxng/searxng/issues/384
3 years ago
Alexandre Flament 2b4fef7118 plugins: refactor initialization
add a new function "init" call when the app starts.
The function can:
* return False to disable the plugin.
* modify the Flask app.
3 years ago
Markus Heiser de0249ddae [fix] don't mix loaded modules with imported modules (sys.modules)
The utils.load_module() function is used to load a python file (aka module) and
return the module's namespace.  SearXNG uses this function to load *engines and
answerers* from arbitrary locations with arbitrary modifications.  These are not
real python modules and it is not intended to mix this *engines and answerers*
with the python modules registered in sys.modules.

Closes: https://github.com/searxng/searxng/issues/312
Suggested-by: @dalf in https://github.com/searxng/searxng/issues/312
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser feb2d4dda5
Merge pull request #377 from return42/engine-imdb
[mod] engines - add IMDB / Internet Movie Database
3 years ago
Alexandre Flament 8a897b86f1 [mod] engines - IMDB: add thumbnails 3 years ago
Alexandre Flament 47eb836c65
Merge pull request #375 from dalf/searxng_extra
SearXNG: searx_extra
3 years ago
Alexandre Flament 715c445e9b [build] /static 3 years ago
Paul Alcock 823d44ed0a [mod] engines - add IMDB / Internet Movie Database
Merged from @Guilvareux's commit [1] and slightly modfied / see [2].

[1] https://github.com/searx/searx/pull/2980/commits/f2f90071
[2] https://github.com/searx/searx/pull/2980
3 years ago
Alexandre Flament 1bb82a6b54 SearXNG: searxng_extra 3 years ago
Alexandre Flament 430babca25 SearXNG: environment variables 3 years ago
Alexandre Flament e39a03cc61
Merge pull request #367 from dalf/searxng-github
SearXNG: .github, searxng-docker
3 years ago
Markus Heiser fda5609a17
Merge pull request #360 from return42/ddg-lite
Implement a DuckDuckGo Lite engine
3 years ago
Markus Heiser 97419a770d [upd] make data.languages
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser a5b7ed9550 [mod] engine duckduckgo - update supported_languages_url
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 4c9b8b29ee [mod] engine duckduckgo - use DuckDuckGo-Lite
Implement a scrapper for DuckDuckGo-Lite [1].  The existing DuckDuckGo [2]
engine does not support paging.  DuckDuckgo-Lite is much faster, less verbose
and does have a paging option (reversed engineered from the input form of [1]).

[1] https://lite.duckduckgo.com/lite
[2] https://duckduckgo.com/

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament a53a4d5012 [build] /static 3 years ago
Alexandre Flament 74b0830362 SearXNG: simple theme 3 years ago
Alexandre Flament 499c7b3cce SearXNG: .github, searxng-docker 3 years ago
Alexandre Flament 458575c221
Merge pull request #361 from dalf/searxng-theme-oscar
SearXNG theme oscar
3 years ago
Markus Heiser c74f45b1df
Merge pull request #365 from searxng/update_data_update_wikidata_units.py
Update searx.data - update_wikidata_units.py
3 years ago
Markus Heiser 62ea3c7074
Merge pull request #366 from searxng/update_data_update_languages.py
Update searx.data - update_languages.py
3 years ago
Markus Heiser d814f1182f
Merge pull request #364 from searxng/update_data_update_firefox_version.py
Update searx.data - update_firefox_version.py
3 years ago
Markus Heiser 8479a83411
Merge pull request #363 from searxng/update_data_update_ahmia_blacklist.py
Update searx.data - update_ahmia_blacklist.py
3 years ago
dalf db51a64390 Update searx.data - update_languages.py 3 years ago
dalf a582e55af0 Update searx.data - update_wikidata_units.py 3 years ago
dalf 56c68beb42 Update searx.data - update_firefox_version.py 3 years ago
dalf 453114fbe4 Update searx.data - update_ahmia_blacklist.py 3 years ago
dalf 0c24a03f05 Update searx.data - update_currencies.py 3 years ago
Alexandre Flament 1dfac73a64 [build] /static 3 years ago
Alexandre Flament df06dddc04 SearXNG: oscar theme 3 years ago
Alexandre Flament a582cf3d82
Merge pull request #353 from return42/stackexchange
[mod] engines - add Stack Exchange API v2.3
3 years ago
Markus Heiser ecb3912bd0 [fix] engine stackexchange - decode HTML entities in title & content
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament a9c3c88cc0 [mod] searx.network.stream returns a tuple (response, stream) 3 years ago
Alexandre Flament 29893cf816 [fix] searx.network.stream: fix memory leak 3 years ago
Markus Heiser 5efe77bdf5 [mod] engines - add superuser.com (Stack Exchange API)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 29eb06ab16 [mod] engines - add askubuntu.com (Stack Exchange API)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser b62851559b [mod] replace old stackoverflow engine by Stack Exchange API v2.3
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 55fee1e45d [mod] engines - add Stack Exchange API v2.3
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament 2eab89b4ca [fix] checker: fix memory usage
* download images using the "image_proxy" network (HTTP/1 instead of HTTP/2)
* don't cache data: URL (reduce memory usage)
* after each test: purge image URL cache then call garbage collector
* download only the first 64kb of images
3 years ago
MrPaulBlack 43fc136207 [build] /static 3 years ago
MrPaulBlack 57f58f85bd [fix] position of engine description tooltip 3 years ago
Markus Heiser 73cb80e71d [build] /static 3 years ago
Markus Heiser 8ac776765c [theme] simple: do not prefer arial over user's sans-serif font
The default *sans-serif* font from the browsers most often renders much better
compared to Arial font.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
MrPaulBlack e8ed56251a [build] /static 3 years ago
MrPaulBlack f81afed60b [fix] backtotop and pagination on image result page 3 years ago
MrPaulBlack 8268873701 [fix] several issues in the simple theme
- using more rem in style and definitions
- mobile width in preferences.less fix max-width: 75em to 80em (normalized with
  style.less and other)
- do not display #backToTop position on tablet (when max-width: 80em)
- fix answer box on mobile (when max-width: 50em)
3 years ago
MrPaulBlack b6ae1f1c7a [theme] margin around checkboxes is bigger now, index centers the search more and the pref, stats and about page have a max-width now 3 years ago
MrPaulBlack 859179f0c9 [theme] optimize switching between different device types 3 years ago
MrPaulBlack 385dce213b [theme] cleanup grid layout and remove various margins and paddings from elements 3 years ago
Alexandre Flament cc3c54f4cd simple theme: update 3 years ago
MrPaulBlack 88c17d106d [fix] make selected tabs not change wifth anymore compared to not being selected 3 years ago
Alexandre Flament b046322c7b
Merge pull request #333 from dalf/enh-engine-descriptions
RFC: /preferences: display engine descriptions
3 years ago
Alexandre Flament 0c3314b338
Merge pull request #344 from dalf/osm-lang-all-fix
[fix] openstreetmap engine: map "all" language to English
3 years ago
Alexandre Flament ab569c1e12 [fix] openstreetmap engine: optmizer SPARQL query
add
hint:Query hint:optimizer "None".
to the SPARQL query to keep the response time small.

It tells the optimizer to follow the path from ?item to the different property values
instead of the other way around.
See https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/query_optimization#Property_paths
3 years ago
Alexandre Flament 39876d9f14 [build] /static 3 years ago
Alexandre Flament 2f1384f198 [enh] themes: display the engine descriptions 3 years ago
Alexandre Flament bfd24d1226 [mod] add /engine_descriptions.json endpoint
returns engine descriptions (JSON):
* key: engine name
* value: description in the user locale, use English description as a fallback
3 years ago
Alexandre Flament 8961131497 [fix] fix the about section of some engines 3 years ago
Alexandre Flament 6f11b61cd5 [fix] openstreetmap engine: map "all" language to English 3 years ago
MrPaulBlack 3017227c14 [preferences] remove checkbox class from query section 3 years ago
MrPaulBlack 30a9146bdc [build] /static 3 years ago
MrPaulBlack 7cb5099c88 [preferences] make elements in query selecter align left 3 years ago
MrPaulBlack d24bcbb6f0 [preferences] reorder tabs in simple to match oscar theme 3 years ago
MrPaulBlack 45eb2b5637 [preferences] cleanup whitespaces 3 years ago
MrPaulBlack 2d26ace265 [preferences] add special queries tab to preferences 3 years ago
MrPaulBlack fa95cd91a8 [build] /static 3 years ago
MrPaulBlack 79351c2e4d [fix] searx.js null pointer exception when category div is missing 3 years ago
Alexandre Flament 12c90a1b47
Merge pull request #336 from MrPaulBlack/simple-search-form
[theme] remove categories, language and time range from index.html
3 years ago
MrPaulBlack 575483ad2a [theme] add newline at the end of the file 3 years ago
MrPaulBlack 880fe67172 [theme] remove categories, language and time range from index 3 years ago
Alexandre Flament e8e81c7432 [build] /static 3 years ago
Alexandre Flament 2cc6301c98 [fix] Oscar theme: fix input field on Safari
close #323
3 years ago
searxng-bot 1bc50c1eb4 [translations] update
3834942d - 2021-09-15 - ButterflyOfFire <butterflyoffire@protonmail.com>
3 years ago
Alexandre Flament dc74df3a55
Merge pull request #261 from dalf/upgrade_httpx
[upd] upgrade httpx 0.19.0
3 years ago
Markus Heiser 443bf35e09 [pylint] fix global-variable-not-assigned issues
If there is no write access, there is no need for global.  Remove global
statement if there is no assignment.

global-variable-not-assigned:
  Using global for names but no assignment is done Used when a variable is
  defined through the "global" statement but no assignment to this variable is
  done.

In Pylint 2.11 the global-variable-not-assigned checker now catches global
variables that are never reassigned in a local scope and catches (reassigned)
functions [1][2]

[1] https://pylint.pycqa.org/en/latest/whatsnew/2.11.html
[2] https://github.com/PyCQA/pylint/issues/1375

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament 41f6359d06 [fix] error recorder: avoid RuntimeError on some rare occasion
httpx.RequestError (subclass of httpx.HTTPError) has a property request.
This property raises a RuntimeError if the attributes _request is None.
To avoid a cascade of errors, this commit reads directly the _request attribute.
3 years ago
Alexandre Flament b10403d3a1 [mod] searx.network: remove redundant code
searx.client.new_client: the proxies parameter is a dictonnary,
and the protocol (key of the dictionnary) is already normalized
(see usage of searx.network.network.PROXY_PATTERN_MAPPING)
3 years ago
Alexandre Flament 8e73438cbe [upd] upgrade httpx 0.19.0
adjust searx.network module to the new internal API
see https://github.com/encode/httpx/pull/1522
3 years ago
Alexandre Flament 602cbc2c99
Merge pull request #297 from dalf/engine-logger-enh
debug mode: more readable logging
3 years ago
Markus Heiser de5a8ee7d7 [fix] settings - displayed name is SearXNG
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 97acedd9fe
Merge pull request #305 from dalf/fix-engine-logger
[fix] logger per engine: make .logger is always initialized
3 years ago
Alexandre Flament f8793fbda0 [fix] logger per engine: make .logger is always initialized
the openstreetmap engine imports code from the wikidata engine.
before this commit, specific code make sure to copy the logger variable to the wikidata engine.

with this commit searx.engines.load_engine makes sure the .logger is initialized.
The implementation scans sys.modules for module name starting with searx.engines.
3 years ago
MrPaulBlack 644ccf16be [theme] make engine stats clickable 3 years ago
MrPaulBlack 48d4296caa [theme] link to engine stats in footer 3 years ago
Markus Heiser ecd6ca86d0
Merge pull request #308 from dalf/plugin_hostname_replace
[enh] add hostname_replace plugin
3 years ago
Alexandre Flament 0f43b39eac [enh] add hostname_replace plugin
* backport of https://github.com/searx/searx/pull/2724
* allow to remove result if the replacement is the boolean value false
3 years ago
Alexandre Flament 2b53d718e4 [fix] PR #257: use the image_proxy network instead of the default network 3 years ago
Alexandre Flament 6abc37efc7 [mod] production mode: make sure to set the log level to WARNING
set:
* the root logger level to WARNING
* the werkzeug logger level to WARNING (useful when there is no uwsgi)
3 years ago
Alexandre Flament 0e42db9da1 [mod] xpath engine: remove logging of the requested URL 3 years ago
Alexandre Flament 91a6d80e82 [mod] debug mode: log HTTP requests with network name
For example wikipedia requests use the logger name "searx.network.wikipedia"

Log is disable when searx_debug is False
3 years ago
Alexandre Flament 577998106a [mod] debug mode: use coloredlogs package
in debug mode, add some padding to make the output more readable
in procution mode, add the timestamp
3 years ago
Alexandre Flament b513917ef9 [mod] searx.metrics & searx.search: use the engine loggers
metrics & processors use the engine logger
3 years ago
Alexandre Flament d9658a7f91 [fix] searx/results.py: strip result['content'] only if it exists
fix PR #302
3 years ago
Markus Heiser b671e0364f
Merge pull request #302 from dalf/mod_plugin_on_result
[mod] plugin: call on_result for each result of each engines.
3 years ago
Alexandre Flament 0b27c8698f [doc] update docs/dev/plugins.rst 3 years ago
Alexandre Flament 70c9bb6f2c [fix] remove references to transifex 3 years ago
Alexandre Flament b941763e20 [mod] ahmia_filter: use on_result instead of post_search
see commit 6c9ae7911e9639bc46cd53af215734b4bdb61ba9
3 years ago
Alexandre Flament 660c180170 [mod] plugin: call on_result after each engine from the ResultContainer
Currently, searx.search.Search calls on_result once the engine results have been merged
(ResultContainer.order_results).

on_result plugins can rewrite the results: once the URL(s) are modified, even they can be merged,
it won't be the case since ResultContainer.order_results has already be called.

This commit call on_result inside for each result of each engines.
In addition the on_result function can return False to remove the result.

Note: the on_result function now run on the engine thread instead of the Flask thread.
3 years ago
Alexandre Flament fc20c561bf [mod] oa_doi_rewrite plugin: get_doi_resolver: remove args parameter
doi_resolvers.get_value('preferences') already contains the value from
request.args.get('doi_resolver')
3 years ago
Alexandre Flament a3789b3bb4
Merge pull request #295 from return42/fix-csp-2
[fix] make HTML templates more CSP compliant
3 years ago
Markus Heiser b0623ac9f6 [build] /static 3 years ago
Markus Heiser 0240ca9567 [fix] oscar template - merge the two class attributes into one
Suggested-by: @dalf https://github.com/searxng/searxng/pull/295#discussion_r703337237
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser bd830f7449 [fix] simple theme - preferences: fix tooltip overflow
Suggested-by: @dalf https://github.com/searxng/searxng/pull/295#discussion_r703370358
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 2a3b9a2e26 [pylint] searx: drop no longer needed 'missing-function-docstring'
Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914168470
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser f0059b80ed [pylint] engines: drop no longer needed 'missing-function-docstring'
Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914168470
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 9068a72c6a [fix] oscar theme - preferences: rename col-checkbox/col-stat
[1] https://github.com/searxng/searxng/pull/295#discussion_r703318053
[2] https://github.com/searxng/searxng/pull/295#discussion_r703337237

Suggested-by: @dalf [1] [2]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 82847df300 [fix] add 'categories' to PYLINT_ADDITIONAL_BUILTINS_FOR_ENGINES
androp no longer needed (see line 591 in 7b235a1)::

    # pylint: disable=undefined-variable

Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914068609
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser cd033b5416 [fix] drop useless pylint: disable=undefined-variable
Since 7b235a1 (see line 591) it is no longer needed to disable
'undefined-variable' for names defined in::

   PYLINT_ADDITIONAL_BUILTINS_FOR_ENGINES

Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914068609
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 222031e975 [fix] make oscar theme more CSP compliant - col-checkbox
Add col-checkbox in::

    searx/static/themes/oscar/src/less/preferences.less

Replaced style in file::

    searx/templates/oscar/preferences.html

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 9d106ae972 [fix] make oscar theme more CSP compliant - default-image-style
Add default-image-style in::

    searx/static/themes/oscar/src/less/result_templates.less

Replaced style= in files::

    ./oscar/result_templates/default.html:19:        <img src="{{ image_proxify(result.img_src or result.thumbnail) }}" title="{{ result.title|striptags }}" style="width: auto; max-height: 60px; min-height: 60px;" class="col-xs-2 col-sm-4 col-md-4 result-content">
    ./oscar/result_templates/files.html:24:<img src="{{ image_proxify(result.img_src) }}" alt="{{ result.title|striptags }}" title="{{ result.title|striptags }}" style="width: auto; max-height: 60px; min-height: 60px;" class="col-xs-2 col-sm-4 col-md-4 result-content">

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser e564cef0fe [fix] make /preferences more CSP compliant - simple & oscar theme
In simple & oscar theme remove ``style=`` properties which violates CSP but seem
not to have a magnification in GUI (UX).

Fixed files::

    ./oscar/preferences.html:25:<td class="{{ label }}" style="padding: 2px">{{- "" -}}
    ./simple/preferences.html:39:<td class="{{ label }}" style="padding: 2px; width: 13rem;">{{- "" -}}
    ./simple/preferences.html:77:    <div class="engine-tooltip" style="right: 12rem;" role="tooltip" id="{{engine_name}}_reliability">
    ./simple/preferences.html:258:     <p class="text-muted" style="margin:20px 0;">

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament ea60c03827 [fix] fix openstreetmap engine
close #298

This is a workaround: inside engine code, any call to function in another engine can crash
since the logger won't be initialized except if it is done explicitly.
3 years ago
Markus Heiser aecfb2300d [mod] one logger per engine - drop obsolete logger.getChild
Remove the no longer needed `logger = logger.getChild(...)` from engines.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 7b235a1c36 [mod] one logger per engine
Suggested-by: @dalf in https://github.com/searxng/searxng/issues/98#issuecomment-849013518
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 0d070fde92 [build] /static 3 years ago
Markus Heiser ad528c706e [fix] make result template map more CSP compliant - simple & oscar
Add osm-map-box in::

    searx/static/themes/__common__/less/result_templates.less

Replaced sty= in files::

    ./oscar/result_templates/map.html:64:        <div style="height:300px; width:100%; margin: 10px 0;" id="osm-map-{{ index }}"></div>
    ./simple/result_templates/map.html:65:    <div id="result-map-{{ index }}" class="invisible"><div id="osm-map-{{ index }}" style="height:300px; width:100%; margin: 10px 0;" ></div></div>

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 48c806954e [fix] make /preferences more CSP compliant - simple & oscar theme
in simple & oscar theme replace bar-graph's *styles* by CSP compliant
implementation in::

    searx/static/themes/__common__/less/stats.less

Fixed files::

    ./oscar/preferences.html:29:        <span style="width: calc(max(2px, 100%*{{ (stats[engine_name].time / max_rate95)|round(3) }}))" class="stacked-bar-chart-median"></span>{{- "" -}}
    ./oscar/preferences.html:30:        <span style="width: calc(100%*{{ ((stats[engine_name].rate80 - stats[engine_name].time) / max_rate95)|round(3) }})" class="stacked-bar-chart-rate80"></span>{{- "" -}}
    ./oscar/preferences.html:31:        <span style="width: calc(100%*{{ ((stats[engine_name].rate95 - stats[engine_name].rate80) / max_rate95)|round(3) }})" class="stacked-bar-chart-rate95"></span>{{- "" -}}
    ./simple/preferences.html:43:        <span style="width: calc(max(2px, 100%*{{ (stats[engine_name].time / max_rate95)|round(3) }}))" class="stacked-bar-chart-median"></span>{{- "" -}}
    ./simple/preferences.html:44:        <span style="width: calc(100%*{{ ((stats[engine_name].rate80 - stats[engine_name].time) / max_rate95)|round(3) }})" class="stacked-bar-chart-rate80"></span>{{- "" -}}
    ./simple/preferences.html:45:        <span style="width: calc(100%*{{ ((stats[engine_name].rate95 - stats[engine_name].rate80) / max_rate95)|round(3) }})" class="stacked-bar-chart-rate95"></span>{{- "" -}}

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament 33ea1d5909
Merge pull request #289 from return42/fix-xpath
[fix] remove minimum length of content for XPath engine
3 years ago
Alexandre Flament 39bbd15b39
Merge pull request #287 from return42/azlyrics
[enh] add engine AZLyrics.com (XPath)
3 years ago
Markus Heiser 9ff881f937 [fix] remove minimum length of content for XPath engine
Instead of raising an exception and therefore hiding all results of the engine.

It make sense to remove that requirement in order to allow the implementation of
search engines that do not always have a description.  In fact some search
engines that in 99% of the case have a description like Brave Search or Mojeek
crash completely if they for some reason included a result with no description.

To test this patch try Mojeek:

    !mjk xyz

before and after the patch.

Suggested-by: 0xhtml in https://github.com/searx/searx/discussions/2933
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 730073b850 [enh] add suggestions to brave engine
Suggestions should be added too.

    suggestion_xpath: //div[@class="text-gray h6"]/a

You can try it with:

    !brave recurzuoin

Suggested-by: @allendema in https://github.com/searx/searx/issues/2857#issuecomment-904837023
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Allen 0648757e7f [enh] add engine AZLyrics.com (XPath)
Upstream example query:
  https://search.azlyrics.com/search.php?q=The+verbal+acupunture&w=lyrics&p=1

Cherry picked from [1] and slightly modified to pass yamllint test.

[1] https://github.com/allendema/searx/commit/8b6d5a0e5

Reviewed-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Allen a5a0a4e106 [fix] Correct engine name in for Rumble 3 years ago
Allen 49bbd250d9 [fix] Update about section of Invidious
Another website and new documentation
3 years ago
Markus Heiser ab4456b0d0
Merge pull request #281 from dalf/cleanup
Cleanup
3 years ago
Markus Heiser df351937f4 [fix] typo in searx.webadapter.parse_lang
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
searxng-bot 3120c9ca79 [translations] update
4fd5bb5a - 2021-08-27 - Allan Nordhøy <epost@anotheragency.no>
3 years ago
Alexandre Flament a14ea6396e [mod] remove searx.raise_for_httperror
Not used anymore
3 years ago
Alexandre Flament a7d781c49d [mod] remove remaining references to locales in settings.yml
see #247
3 years ago
Alexandre Flament 3c377c0aa8 [mod] searx.webapp: code clean up 3 years ago
Alexandre Flament b9c73fb697 [mod] move searx/testing.py to the tests directory
move robot tests to tests.robot
manage calls "python -m tests.robot"
3 years ago
Markus Heiser 065b4dab56
Merge pull request #278 from searxng/update_data_update_ahmia_blacklist.py
Update searx.data - update_ahmia_blacklist.py
3 years ago
Markus Heiser a7a879181d
Merge pull request #279 from searxng/update_data_update_currencies.py
Update searx.data - update_currencies.py
3 years ago
Markus Heiser 00a51fa17f
Merge pull request #280 from searxng/update_data_update_firefox_version.py
Update searx.data - update_firefox_version.py
3 years ago
dalf 9e3113865d Update searx.data - update_firefox_version.py 3 years ago
dalf a45fc66e83 Update searx.data - update_currencies.py 3 years ago
dalf 44f7e1fcaa Update searx.data - update_ahmia_blacklist.py 3 years ago
dalf a556d0c1aa Update searx.data - update_wikidata_units.py 3 years ago
Markus Heiser 37b3d8c33b
Merge pull request #276 from dalf/simple_eslint
Simple theme: eslint and bug fix
3 years ago
Markus Heiser 840a327564
Merge pull request #275 from koelle25/add-packagist
[enh] Add Packagist search engine
3 years ago
Kevin Köllmann 2b3687780c [enh] Add Packagist search engine (JSON)
Packagist's API: https://packagist.org/apidoc
3 years ago
Alexandre Flament d246679dbe [build] /static 3 years ago
Alexandre Flament f77f797f8c [mod] simple theme: fix eslint errors, fix VIM keys
* VIM keys: fix the next page and previous pages (n, p keys)
* Map: Enable the wikipedia map (the layer was initialized but not included)
3 years ago
Alexandre Flament 66a27757fd
Merge pull request #244 from return42/eslint-simple-client
[mod] simple theme - add ESLint
3 years ago
Alexandre Flament 0ee316f3d1 eslint: grunt integration 3 years ago
Markus Heiser 2948a99b6e [mod] simple theme - add ESLint
[1] https://eslint.org/
[2] https://eslint.org/docs/user-guide/configuring/
[3] https://eslint.org/docs/user-guide/command-line-interface

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament 7dc370efe9 [enh] new issue: include information from searx.version 3 years ago
Markus Heiser 1e942435be
Merge pull request #273 from searxng/dependabot/pip/master/pylint-2.10.2
Bump pylint from 2.9.6 to 2.10.2
3 years ago
Markus Heiser 03e7d423be [pylint] Pylint 2.10 - unused-variable
Pylint 2.10 fixed [1]:

  Fixed bug with cell-var-from-loop checker: it no longer has false negatives
  when both unused-variable and used-before-assignment are disabled.

[1] https://pylint.pycqa.org/en/latest/whatsnew/2.10.html

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser e856b563dd [pylint] Pylint 2.10 - unspecified-encoding
Pylint 2.10 added new default checks [1]:

unspecified-encoding:
  Emitted when open() is called without specifying an encoding [2]

[1] https://pylint.pycqa.org/en/latest/whatsnew/2.10.html
[2] https://github.com/PyCQA/pylint/issues/3826

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser b83c14cf6b [pylint] Pylint 2.10 - fix use-list-literal & use-dict-literal
Pylint 2.10 added new default checks [1]:

use-list-literal
  Emitted when list() is called with no arguments instead of using []

use-dict-literal
  Emitted when dict() is called with no arguments instead of using {}

[1] https://pylint.pycqa.org/en/latest/whatsnew/2.10.html

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
searxng-bot 9e63e20c36 [translations] update
18fae620 - 2021-08-25 - Markus Heiser <markus.heiser@darmarit.de>
2e76843c - 2021-08-25 - ewm <gnu.ewm@protonmail.com>
172a1a88 - 2021-08-25 - Alexandre Flament <alex@al-f.net>
4567065b - 2021-08-25 - Full Name <cheez.1337@mail.ru>
59d9d549 - 2021-08-25 - Markus Heiser <markus.heiser@darmarit.de>
5f135b41 - 2021-08-23 - Markus Heiser <markus.heiser@darmarit.de>
91b3a32e - 2021-08-24 - Full Name <cheez.1337@mail.ru>
0198ea11 - 2021-08-22 - Allan Nordhøy <epost@anotheragency.no>
6efc4b9f - 2021-08-22 - Markus Heiser <markus.heiser@darmarit.de>
a263af1e - 2021-08-23 - Alexandre Flament <alex@al-f.net>
713b07f7 - 2021-08-22 - Markus Heiser <markus.heiser@darmarit.de>
0c7b7859 - 2021-08-22 - Markus Heiser <markus.heiser@darmarit.de>
408c85f9 - 2021-08-20 - Alexandre Flament <alex@al-f.net>
3 years ago
Alexandre Flament 697ebeddcc [mod] searx.utils.dict_subset: rewrite with comprehension 3 years ago
Alexandre Flament 3b0f70ed0f [mod] /image_proxy: use HTTP/1 instead of HTTP/2
httpx: HTTP/2 is slow when a lot data is downloaded.
https://github.com/dalf/pyhttp-benchmark

also, the usage of HTTP/1 decreases the load average
3 years ago
Alexandre Flament 43fcaa642a [fix] image_proxy: always close the httpx respone
previously, when the content type was not an image and some other error,
the httpx response was not closed
3 years ago
Alexandre Flament df15c655f7 [mod] /image_proxy: don't decompress images 3 years ago
Markus Heiser 29b40338d6
Merge pull request #260 from dalf/mod_locale
[mod] searx.webapp.get_locale: read locale from the preferences
3 years ago
Noémi Ványi 3d5e6e0abb [enh] google: add filter=0 to Google engine for more results
backport from searx ( 23b3b56a06ef831af0a1b30a12c26ebd50e329bb )
3 years ago
searxng-bot eafa373cff [translations] update
b25c9d58 - 2021-08-17 - Allan Nordhøy <epost@anotheragency.no>
3 years ago
Markus Heiser 8290f70e24 [fix] /config add missing GIT_BRANCH value
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
searxng-bot 02b6f0c370 [translations] update
fc3041a0 - 2021-08-10 - Allan Nordhøy <epost@anotheragency.no>
fcc3732d - 2021-08-06 - Allan Nordhøy <epost@anotheragency.no>
894bd655 - 2021-08-06 - Allan Nordhøy <epost@anotheragency.no>
3 years ago
Alexandre Flament 28ccf47ca4 [mod] searx.webapp.get_locale: read locale from the preferences
pre_request already set the locale.
3 years ago
Alexandre Flament a925254c3d
Merge pull request #258 from dalf/plugin_minor_change
[mod] plugins: minor change
3 years ago
Alexandre Flament 2f363858b8 [fix] searx.search.checker.get_result() always return a dict
So checker_results['status'] == 'ok' is enough to check the checker result.
See searx/webapp.py, /preferences endpoint
3 years ago
Alexandre Flament 3f3b5d6181 [mod] plugins: minor change
required attributes: display a different message
when the attribute has the wrong type
3 years ago
Markus Heiser 15bf9fc850 [fix] settings.yml - replace http:// URLs by https://
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament 0d20e5dfe3 [mod] searx/locales.py: language names based on Unicode CLDR
rename "oc" to "Occitan":
* https://github.com/unicode-org/cldr/blob/35.1/seed/main/oc.xml#L115
* https://oc.wikipedia.org/wiki/Occitan

see https://github.com/searxng/searxng/pull/247#issuecomment-892382001
3 years ago
Markus Heiser 809bf1a105 [mod] pylint & document searx.locales (settings.yml: remove locales)
- Add ``# lint: pylint`` header to pylint this python file.
- Fix issues reported by pylint.
- Add source code documentation of modul searx.locales

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament f30d01ffab [mod] settings.yml: remove locales
There are detected from the searx/translations directory
3 years ago
Alexandre Flament bd17544f82 [fix] make test.robot: firefox profile: intl.accept_languages=en
make sure Firefox in test.robot asks for English pages
3 years ago
Alexandre Flament 7fa7177759 Update translations (pot, po) 3 years ago
Markus Heiser e0ca749ff5
Merge pull request #242 from searxng/update_data_update_currencies.py
Update searx.data - update_currencies.py
3 years ago
Alexandre Flament b6711ba0ad
Merge pull request #241 from searxng/update_data_update_languages.py
Update searx.data - update_languages.py
3 years ago
Markus Heiser 274edd523c [fix] get common used languages back (min_engines_per_lang = 13)
In commit 4d3f2f48d common used languages has been droped.  By reducing the
number of `min_engines_per_lang` from 15 to 13 we get theses languages back.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 0fc1dcbfa5
Merge pull request #243 from searxng/update_data_update_wikidata_units.py
Update searx.data - update_wikidata_units.py
3 years ago
dalf 7fae5647d3 Update searx.data - update_wikidata_units.py 3 years ago
dalf 2963c83daa Update searx.data - update_currencies.py 3 years ago
dalf 4d3f2f48d2 Update searx.data - update_languages.py 3 years ago
dalf f5389fd722 Update searx.data - update_firefox_version.py 3 years ago
Alexandre Flament 38ee88b390
Merge pull request #229 from dalf/version-from-git
version based on git
3 years ago
Alexandre Flament 4b43775c91 version based on the git repository
This commit remove the need to update the brand for GIT_URL and GIT_BRANCH:
there are read from the git repository.

It is possible to call python -m searx.version freeze to freeze the current version.
Useful when the code is installed outside git (distro package, docker, etc...)
3 years ago
Alexandre Flament feef577f99
Merge pull request #230 from dalf/remove-zh-hant-tw
Remove zh_Hant_TW translation
3 years ago
Alexandre Flament 0b2dca088f Remove zh_Hant_TW translation
See https://github.com/searxng/searxng/issues/195#issuecomment-887518808
3 years ago
Alexandre Flament ed0db4d61a
Merge pull request #226 from return42/hardening
[mod] hardening SearXNG instances by default (formats)
3 years ago
Alexandre Flament 24fcd7475a Revert "[brand] !!DO NOT MERGE!! this patch uses branch fix-searx.sh @return42"
This reverts commit e946c3462c.
3 years ago
Markus Heiser beff764d74
Merge pull request #216 from return42/fix-searx.sh
normalize environment of installation tasks (shell) with YAML settings
3 years ago
Markus Heiser 80f5241387
Merge pull request #225 from return42/docs-nosql
[docs] document NoSLQ engines MongoDB and redis
3 years ago
Samuel Dudik 7a7ef9cea6 [fix] Seznam engine - some XPath selectors has been changed
Merged from https://github.com/dudik/searx/commit/5a4207759

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 7b40c736e6 [mod] hardening SearXNG instances by default (formats)
Deny formats has been implemented in 6ed4616d.

To harden SearXNG instances by default, other formats than HTML should be
denied.  Most of JSON, RSS and CSV requests are bots [1]::

    Bots are the only users of this feature on a public instance, and they abuse
    it too much that the engines rate limit pretty quickly the IP address of the
    instance.

[1] https://github.com/searxng/searxng/issues/95

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Noémi Ványi 5e648e77ab [docs] document NoSLQ engines MongoDB and redis
Slightly modified merge of commit [3b192e638]

[3b192e638] https://github.com/searx/searx/commit/3b192e638

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago