Commit Graph

2828 Commits (34859d0e5872499c72fcaebb67ca57f95beb9864)

Author SHA1 Message Date
Markus Heiser 34859d0e58 [fix] make /stats more CSP compliant - oscar theme
Replace oscar theme's *styles* (see below) by CSP compliant implementation in
``searx/static/themes/__common__/less/stats.less`` ::

    ./oscar/stats.html:29:  <th scope="col" style="width:20rem;">{{ th_sort('name', _("Engine name")) }}</th>
    ./oscar/stats.html:30:  <th scope="col" style="width:7rem; text-align: right;">{{ th_sort('score', _('Scores')) }}</th>
    ./oscar/stats.html:33:  <th scope="col" style="text-align: right;">{{ th_sort('reliability', _('Reliability')) }}</th>
    ./oscar/stats.html:38:  <td style="text-align: right;">
    ./oscar/stats.html:91:  <td style="text-align: right;"> {{ engine_reliabilities.get(engine_stat.name, {}).get('reliablity') }}</td>
    ./oscar/stats.html:109: <tbody style="padding-top: 1rem;">
    ./oscar/stats.html:112: <th scope="row" style="width: 10rem">{{ _('Exception') }}</th><td>{{ error.exception_classname }}</td>
    ./oscar/stats.html:114: <th scope="row" style="width: 10rem">{{ _('Message') }}</th><td>{{ error.log_message }}</td>
    ./oscar/stats.html:116: <th scope="row" style="width: 10rem">{{ _('Percentage') }}</th><td style="width: 10rem">{{ error.percentage }}</td>
    ./oscar/stats.html:121: <span style="border-right: 1px solid gray; padding: 0 1rem 0 0; margin: 0 0 0 0.5rem;">{{ param }}</span>
    ./oscar/stats.html:138: <th scope="col" style="width: 10rem">{{ _('Failed test') }}</th>

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 0a6c488417 [fix] make /stats more CSP compliant - simple theme
Replace simple theme's *styles* (see below) by CSP compliant implementation in
``searx/static/themes/simple/less/stats.less`` ::

    ./simple/stats.html:26:  <table style="max-width: 1280px; margin: 0 auto 0 0;">
    ./simple/stats.html:28:  <th scope="col" style="width:20rem;">{{ th_sort('name', _("Engine name")) }}</th>
    ./simple/stats.html:29:  <th scope="col" style="width:7rem; text-align: right;">{{ th_sort('score', _('Scores')) }}</th>
    ./simple/stats.html:32:  <th scope="col" style="text-align: right;">{{ th_sort('reliability', _('Reliability')) }}</th>
    ./simple/stats.html:37:  <td style="text-align: right;">
    ./simple/stats.html:90:  <td style="text-align: right;"> {{ engine_reliabilities.get(engine_stat.name, {}).get('reliablity') }}</td>
    ./simple/stats.html:106: <table style="max-width: 1280px; margin: 1rem; border: 1px solid gray;">
    ./simple/stats.html:107: <tbody style="padding-top: 1rem;">
    ./simple/stats.html:110: <th scope="row" style="width: 10rem">{{ _('Exception') }}</th><td>{{ error.exception_classname }}</td>
    ./simple/stats.html:112: <th scope="row" style="width: 10rem">{{ _('Message') }}</th><td>{{ error.log_message }}</td>
    ./simple/stats.html:114: <th scope="row" style="width: 10rem">{{ _('Percentage') }}</th><td style="width: 10rem">{{ error.percentage }}</td>
    ./simple/stats.html:119: <span style="border-right: 1px solid gray; padding: 0 1rem 0 0; margin: 0 0 0 0.5rem;">{{ param }}</span>
    ./simple/stats.html:136: <th scope="col" style="width: 10rem">{{ _('Failed test') }}</th>

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser cdfb4b7ff9 [fix] make /stats more CSP compliant - bar graph
Replace bar graph's *styles* (see below) by CSP compliant implementation in
``searx/static/themes/__common__/less/stats.less`` ::

    ./simple/stats.html:49: <span style="width: calc(max(2px, 100%*{{ (engine_stat.result_count / engine_stats.max_result_count )|round(3) }}))" class="stacked-bar-chart-serie1"></span>{{- "" -}}
    ./simple/stats.html:57: <span style="width: calc(max(2px, 100%*{{ (engine_stat.http / engine_stats.max_time )|round(3) }}))" class="stacked-bar-chart-serie1"></span>{{- "" -}}
    ./simple/stats.html:58: <span style="width: calc(100%*{{ engine_stat.processing / engine_stats.max_time |round(3) }})" class="stacked-bar-chart-serie2"></span>{{- "" -}}
    ./oscar/stats.html:50:  <span style="width: calc(max(2px, 100%*{{ (engine_stat.result_count / engine_stats.max_result_count )|round(3) }}))" class="stacked-bar-chart-serie1"></span>{{- "" -}}
    ./oscar/stats.html:58:  <span style="width: calc(max(2px, 100%*{{ (engine_stat.http / engine_stats.max_time )|round(3) }}))" class="stacked-bar-chart-serie1"></span>{{- "" -}}
    ./oscar/stats.html:59:  <span style="width: calc(100%*{{ engine_stat.processing / engine_stats.max_time |round(3) }})" class="stacked-bar-chart-serie2"></span>{{- "" -}}

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 965817f294 [fix] simple theme - generate missing sourceMap file
C&P from searx/static/themes/oscar/gruntfile.js

Suggested-by: @dalf in commit 1204e4f0
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 93594a7b42 [less] update grunt-contrib-less v3.3.0
Upgraded [v3.3.0] otherwise::

`  width: calc(100% - 5rem);`

becomes `width: 95%` once compiled by less version 1.4.1.

[v3.3.0] https://github.com/gruntjs/grunt-contrib-less/releases/tag/v3.0.0

Suggested-by: @dalf in commit 1204e4f0
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament 90fe232e08 [fix] webapp.py: pylint: disable=assigning-non-slot for request.*
Note: actually, the code should use flask.g.* instead of request.*
3 years ago
Alexandre Flament d12033e166 [mod] Server-Timing header: add render field
Add a "render" field to the Server-Timing header
This field is the time spent in the searx.webapp.render function (Jinja2 rendering)

See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing
4 years ago
Alexandre Flament 0f4e995ab4 [mod] searx.network.client: the same configuration reuses the same ssl.SSLContext
before there was one ssl.SSLContext per client.

see https://github.com/encode/httpx/issues/978
4 years ago
Markus Heiser fa0d05c313 [pylint] checker/__main__.py & checker/background.py
Lint files that has been touched by [PR #58]

[PR #58] https://github.com/searxng/searxng/pull/58

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Alexandre Flament 8c1a65d32f [mod] multithreading only in searx.search.* packages
it prepares the new architecture change,
everything about multithreading in moved in the searx.search.* packages

previously the call to the "init" function of the engines was done in searx.engines:
* the network was not set (request not sent using the defined proxy)
* it requires to monkey patch the code to avoid HTTP requests during the tests
4 years ago
Markus Heiser d36adfa59f
Merge pull request #55 from searxng/upd-relialibility-column
[themes] /preferences, reliablity: warning icons & error logs link
4 years ago
Markus Heiser 5253df946b
[mod] option to enable or disable "proxy" button next to each result (#54)
* [mod] option to enable or disable "proxy" button next to each result

Closes: https://github.com/searxng/searxng/issues/51
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Co-authored-by: Alexandre Flament <alex@al-f.net>
4 years ago
Alexandre Flament e83862c22f [themes] /preferences, reliablity: warning icons & error logs link
When there is at least one errors or one failed checker test:
* the warning icon is displayed in the reliability column
* the link "View error logs and submit a bug report" is displayed on engine name tooltip.

Before:
* the warning icon was displayed only when one or more checker test(s) failed.
* the link "View error logs and submit a bug report" was not shown when a checker test failed but there were no error.
4 years ago
Markus Heiser 45293270e2
Merge pull request #44 from searxng/dependabot/pip/master/pylint-2.8.2
Bump pylint from 2.7.4 to 2.8.2
4 years ago
dependabot[bot] 2a53f79a2d Bump pylint from 2.7.4 to 2.8.2
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.7.4 to 2.8.2.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Changelog](https://github.com/PyCQA/pylint/blob/master/ChangeLog)
- [Commits](https://github.com/PyCQA/pylint/compare/pylint-2.7.4...v2.8.2)

Signed-off-by: dependabot[bot] <support@github.com>
4 years ago
Marc Abonce Seguin 448bfe6005 fix Qwant's fetch_languages function 4 years ago
Alexandre Flament 2756e170ce
Merge pull request #49 from searxng/update_data_update_ahmia_blacklist.py
Update searx.data - update_ahmia_blacklist.py
4 years ago
Alexandre Flament a3a7be90ff
Merge pull request #50 from searxng/update_data_update_currencies.py
Update searx.data - update_currencies.py
4 years ago
Alexandre Flament 4558c0c027
Merge pull request #48 from searxng/update_data_update_wikidata_units.py
Update searx.data - update_wikidata_units.py
4 years ago
dalf dd8a5a00dc Update searx.data - update_currencies.py 4 years ago
dalf de54209f9e Update searx.data - update_ahmia_blacklist.py 4 years ago
dalf 982a5a7d2e Update searx.data - update_wikidata_units.py 4 years ago
dalf c004712eef Update searx.data - update_firefox_version.py 4 years ago
Alexandre Flament e5bcc3cbbe
Merge pull request #40 from searxng/link-to-stats
[mod] engine errors: link to the stats to create an github issue
4 years ago
Markus Heiser 8026ed0d37 [mod] UI: engine pref-page, about toolbox: add link to stats
In the preference page, in the 'about' toolbox of an engine, add a link to the
stats page of the engine, if the engine had one or more errors.

Condition is::

    reliabilities[<engine.name>].errors

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Alexandre Flament 34bced29ae [mod] engine errors: link to the stats to create an github issue 4 years ago
Michael Ilsaas 0c43cf89ca [fix] URL to solidtorrent result page
Reported-by: https://github.com/searx/searx/pull/2786
4 years ago
Markus Heiser 8291804082 [static] make themes.all - from commit 9e8171e38
Based on commit 9e8171e38 this patch is generated by::

     make themes.all

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser 70cc196e2d [fix] new_issue.html: drop inline style attributes (CSP conformance)
Inline styles are blocked by default with Content Security Policy (CSP).  Move
the inline styles from 'new_issue.html' to::

    searx/static/themes/__common__/less/new_issue.less

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser 7f2dc530b2 [fix] simple theme - remove no longer used stats.less file
File searx/static/themes/simple/less/stats.less is not used (imported) in any
other less file.  I can't say when it's usage was dropped or if it has ever been
used.  ATM this file is without any usage.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser 54d57c8296 [fix] grunt/less/development: oscar light & dark logicodev mixes CSS
closes: https://github.com/searxng/searxng/issues/39

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Alexandre Flament 283ae7bfad [fix] searx.network: fix rare cases where LOOP is None
* searx.network.client.LOOP is initialized in a thread
* searx.network.__init__ imports LOOP which may happen
  before the thread has initialized LOOP

This commit adds a new function "searx.network.client.get_loop()"
to fix this issue
4 years ago
Alexandre Flament 87e914e398
Merge pull request #28 from searxng/mod-processors-error-message
[mod] processors: show identical error messages on /search and /stats
4 years ago
Markus Heiser 924f9afea3 [lint] pylint searx/search/processors files / BTW add some doc-strings
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Alexandre Flament b1557b5443 [mod] processors: show identical error messages on /search and /stats 4 years ago
Alexandre Flament b8f5adf500 [mod] searx.metrics.error_recorder: store relative file name instead of the full absolute file name. 4 years ago
Markus Heiser dc29f1d826 [pylint] tag PYLINT_FILES by comment `# lint: pylint`
These py files are linted by `test.pylint`, all other files are linted by
`test.pep8`.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser 6b92e8c6fd [upd] ./manage pyenv.cmd searx_extra/update/update_pygments.py
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser 1a402be175 [fix] debug log: UnicodeEncodeError: 'ascii' codec can't encode
The issue exists only in the debug log::

     --- Logging error ---
     Traceback (most recent call last):
       File "/usr/lib/python3.9/logging/__init__.py", line 1086, in emit
	 stream.write(msg + self.terminator)
     UnicodeEncodeError: 'ascii' codec can't encode characters in position 79-89: ordinal not in range(128)
     Call stack:
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/app.py", line 2464, in __call__
	 return self.wsgi_app(environ, start_response)
       File "/usr/local/searx/searx-src/searx/webapp.py", line 1316, in __call__
	 return self.app(environ, start_response)
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/werkzeug/middleware/proxy_fix.py", line 169, in __call__
	 return self.app(environ, start_response)
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app
	 response = self.full_dispatch_request()
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request
	 rv = self.dispatch_request()
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
	 return self.view_functions[rule.endpoint](**req.view_args)
       File "/usr/local/searx/searx-src/searx/webapp.py", line 766, in search
	 number_of_results=format_decimal(number_of_results),
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask_babel/__init__.py", line 458, in format_decimal
	 locale = get_locale()
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask_babel/__init__.py", line 226, in get_locale
	 rv = babel.locale_selector_func()
       File "/usr/local/searx/searx-src/searx/webapp.py", line 249, in get_locale
	 logger.debug("%s uses locale `%s` from %s", request.url, locale, locale_source)
     Unable to print the message and arguments - possible formatting error.
     Use the traceback above to help find the error.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Alexandre Flament 51494849fc
Merge pull request #23 from searxng/metrics-stats-engines
Metrics stats engines
4 years ago
Alexandre Flament df41b77121 [mod] /stats : detail per engine
allow to submit a github issue including the technical details
(exceptions, errors, warning, checker result)
4 years ago
Markus Heiser 28b25185c5 [brand] searxng -- fix links to issue tracker & WEB-GUI
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser c6a5cc019a [brand] searxng is a fork from searx
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser 6d41255eb1 [fix] Springer-Nature & core.ac need API key to work
It is the whole engine definition that should be comment out : without the
api_key nothing works.

Signed-off-by: Markus Heiser <markus@darmarit.de>
4 years ago
Markus Heiser 8efabd3ab7 [mod] core.ac.uk engine
- add to list of pylint scripts
- add debug log messages
- move API key int `settings.yml`
- improved readability
- add some metadata to results

Signed-off-by: Markus Heiser <markus@darmarit.de>
4 years ago
spongebob33 7528e38c8a add core.ac.uk engine 4 years ago
Alexandre Flament 525bfe7830
Fix PEP8 4 years ago
Alexandre Flament 7032f7521c [mod] /stats: simple theme implementation 4 years ago
Alexandre Flament 09e7ecdce2 [mod] /stats : add reliability column and sort by column links 4 years ago
Markus Heiser c54bf42cb9 [mod] metrics: add to pylint and justify indentations
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago