From e7788980ad1bb82e9ae084428d05a79b235c17ff Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Tue, 12 May 2020 14:17:47 +0200 Subject: [PATCH 01/30] [fix] about page- 'How to set as the default search engine?' Searx supports OpenSearch, remove the inline JS from the about page. close: https://github.com/asciimoo/searx/issues/1949 close: https://github.com/asciimoo/searx/issues/1616 Signed-off-by: Markus Heiser --- searx/templates/__common__/about.html | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/searx/templates/__common__/about.html b/searx/templates/__common__/about.html index d3e8d06a..9741b516 100644 --- a/searx/templates/__common__/about.html +++ b/searx/templates/__common__/about.html @@ -59,12 +59,16 @@

How to set as the default search engine?

-
Firefox
+

+ Searx supports OpenSearch. + For more information on changing your default search engine, see your browser's documentation: +

-
- Install - searx as a search engine on any version of Firefox! (javascript required) -
+

Where to find anonymous usage statistics of this instance ?

From 250e05e826c5aa072c0f05b0c1f0a8f66ba47631 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Tue, 12 May 2020 14:57:45 +0200 Subject: [PATCH 02/30] [README] update and polish up a little by using badges about markup & badges - https://github.com/ralsina/rst-cheatsheet/blob/master/rst-cheatsheet.rst - https://github.com/Naereen/badges/blob/master/README.rst - https://shields.io/ - https://gist.github.com/tterb/982ae14a9307b80117dbf49f624ce0e8 Signed-off-by: Markus Heiser --- README.rst | 90 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 47 insertions(+), 43 deletions(-) diff --git a/README.rst b/README.rst index a7b75d6e..c0008acd 100644 --- a/README.rst +++ b/README.rst @@ -1,63 +1,67 @@ -searx -===== +.. SPDX-License-Identifier: AGPL-3.0-or-later -A privacy-respecting, hackable `metasearch -engine `__. +.. figure:: https://raw.githubusercontent.com/asciimoo/searx/master/searx/static/themes/oscar/img/logo_searx_a.png + :target: https://asciimoo.github.io/searx/ + :alt: searX + :width: 100% + :align: center -Pronunciation: səːks - -List of `running -instances `__. - -See the `documentation `__ and the `wiki `__ for more information. +------- +|searx install| +|searx homepage| +|searx wiki| +|AGPL License| +|Issues| +|commits| |OpenCollective searx backers| |OpenCollective searx sponsors| -Installation -~~~~~~~~~~~~ +Privacy-respecting, hackable `metasearch engine`_ / *pronunciation* **səːks**. -With Docker ------------ -Go to the `searx-docker `__ project. +.. _metasearch engine: https://en.wikipedia.org/wiki/Metasearch_engine -Without Docker --------------- -For all of the details, follow this `step by step installation `__. +.. |searx install| image:: https://img.shields.io/badge/-install-blue + :target: https://asciimoo.github.io/searx/admin/installation.html -Note: the documentation needs to be updated. +.. |searx homepage| image:: https://img.shields.io/badge/-homepage-blue + :target: https://asciimoo.github.io/searx -If you are in a hurry ---------------------- -- clone the source: - ``git clone https://github.com/asciimoo/searx.git && cd searx`` -- install dependencies: ``./manage.sh update_packages`` -- edit your - `settings.yml `__ - (set your ``secret_key``!) -- run ``python searx/webapp.py`` to start the application +.. |searx wiki| image:: https://img.shields.io/badge/-wiki-blue + :target: https://github.com/asciimoo/searx/wiki +.. |AGPL License| image:: https://img.shields.io/badge/license-AGPL-blue.svg + :target: https://github.com/asciimoo/searx/blob/master/LICENSE -Bugs -~~~~ +.. |Issues| image:: https://img.shields.io/github/issues/asciimoo/searx?color=yellow&label=issues + :target: https://github.com/asciimoo/searx/issues -Bugs or suggestions? Visit the `issue -tracker `__. - -`License `__ -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -More about searx -~~~~~~~~~~~~~~~~ - -- `openhub `__ -- `twitter `__ -- IRC: #searx @ freenode +.. |PR| image:: https://img.shields.io/github/issues-pr-raw/asciimoo/searx?color=yellow&label=PR + :target: https://github.com/asciimoo/searx/pulls +.. |commits| image:: https://img.shields.io/github/commit-activity/y/asciimoo/searx?color=yellow&label=commits + :target: https://github.com/asciimoo/searx/commits/master .. |OpenCollective searx backers| image:: https://opencollective.com/searx/backers/badge.svg :target: https://opencollective.com/searx#backer - .. |OpenCollective searx sponsors| image:: https://opencollective.com/searx/sponsors/badge.svg :target: https://opencollective.com/searx#sponsor + + +If you are looking for running instances, ready to use, then visit searx.space_. + +Otherwise jump to the user_, admin_ and developer_ handbooks you will find on +our homepage_. + +.. _searx.space: https://searx.space +.. _user: https://asciimoo.github.io/searx/user +.. _admin: https://asciimoo.github.io/searx/admin +.. _developer: https://asciimoo.github.io/searx/dev +.. _homepage: https://asciimoo.github.io/searx + +contact: + openhub_ // twitter_ // IRC: #searx @ freenode + +.. _openhub: https://www.openhub.net/p/searx +.. _twitter: https://twitter.com/Searx_engine From f8b16a6635d553f91cc3d2e81b816e10517e9a72 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Tue, 12 May 2020 15:02:53 +0200 Subject: [PATCH 03/30] [README] searx, world's first gluten free search engine ;) Signed-off-by: Markus Heiser --- README.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/README.rst b/README.rst index c0008acd..55deb674 100644 --- a/README.rst +++ b/README.rst @@ -65,3 +65,9 @@ contact: .. _openhub: https://www.openhub.net/p/searx .. _twitter: https://twitter.com/Searx_engine + +------- + +|gluten free| + +.. |gluten free| image:: https://forthebadge.com/images/featured/featured-gluten-free.svg From de179ecc5b8a966f0a46d266c1a04bb2865b27c4 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Thu, 28 May 2020 15:17:59 +0200 Subject: [PATCH 04/30] [fix] remove debug print from commit e5305f8 Signed-off-by: Markus Heiser --- searx/engines/gigablast.py | 1 - 1 file changed, 1 deletion(-) diff --git a/searx/engines/gigablast.py b/searx/engines/gigablast.py index 2bb29a9f..607bbfcb 100644 --- a/searx/engines/gigablast.py +++ b/searx/engines/gigablast.py @@ -62,7 +62,6 @@ def init(engine_settings=None): # do search-request def request(query, params): - print("EXTRAPARAM:", extra_param) offset = (params['pageno'] - 1) * number_of_results if params['language'] == 'all': From 151038255810ed972187ee5d7921e3f044557665 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Thu, 28 May 2020 15:18:47 +0200 Subject: [PATCH 05/30] utils/fetch_languages.py: print more meaningfull messages Signed-off-by: Markus Heiser --- utils/fetch_languages.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/utils/fetch_languages.py b/utils/fetch_languages.py index ddebb411..77ec0bf2 100644 --- a/utils/fetch_languages.py +++ b/utils/fetch_languages.py @@ -28,10 +28,10 @@ def fetch_supported_languages(): names.sort() for engine_name in names: - print("fetching languages of engine %s" % engine_name) - if hasattr(engines[engine_name], 'fetch_supported_languages'): engines_languages[engine_name] = engines[engine_name].fetch_supported_languages() + print("fetched %s languages from engine %s" % ( + len(engines_languages[engine_name]), engine_name)) if type(engines_languages[engine_name]) == list: engines_languages[engine_name] = sorted(engines_languages[engine_name]) From 57c7b90edd30ad0d24231a7e338c770c09de2dd4 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Thu, 28 May 2020 15:21:01 +0200 Subject: [PATCH 06/30] [fix] gigablast does no longer support *supported_languages_url* Since there are zero results, we can remove it: $ make engines.languages fetch languages .. ... fetched 0 languages from engine gigablast Signed-off-by: Markus Heiser --- searx/engines/gigablast.py | 20 -------------------- 1 file changed, 20 deletions(-) diff --git a/searx/engines/gigablast.py b/searx/engines/gigablast.py index 607bbfcb..a71dd116 100644 --- a/searx/engines/gigablast.py +++ b/searx/engines/gigablast.py @@ -41,8 +41,6 @@ url_xpath = './/url' title_xpath = './/title' content_xpath = './/sum' -supported_languages_url = 'https://gigablast.com/search?&rxikd=1' - extra_param = '' # gigablast requires a random extra parameter # which can be extracted from the source code of the search page @@ -108,21 +106,3 @@ def response(resp): # return results return results - - -# get supported languages from their site -def _fetch_supported_languages(resp): - supported_languages = [] - dom = fromstring(resp.text) - links = eval_xpath(dom, '//span[@id="menu2"]/a') - for link in links: - href = eval_xpath(link, './@href')[0].split('lang%3A') - if len(href) == 2: - code = href[1].split('_') - if len(code) == 2: - code = code[0] + '-' + code[1].upper() - else: - code = code[0] - supported_languages.append(code) - - return supported_languages From a18760b322ba0006672b3d8f09362b407b39b4c1 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Thu, 28 May 2020 19:24:03 +0200 Subject: [PATCH 07/30] [fix] revise of the gigablast engine (WIP) The gigablast API has changed and seems to have some quirks, this is the first revise. More work (hacks) are needed. Signed-off-by: Markus Heiser --- searx/engines/gigablast.py | 127 +++++++++++++++++-------------------- 1 file changed, 59 insertions(+), 68 deletions(-) diff --git a/searx/engines/gigablast.py b/searx/engines/gigablast.py index a71dd116..37221c55 100644 --- a/searx/engines/gigablast.py +++ b/searx/engines/gigablast.py @@ -9,14 +9,11 @@ @stable yes @parse url, title, content """ +# pylint: disable=missing-function-docstring, invalid-name -import random -from json import loads from time import time -from lxml.html import fromstring -from searx.poolrequests import get +from json import loads from searx.url_utils import urlencode -from searx.utils import eval_xpath # engine dependent config categories = ['general'] @@ -26,83 +23,77 @@ language_support = True safesearch = True # search-url + base_url = 'https://gigablast.com/' -search_string = 'search?{query}'\ - '&n={number_of_results}'\ - '&c=main'\ - '&s={offset}'\ - '&format=json'\ - '&langcountry={lang}'\ - '&ff={safesearch}'\ - '&rand={rxikd}' -# specific xpath variables -results_xpath = '//response//result' -url_xpath = './/url' -title_xpath = './/title' -content_xpath = './/sum' - -extra_param = '' # gigablast requires a random extra parameter -# which can be extracted from the source code of the search page - - -def parse_extra_param(text): - global extra_param - param_lines = [x for x in text.splitlines() if x.startswith('var url=') or x.startswith('url=url+')] - extra_param = '' - for l in param_lines: - extra_param += l.split("'")[1] - extra_param = extra_param.split('&')[-1] - - -def init(engine_settings=None): - parse_extra_param(get('http://gigablast.com/search?c=main&qlangcountry=en-us&q=south&s=10').text) - # do search-request -def request(query, params): - offset = (params['pageno'] - 1) * number_of_results +def request(query, params): # pylint: disable=unused-argument - if params['language'] == 'all': - language = 'xx' - else: - language = params['language'].replace('-', '_').lower() - if language.split('-')[0] != 'zh': - language = language.split('-')[0] + # see API http://www.gigablast.com/api.html#/search + # Take into account, that the API has some quirks .. + + query_args = dict( + c = 'main' + , n = number_of_results + , format = 'json' + , q = query + # The gigablast HTTP client sends a random number and a nsga argument + # (the values don't seem to matter) + , rand = int(time() * 1000) + , nsga = int(time() * 1000) + ) + + page_no = (params['pageno'] - 1) + if page_no: + # API quirk; adds +2 to the number_of_results + offset = (params['pageno'] - 1) * number_of_results + query_args['s'] = offset + + if params['language'] and params['language'] != 'all': + query_args['qlangcountry'] = params['language'] + query_args['qlang'] = params['language'].split('-')[0] if params['safesearch'] >= 1: - safesearch = 1 - else: - safesearch = 0 + query_args['ff'] = 1 - # rxieu is some kind of hash from the search query, but accepts random atm - search_path = search_string.format(query=urlencode({'q': query}), - offset=offset, - number_of_results=number_of_results, - lang=language, - rxikd=int(time() * 1000), - safesearch=safesearch) - - params['url'] = base_url + search_path + '&' + extra_param + search_url = 'search?' + urlencode(query_args) + params['url'] = base_url + search_url return params - # get response from search-request def response(resp): results = [] - # parse results - try: - response_json = loads(resp.text) - except: - parse_extra_param(resp.text) - raise Exception('extra param expired, please reload') - + response_json = loads(resp.text) for result in response_json['results']: - # append result - results.append({'url': result['url'], - 'title': result['title'], - 'content': result['sum']}) + # see "Example JSON Output (&format=json)" + # at http://www.gigablast.com/api.html#/search + + # sort out meaningless result + + title = result.get('title') + if len(title) < 2: + continue + + url = result.get('url') + if len(url) < 9: + continue + + content = result.get('sum') + if len(content) < 5: + continue + + # extend fields + + subtitle = result.get('title') + if len(subtitle) > 3: + title += " - " + subtitle + + results.append(dict( + url = url + , title = title + , content = content + )) - # return results return results From d44b4eda645446293cae664b8df823181a432c2e Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Fri, 29 May 2020 08:52:11 +0200 Subject: [PATCH 08/30] [fix] gigablaste engine: switch from pep8 to pylint test Signed-off-by: Markus Heiser --- Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index abd0eddb..494ff401 100644 --- a/Makefile +++ b/Makefile @@ -195,6 +195,7 @@ test.pylint: pyenvinstall $(call cmd,pylint,\ searx/preferences.py \ searx/testing.py \ + searx/engines/gigablast.py \ ) endif @@ -204,7 +205,7 @@ endif test.pep8: pyenvinstall @echo "TEST pep8" - $(Q)$(PY_ENV_ACT); pep8 --exclude=searx/static --max-line-length=120 --ignore "E402,W503" searx tests + $(Q)$(PY_ENV_ACT); pep8 --exclude='searx/static, searx/engines/gigablast.py' --max-line-length=120 --ignore "E402,W503" searx tests test.unit: pyenvinstall @echo "TEST tests/unit" From ee5d2b319b878e34308cf0761375ca1481edf473 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Fri, 29 May 2020 12:48:58 +0200 Subject: [PATCH 09/30] [fix] gigablast requires a random extra parameter Signed-off-by: Markus Heiser --- searx/engines/gigablast.py | 63 +++++++++++++++++++++++++++----------- 1 file changed, 45 insertions(+), 18 deletions(-) diff --git a/searx/engines/gigablast.py b/searx/engines/gigablast.py index 37221c55..b139c2a9 100644 --- a/searx/engines/gigablast.py +++ b/searx/engines/gigablast.py @@ -1,3 +1,4 @@ +# SPDX-License-Identifier: AGPL-3.0-or-later """ Gigablast (Web) @@ -11,20 +12,52 @@ """ # pylint: disable=missing-function-docstring, invalid-name -from time import time +import re from json import loads +# from searx import logger from searx.url_utils import urlencode +from searx.poolrequests import get # engine dependent config categories = ['general'] -paging = True -number_of_results = 10 +# gigablast's pagination is totally damaged, don't use it +paging = False language_support = True safesearch = True # search-url +base_url = 'https://gigablast.com' + +# ugly hack: gigablast requires a random extra parameter which can be extracted +# from the source code of the gigablast HTTP client +extra_param = '' +extra_param_path='/search?c=main&qlangcountry=en-us&q=south&s=10' + +def parse_extra_param(text): + + # example: + # + # var uxrl='/search?c=main&qlangcountry=en-us&q=south&s=10&rand=1590740241635&n'; + # uxrl=uxrl+'sab=730863287'; + # + # extra_param --> "rand=1590740241635&nsab=730863287" + + global extra_param # pylint: disable=global-statement + re_var= None + for line in text.splitlines(): + if re_var is None and extra_param_path in line: + var = line.split("=")[0].split()[1] # e.g. var --> 'uxrl' + re_var = re.compile(var + "\\s*=\\s*" + var + "\\s*\\+\\s*'" + "(.*)" + "'(.*)") + extra_param = line.split("'")[1][len(extra_param_path):] + continue + if re_var is not None and re_var.search(line): + extra_param += re_var.search(line).group(1) + break + # logger.debug('gigablast extra_param="%s"', extra_param) + +def init(engine_settings=None): # pylint: disable=unused-argument + parse_extra_param(get(base_url + extra_param_path).text) -base_url = 'https://gigablast.com/' # do search-request def request(query, params): # pylint: disable=unused-argument @@ -34,21 +67,12 @@ def request(query, params): # pylint: disable=unused-argument query_args = dict( c = 'main' - , n = number_of_results , format = 'json' , q = query - # The gigablast HTTP client sends a random number and a nsga argument - # (the values don't seem to matter) - , rand = int(time() * 1000) - , nsga = int(time() * 1000) + , dr = 1 + , showgoodimages = 0 ) - page_no = (params['pageno'] - 1) - if page_no: - # API quirk; adds +2 to the number_of_results - offset = (params['pageno'] - 1) * number_of_results - query_args['s'] = offset - if params['language'] and params['language'] != 'all': query_args['qlangcountry'] = params['language'] query_args['qlang'] = params['language'].split('-')[0] @@ -56,8 +80,8 @@ def request(query, params): # pylint: disable=unused-argument if params['safesearch'] >= 1: query_args['ff'] = 1 - search_url = 'search?' + urlencode(query_args) - params['url'] = base_url + search_url + search_url = '/search?' + urlencode(query_args) + params['url'] = base_url + search_url + extra_param return params @@ -66,6 +90,9 @@ def response(resp): results = [] response_json = loads(resp.text) + + # logger.debug('gigablast returns %s results', len(response_json['results'])) + for result in response_json['results']: # see "Example JSON Output (&format=json)" # at http://www.gigablast.com/api.html#/search @@ -87,7 +114,7 @@ def response(resp): # extend fields subtitle = result.get('title') - if len(subtitle) > 3: + if len(subtitle) > 3 and subtitle != title: title += " - " + subtitle results.append(dict( From 385e9b5c9e2d1caa73f99dac0bf1be1c46505121 Mon Sep 17 00:00:00 2001 From: Gordon Quad Date: Sat, 13 Jun 2020 22:42:16 +0100 Subject: [PATCH 10/30] add correction support for duckduckgo --- searx/engines/duckduckgo.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/searx/engines/duckduckgo.py b/searx/engines/duckduckgo.py index 0d2c0af2..6e07b502 100644 --- a/searx/engines/duckduckgo.py +++ b/searx/engines/duckduckgo.py @@ -50,6 +50,7 @@ result_xpath = '//div[@class="result results_links results_links_deep web-result url_xpath = './/a[@class="result__a"]/@href' title_xpath = './/a[@class="result__a"]' content_xpath = './/a[@class="result__snippet"]' +correction_xpath = '//div[@id="did_you_mean"]//a' # match query's language to a region code that duckduckgo will accept @@ -125,6 +126,11 @@ def response(resp): 'content': content, 'url': res_url}) + # parse correction + for correction in eval_xpath(doc, correction_xpath): + # append correction + results.append({'correction': extract_text(correction)}) + # return results return results From 4c7b7870044e17167e1f3025bc192a492b908be9 Mon Sep 17 00:00:00 2001 From: Dalf Date: Mon, 22 Jun 2020 13:57:33 +0200 Subject: [PATCH 11/30] [mod] don't try to proxify data URL. Previously only image/jpeg was not proxied. This commit don't proxify all MIME types starting with "image/". This is a quick fix for the PR #1985 : the google_image engine can returns some data URL. --- searx/webapp.py | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/searx/webapp.py b/searx/webapp.py index 35495a0f..e1b6bea1 100755 --- a/searx/webapp.py +++ b/searx/webapp.py @@ -335,8 +335,15 @@ def image_proxify(url): if not request.preferences.get_value('image_proxy'): return url - if url.startswith('data:image/jpeg;base64,'): - return url + if url.startswith('data:image/'): + # 50 is an arbitrary number to get only the beginning of the image. + partial_base64 = url[len('data:image/'):50].split(';') + if len(partial_base64) == 2 \ + and partial_base64[0] in ['gif', 'png', 'jpeg', 'pjpeg', 'webp', 'tiff', 'bmp']\ + and partial_base64[1].startswith('base64,'): + return url + else: + return None if settings.get('result_proxy'): return proxify(url) From cc84566e0a828b84f81def47d710e2519469fb89 Mon Sep 17 00:00:00 2001 From: Adam Tauber Date: Tue, 23 Jun 2020 21:43:32 +0200 Subject: [PATCH 12/30] [enh] add visually hidden but accessible css class - #350 --- searx/static/themes/oscar/less/logicodev/search.less | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/searx/static/themes/oscar/less/logicodev/search.less b/searx/static/themes/oscar/less/logicodev/search.less index d65e3056..ff94bfef 100644 --- a/searx/static/themes/oscar/less/logicodev/search.less +++ b/searx/static/themes/oscar/less/logicodev/search.less @@ -77,4 +77,14 @@ Ny0yNFQxMToxNTowMCswMjowMP7RDgQAAAAZdEVYdFNvZnR3YXJlAHd3dy5pbmtzY2FwZS5vcmeb .search-margin { margin-bottom: 0.6em; -} \ No newline at end of file +} + +.visually-hidden { + position: absolute !important; + height: 1px; + width: 1px; + overflow: hidden; + clip: rect(1px 1px 1px 1px); /* IE6, IE7 */ + clip: rect(1px, 1px, 1px, 1px); + white-space: nowrap; /* added line */ +} From 2ff6214a4d6909867ddc176fcaf89a28521b7af7 Mon Sep 17 00:00:00 2001 From: Adam Tauber Date: Tue, 23 Jun 2020 21:44:13 +0200 Subject: [PATCH 13/30] [fix] remove alt text which doesnt add extra info - #350 --- searx/templates/oscar/infobox.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/searx/templates/oscar/infobox.html b/searx/templates/oscar/infobox.html index 04f2d5f2..6ae7965b 100644 --- a/searx/templates/oscar/infobox.html +++ b/searx/templates/oscar/infobox.html @@ -5,7 +5,7 @@ {% for u in infobox.urls %}{% if u.official %} {{ u.domain }}{% endif %}{% endfor %}
- {% if infobox.img_src %}{{ infobox.infobox }}{% endif %} + {% if infobox.img_src %}{% endif %} {% if infobox.content %}

{{ infobox.content | safe }}

{% endif %} From a984afd6a32ec3bd5011659362afaf81a4bc787e Mon Sep 17 00:00:00 2001 From: Adam Tauber Date: Tue, 23 Jun 2020 21:44:44 +0200 Subject: [PATCH 14/30] [enh] make checkboxes in preferences accessible using keyboard - #350 --- .../themes/oscar/css/logicodev-dark.css | 32 ++++++++++++++++-- .../themes/oscar/css/logicodev-dark.min.css | Bin 13417 -> 13864 bytes searx/static/themes/oscar/css/logicodev.css | 32 ++++++++++++++++-- .../static/themes/oscar/css/logicodev.min.css | Bin 9977 -> 10424 bytes .../themes/oscar/less/logicodev/onoff.less | 21 ++++++++++-- searx/templates/oscar/macros.html | 1 + 6 files changed, 80 insertions(+), 6 deletions(-) diff --git a/searx/static/themes/oscar/css/logicodev-dark.css b/searx/static/themes/oscar/css/logicodev-dark.css index b2cf38ba..38b9dbaa 100644 --- a/searx/static/themes/oscar/css/logicodev-dark.css +++ b/searx/static/themes/oscar/css/logicodev-dark.css @@ -70,7 +70,21 @@ input[type=checkbox]:not(:checked) + .label_hide_if_checked + .label_hide_if_not -ms-user-select: none; } .onoffswitch-checkbox { - display: none; + opacity: 0; + position: absolute; +} +.onoffswitch-checkbox:before { + content: ""; + display: inline-block; + width: 16px; + height: 16px; + margin-right: 10px; + position: absolute; + left: 0; + bottom: 1px; + background-color: #fff; + border: 1px solid #ccc; + border-radius: 0px; } .onoffswitch-label { display: block; @@ -104,7 +118,7 @@ input[type=checkbox]:not(:checked) + .label_hide_if_checked + .label_hide_if_not top: 0; bottom: 0; right: 0px; - border: 2px solid #FFFFFF !important; + border: 2px solid #FFFFFF; border-radius: 50px !important; transition: all 0.3s ease-in 0s; } @@ -115,6 +129,9 @@ input[type=checkbox]:not(:checked) + .label_hide_if_checked + .label_hide_if_not right: 71px; background-color: #A1A1A1; } +.onoffswitch-checkbox:focus + .onoffswitch-label .onoffswitch-switch { + border: 3px solid #444444; +} .result_header { margin-top: 0px; margin-bottom: 2px; @@ -377,6 +394,17 @@ Ny0yNFQxMToxNTowMCswMjowMP7RDgQAAAAZdEVYdFNvZnR3YXJlAHd3dy5pbmtzY2FwZS5vcmeb .search-margin { margin-bottom: 0.6em; } +.visually-hidden { + position: absolute !important; + height: 1px; + width: 1px; + overflow: hidden; + clip: rect(1px 1px 1px 1px); + /* IE6, IE7 */ + clip: rect(1px, 1px, 1px, 1px); + white-space: nowrap; + /* added line */ +} #advanced-search-container { display: none; text-align: left; diff --git a/searx/static/themes/oscar/css/logicodev-dark.min.css b/searx/static/themes/oscar/css/logicodev-dark.min.css index 603aedfa9a3def13baea1426fb6eb67548f744d8..6c7cf749a01ee88e35c7003fe2c28cacf896ccd6 100644 GIT binary patch delta 426 zcmZWmO-lnY5XH*UgQp%!L95pG(qdOyEFQAILhuh{lWccrnh8m^YbnxSAc*-BotvxFs1S8Z9_LMQCbK7EbtwsI^W-hDNLy zDkqd9-EP9cYDt#_lmsQ>R3e)tC2*IM@s++!askuaRsEcrDX7>~lTl#*T}hmZVycFbSF%!CZf5_!kC$sbQdo?Oh2NBxMdx#5Vjw5QU|o`p)ki_BOh;ZGQr) CL!4{? delta 52 zcmV-40L%ZVZ0TsQ)LFVR<@kZ*FCMlc6yav)Tjm0h1gACzI+0+_N7CQW2A4 K85*-D83G+vG87O1 diff --git a/searx/static/themes/oscar/css/logicodev.css b/searx/static/themes/oscar/css/logicodev.css index d5a2aa6a..ad6d967b 100644 --- a/searx/static/themes/oscar/css/logicodev.css +++ b/searx/static/themes/oscar/css/logicodev.css @@ -43,7 +43,21 @@ input[type=checkbox]:not(:checked) + .label_hide_if_checked + .label_hide_if_not -ms-user-select: none; } .onoffswitch-checkbox { - display: none; + opacity: 0; + position: absolute; +} +.onoffswitch-checkbox:before { + content: ""; + display: inline-block; + width: 16px; + height: 16px; + margin-right: 10px; + position: absolute; + left: 0; + bottom: 1px; + background-color: #fff; + border: 1px solid #ccc; + border-radius: 0px; } .onoffswitch-label { display: block; @@ -77,7 +91,7 @@ input[type=checkbox]:not(:checked) + .label_hide_if_checked + .label_hide_if_not top: 0; bottom: 0; right: 0px; - border: 2px solid #FFFFFF !important; + border: 2px solid #FFFFFF; border-radius: 50px !important; transition: all 0.3s ease-in 0s; } @@ -88,6 +102,9 @@ input[type=checkbox]:not(:checked) + .label_hide_if_checked + .label_hide_if_not right: 71px; background-color: #A1A1A1; } +.onoffswitch-checkbox:focus + .onoffswitch-label .onoffswitch-switch { + border: 3px solid #444444; +} .result_header { margin-top: 0px; margin-bottom: 2px; @@ -350,6 +367,17 @@ Ny0yNFQxMToxNTowMCswMjowMP7RDgQAAAAZdEVYdFNvZnR3YXJlAHd3dy5pbmtzY2FwZS5vcmeb .search-margin { margin-bottom: 0.6em; } +.visually-hidden { + position: absolute !important; + height: 1px; + width: 1px; + overflow: hidden; + clip: rect(1px 1px 1px 1px); + /* IE6, IE7 */ + clip: rect(1px, 1px, 1px, 1px); + white-space: nowrap; + /* added line */ +} #advanced-search-container { display: none; text-align: left; diff --git a/searx/static/themes/oscar/css/logicodev.min.css b/searx/static/themes/oscar/css/logicodev.min.css index 7f221d5eca4dd5e3063ea6d914ea37bb212992a3..a39cc36b927c6f7ccc11ecc2b00844f15b1460c3 100644 GIT binary patch delta 412 zcmZWl%}N6?5Qb7)@K8}K{?tQl5nAX{Eho+g3&30Rg^da&hLN7ju z@8C1^AR1W_rH2`Y`99|RzS-Wr-oHFlTWA;uwy3L zYA3ZLoemX{8O4?av;r-oSRtR%0)$J+=+-P~Dq)hka-K0e0UgAT zmNar>7VVG(M7uj4k2gvE@o;@Thb(7GEu$2Kkb1S_=MrWHtz+8#CJB~)@`VZK(k2QE yQXZgOfgx6M*9+F$f2#i<8U$!?@!dl8X_11HQ5L>0iN?Y*^Xx}w=bN|FtHu| +
{%- endmacro %} From 18805ee35296e6e34be59f37083190990f4afcbb Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Wed, 24 Jun 2020 15:15:15 +0200 Subject: [PATCH 15/30] [fix] markup of auto generated installation instruction (uWSGI part) Signed-off-by: Markus Heiser --- docs/admin/installation-uwsgi.rst | 11 +++--- utils/searx.sh | 66 +++++++++++++++++++------------ 2 files changed, 46 insertions(+), 31 deletions(-) diff --git a/docs/admin/installation-uwsgi.rst b/docs/admin/installation-uwsgi.rst index ac4c463b..7b482975 100644 --- a/docs/admin/installation-uwsgi.rst +++ b/docs/admin/installation-uwsgi.rst @@ -108,6 +108,7 @@ restart the uwsgi application. :start-after: START searx uwsgi-description ubuntu-20.04 :end-before: END searx uwsgi-description ubuntu-20.04 + .. hotfix: a bug group-tab need this comment .. group-tab:: Arch Linux @@ -115,6 +116,7 @@ restart the uwsgi application. :start-after: START searx uwsgi-description arch :end-before: END searx uwsgi-description arch + .. hotfix: a bug group-tab need this comment .. group-tab:: Fedora / RHEL @@ -128,22 +130,21 @@ restart the uwsgi application. .. group-tab:: Ubuntu / debian .. kernel-include:: $DOCS_BUILD/includes/searx.rst - :code: ini :start-after: START searx uwsgi-appini ubuntu-20.04 :end-before: END searx uwsgi-appini ubuntu-20.04 + .. hotfix: a bug group-tab need this comment + .. group-tab:: Arch Linux .. kernel-include:: $DOCS_BUILD/includes/searx.rst - :code: ini :start-after: START searx uwsgi-appini arch :end-before: END searx uwsgi-appini arch + .. hotfix: a bug group-tab need this comment + .. group-tab:: Fedora / RHEL .. kernel-include:: $DOCS_BUILD/includes/searx.rst - :code: ini :start-after: START searx uwsgi-appini fedora :end-before: END searx uwsgi-appini fedora - - diff --git a/utils/searx.sh b/utils/searx.sh index 4009fddb..886b38f8 100755 --- a/utils/searx.sh +++ b/utils/searx.sh @@ -816,47 +816,61 @@ rst-doc() { case $DIST_ID-$DIST_VERS in ubuntu-*|debian-*) cat < /usr/share/doc/uwsgi/README.Debian.gz -# For uWSGI debian uses the LSB init process, this might be changed -# one day, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=833067 -create ${uWSGI_APPS_AVAILABLE}/${SEARX_UWSGI_APP} -enable: sudo -H ln -s ${uWSGI_APPS_AVAILABLE}/${SEARX_UWSGI_APP} ${uWSGI_APPS_ENABLED}/ -start: sudo -H service uwsgi start ${SEARX_UWSGI_APP%.*} -restart: sudo -H service uwsgi restart ${SEARX_UWSGI_APP%.*} -stop: sudo -H service uwsgi stop ${SEARX_UWSGI_APP%.*} -disable: sudo -H rm ${uWSGI_APPS_ENABLED}/${SEARX_UWSGI_APP} +.. code:: bash + + # init.d --> /usr/share/doc/uwsgi/README.Debian.gz + # For uWSGI debian uses the LSB init process, this might be changed + # one day, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=833067 + + create ${uWSGI_APPS_AVAILABLE}/${SEARX_UWSGI_APP} + enable: sudo -H ln -s ${uWSGI_APPS_AVAILABLE}/${SEARX_UWSGI_APP} ${uWSGI_APPS_ENABLED}/ + start: sudo -H service uwsgi start ${SEARX_UWSGI_APP%.*} + restart: sudo -H service uwsgi restart ${SEARX_UWSGI_APP%.*} + stop: sudo -H service uwsgi stop ${SEARX_UWSGI_APP%.*} + disable: sudo -H rm ${uWSGI_APPS_ENABLED}/${SEARX_UWSGI_APP} + EOF ;; arch-*) cat < /usr/lib/systemd/system/uwsgi@.service -# For uWSGI archlinux uses systemd template units, see -# - http://0pointer.de/blog/projects/instances.html -# - https://uwsgi-docs.readthedocs.io/en/latest/Systemd.html#one-service-per-app-in-systemd -create: ${uWSGI_APPS_ENABLED}/${SEARX_UWSGI_APP} -enable: sudo -H systemctl enable uwsgi@${SEARX_UWSGI_APP%.*} -start: sudo -H systemctl start uwsgi@${SEARX_UWSGI_APP%.*} -restart: sudo -H systemctl restart uwsgi@${SEARX_UWSGI_APP%.*} -stop: sudo -H systemctl stop uwsgi@${SEARX_UWSGI_APP%.*} -disable: sudo -H systemctl disable uwsgi@${SEARX_UWSGI_APP%.*} +.. code:: bash + + # systemd --> /usr/lib/systemd/system/uwsgi@.service + # For uWSGI archlinux uses systemd template units, see + # - http://0pointer.de/blog/projects/instances.html + # - https://uwsgi-docs.readthedocs.io/en/latest/Systemd.html#one-service-per-app-in-systemd + + create: ${uWSGI_APPS_ENABLED}/${SEARX_UWSGI_APP} + enable: sudo -H systemctl enable uwsgi@${SEARX_UWSGI_APP%.*} + start: sudo -H systemctl start uwsgi@${SEARX_UWSGI_APP%.*} + restart: sudo -H systemctl restart uwsgi@${SEARX_UWSGI_APP%.*} + stop: sudo -H systemctl stop uwsgi@${SEARX_UWSGI_APP%.*} + disable: sudo -H systemctl disable uwsgi@${SEARX_UWSGI_APP%.*} + EOF ;; fedora-*) cat < /usr/lib/systemd/system/uwsgi.service -# The unit file starts uWSGI in emperor mode (/etc/uwsgi.ini), see -# - https://uwsgi-docs.readthedocs.io/en/latest/Emperor.html -create: ${uWSGI_APPS_ENABLED}/${SEARX_UWSGI_APP} -restart: sudo -H touch ${uWSGI_APPS_ENABLED}/${SEARX_UWSGI_APP} -disable: sudo -H rm ${uWSGI_APPS_ENABLED}/${SEARX_UWSGI_APP} +.. code:: bash + + # systemd --> /usr/lib/systemd/system/uwsgi.service + # The unit file starts uWSGI in emperor mode (/etc/uwsgi.ini), see + # - https://uwsgi-docs.readthedocs.io/en/latest/Emperor.html + + create: ${uWSGI_APPS_ENABLED}/${SEARX_UWSGI_APP} + restart: sudo -H touch ${uWSGI_APPS_ENABLED}/${SEARX_UWSGI_APP} + disable: sudo -H rm ${uWSGI_APPS_ENABLED}/${SEARX_UWSGI_APP} + EOF ;; esac echo -e ".. END searx uwsgi-description $DIST_NAME" echo -e "\n.. START searx uwsgi-appini $DIST_NAME" - eval "echo \"$(< "${TEMPLATES}/${uWSGI_APPS_AVAILABLE}/${SEARX_UWSGI_APP}")\"" + echo ".. code:: bash" + echo + eval "echo \"$(< "${TEMPLATES}/${uWSGI_APPS_AVAILABLE}/${SEARX_UWSGI_APP}")\"" | prefix_stdout " " echo -e "\n.. END searx uwsgi-appini $DIST_NAME" ) From 9c71f620a7c29ea22e4f1743e733bd267cdb4b69 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Wed, 24 Jun 2020 17:49:38 +0200 Subject: [PATCH 16/30] [add blog post] Developing in Linux containers [202006] Signed-off-by: Markus Heiser --- docs/blog/index.rst | 1 + docs/blog/lxcdev-202006.rst | 413 ++++++++++++++++++++++++++++++++++++ docs/utils/lxc.sh.rst | 7 +- 3 files changed, 418 insertions(+), 3 deletions(-) create mode 100644 docs/blog/lxcdev-202006.rst diff --git a/docs/blog/index.rst b/docs/blog/index.rst index 5a8135fd..04d95350 100644 --- a/docs/blog/index.rst +++ b/docs/blog/index.rst @@ -6,6 +6,7 @@ Blog :maxdepth: 2 :caption: Contents + lxcdev-202006 python3 admin intro-offline diff --git a/docs/blog/lxcdev-202006.rst b/docs/blog/lxcdev-202006.rst new file mode 100644 index 00000000..f9ca3c2c --- /dev/null +++ b/docs/blog/lxcdev-202006.rst @@ -0,0 +1,413 @@ +.. _blog-lxcdev-202006: + +======================================= +Developing in Linux containers [202006] +======================================= + +.. _LXC: https://linuxcontainers.org/lxc/introduction/ + +.. sidebar:: Audience + + This blog post is written for experienced admins and developers / readers + should have a serious meaning about: *distributed*, *merge* and *linux + container*. + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +In PR :PR:`1803` we added a lot of scripts to Searx's boilerplate. In this blog +post I will show you, how you can make use of them in *distributed and +heterogeneous development cycles* (TL;DR; jump to the :ref:`blog-lxcdev-202006 +abstract`). + +Motivation +========== + +Normally in our development cycle, we edit the sources and run some test and/or +builds by using ``make`` before we commit. This cycle is simple and perfect but +might fail in some aspects we should not overlook. + + The environment in which we run all our development processes matters! + +The :ref:`makefile` and the :ref:`make pyenv` encapsulate a lot for us, but they +do not have access to all prerequisites. For example, there may have +dependencies on packages that are installed on the developer's desktop, but +usually are not preinstalled on a server or client system. Another examples +are; settings have been made to the software on the developer's host that would +never be set on a *production* system. + +*Linux Containers* (LXC_) are isolate environments and not to mix up on +developer's all the prerequisites of all the projects he contribute to, is +always a good choice. + +The scripts from PR :PR:`1803` can divide in those to install and maintain +software: + +- :ref:`searx.sh` +- :ref:`filtron.sh` +- :ref:`morty.sh` + +and the script :ref:`lxc.sh`, with we can scale our installation, maintenance or +even development tasks over a stack of containers, what we call: *Searx's lxc +suite*. + +Gentlemen, start your engines! +============================== + +.. _LXD: https://linuxcontainers.org/lxd/introduction/ +.. _archlinux: https://www.archlinux.org/ + +Before you can start with containers, you need to install and initiate LXD_ +once: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: sh + + $ snap install lxd + $ lxd init --auto + +And you need to clone from origin or if you have your own fork, clone from your +fork: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: sh + + $ cd ~/Downloads + $ git clone https://github.com/asciimoo/searx.git + $ cd searx + +The :ref:`lxc-searx.env` consists of several images, see ``export +LXC_SUITE=(...`` near by :origin:`utils/lxc-searx.env#L19`. For this blog post +we exercise on a archlinux_ image. The container of this image is named +``searx-archlinux``. Lets build the container, but be sure that this container +does not already exists, so first lets remove possible old one: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: sh + + $ sudo -H ./utils/lxc.sh remove searx-archlinux + $ sudo -H ./utils/lxc.sh build searx-archlinux + +.. sidebar:: The ``searx-archlinux`` container + + is the base of all our exercises here. + +In this container we install all services :ref:`including searx, morty & filtron +` in once: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: sh + + $ sudo -H ./utils/lxc.sh install suite searx-archlinux + +To proxy HTTP from filtron and morty in the container to the outside of the +container, install nginx into the container. Once for the bot blocker filtron: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: sh + + $ sudo -H ./utils/lxc.sh cmd searx-archlinux \ + ./utils/filtron.sh nginx install + ... + INFO: got 429 from http://10.174.184.156/searx + +and once for the content sanitizer (content proxy morty): + +.. tabs:: + + .. group-tab:: desktop + + .. code:: sh + + $ sudo -H ./utils/lxc.sh cmd searx-archlinux \ + ./utils/morty.sh nginx install + ... + INFO: got 200 from http://10.174.184.156/morty/ + +.. sidebar:: Fully functional searx suite + + From here on you have a fully functional searx suite running with bot blocker + (filtron) and Web content sanitizer (content proxy morty) needed for a + *privacy protecting* search engine. + +On your system, the IP of your ``searx-archlinux`` container differs from +http://10.174.184.156/searx, just open the URL reported in your installation +protocol in your WEB browser from the desktop to test the instance from outside +of the container. + +In such a searx suite admins can maintain and access the debug log of the +different services quite easy. + +.. _working in containers: + +In containers, work as usual +============================ + +Usually you open a root-bash using ``sudo -H bash``. In case of LXC containers +open the root-bash in the container using ``./utils/lxc.sh cmd +searx-archlinux``: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: sh + + $ sudo -H ./utils/lxc.sh cmd searx-archlinux bash + INFO: [searx-archlinux] bash + [root@searx-archlinux searx]# pwd + /share/searx + +The prompt ``[root@searx-archlinux ...]`` signals, that you are the root user in +the searx-container. To debug the running searx instance use: + +.. tabs:: + + .. group-tab:: root@searx-archlinux + + .. code:: sh + + $ ./utils/searx.sh inspect service + ... + use [CTRL-C] to stop monitoring the log + ... + +Back in the browser on your desktop open the service http://10.174.184.156/searx +and run your application tests while the debug log is shown in the terminal from +above. You can stop monitoring using ``CTRL-C``, this also disables the *"debug +option"* in searx's settings file and restarts the searx uwsgi application. To +debug services from filtron and morty analogous use: + +.. tabs:: + + .. group-tab:: root@searx-archlinux + + .. code:: sh + + $ ./utils/filtron.sh inspect service + $ ./utils/morty.sh inspect service + +Another point we have to notice is that each service (:ref:`searx `, +:ref:`filtron ` and :ref:`morty `) runs under dedicated +system user account with the same name (compare :ref:`create searx user`). To +get a shell from theses accounts, simply call one of the scripts: + +.. tabs:: + + .. group-tab:: root@searx-archlinux + + .. code:: sh + + $ ./utils/searx.sh shell + $ ./utils/filtron.sh shell + $ ./utils/morty.sh shell + +To get in touch, open a shell from the service user (searx@searx-archlinux): + +.. tabs:: + + .. group-tab:: desktop + + .. code:: sh + + $ sudo -H ./utils/lxc.sh cmd searx-archlinux \ + ./utils/searx.sh shell + // exit with [CTRL-D] + (searx-pyenv) [searx@searx-archlinux ~]$ ... + +The prompt ``[searx@searx-archlinux]`` signals that you are logged in as system +user ``searx`` in the ``searx-archlinux`` container and the python *virtualenv* +``(searx-pyenv)`` environment is activated. + +.. tabs:: + + .. group-tab:: searx@searx-archlinux + + .. code:: sh + + (searx-pyenv) [searx@searx-archlinux ~]$ pwd + /usr/local/searx + + + +Wrap production into developer suite +==================================== + +In this section we will see how to change the *"Fully functional searx suite"* +from a LXC container (which is quite ready for production) into a developer +suite. For this, we have to keep an eye on the :ref:`installation basic`: + +- searx setup in: ``/etc/searx/settings.yml`` +- searx user's home: ``/usr/local/searx`` +- virtualenv in: ``/usr/local/searx/searx-pyenv`` +- searx software in: ``/usr/local/searx/searx-src`` + +The searx software is a clone of the ``GIT_URL`` (see :ref:`makefile setup`) and +the working tree is checked out from the ``GIT_BRANCH``. With the use of the +:ref:`searx.sh` the searx service was installed as :ref:`uWSGI application +`. To maintain this service, we can use ``systemctl`` (compare +:ref:`service architectures on distributions `). + +.. tabs:: + + .. group-tab:: desktop + + .. code:: sh + + $ sudo -H ./utils/lxc.sh cmd searx-archlinux \ + systemctl stop uwsgi@searx + +With the command above, we stopped the searx uWSGI-App in the archlinux +container. + +The uWSGI-App for the archlinux dsitros is configured in +:origin:`utils/templates/etc/uwsgi/apps-archlinux/searx.ini`, from where at +least you should attend the settings of ``uid``, ``chdir``, ``env`` and +``http``:: + + env = SEARX_SETTINGS_PATH=/etc/searx/settings.yml + http = 127.0.0.1:8888 + + chdir = /usr/local/searx/searx-src/searx + virtualenv = /usr/local/searx/searx-pyenv + pythonpath = /usr/local/searx/searx-src + +If you have read the :ref:`"Good to know section" ` you remember, that +each container shares the root folder of the repository and the command +``utils/lxc.sh cmd`` handles relative path names **transparent**. To wrap the +searx installation into a developer one, we simple have to create a smylink to +the **transparent** reposetory from the desktop. Now lets replace the +repository at ``searx-src`` in the container with the working tree from outside +of the container: + +.. tabs:: + + .. group-tab:: container becomes a developer suite + + .. code:: sh + + $ sudo -H ./utils/lxc.sh cmd searx-archlinux \ + mv /usr/local/searx/searx-src /usr/local/searx/searx-src.old + + $ sudo -H ./utils/lxc.sh cmd searx-archlinux \ + ln -s /share/searx/ /usr/local/searx/searx-src + +Now we can develop as usual in the working tree of our desktop system. Every +time the software was changed, you have to restart the searx service (in the +conatiner): + +.. tabs:: + + .. group-tab:: desktop + + .. code:: sh + + $ sudo -H ./utils/lxc.sh cmd searx-archlinux \ + systemctl restart uwsgi@searx + + +Remember: :ref:`working in containers` .. here are just some examples from my +daily usage: + +.. tabs:: + + .. group-tab:: desktop + + To *inspect* the searx instance (already described above): + + .. code:: sh + + $ sudo -H ./utils/lxc.sh cmd searx-archlinux \ + ./utils/searx.sh inspect service + + Run :ref:`makefile`, e.g. to test inside the container: + + .. code:: sh + + $ sudo -H ./utils/lxc.sh cmd searx-archlinux \ + make test + + To install all prerequisites needed for a :ref:`buildhosts`: + + .. code:: sh + + $ sudo -H ./utils/lxc.sh cmd searx-archlinux \ + ./utils/searx.sh install buildhost + + To build the docs on a buildhost :ref:`buildhosts`: + + .. code:: sh + + $ sudo -H ./utils/lxc.sh cmd searx-archlinux \ + make docs + +.. _blog-lxcdev-202006 abstract: + +Abstract +======== + +We build up a fully functional searx suite in a archlinux container: + +.. code:: sh + + $ sudo -H ./utils/lxc.sh install suite searx-archlinux + +To access HTTP from the desktop we installed nginx for the services inside the +conatiner: + +.. tabs:: + + .. group-tab:: [root@searx-archlinux] + + .. code:: sh + + $ ./utils/filtron.sh nginx install + $ ./utils/morty.sh nginx install + +To wrap the suite into a developer one, we created a symbolic link to the +repository which is shared **transparent** from the desktop's file system into +the container : + +.. tabs:: + + .. group-tab:: [root@searx-archlinux] + + .. code:: sh + + $ mv /usr/local/searx/searx-src /usr/local/searx/searx-src.old + $ ln -s /share/searx/ /usr/local/searx/searx-src + $ systemctl restart uwsgi@searx + +To get remarks from the suite of the archlinux container we can use: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: sh + + $ sudo -H ./utils/lxc.sh show suite searx-archlinux + ... + [searx-archlinux] INFO: (eth0) filtron: http://10.174.184.156:4004/ http://10.174.184.156/searx + [searx-archlinux] INFO: (eth0) morty: http://10.174.184.156:3000/ + [searx-archlinux] INFO: (eth0) docs-live: http://10.174.184.156:8080/ + [searx-archlinux] INFO: (eth0) IPv6: http://[fd42:573b:e0b3:e97e:216:3eff:fea5:9b65] + ... + diff --git a/docs/utils/lxc.sh.rst b/docs/utils/lxc.sh.rst index ae141287..114d0911 100644 --- a/docs/utils/lxc.sh.rst +++ b/docs/utils/lxc.sh.rst @@ -47,9 +47,9 @@ one**:: *Good to know ...* -Eeach container shares the root folder of the repository and the -command ``utils/lxc.sh cmd`` **handles relative path names transparent**, -compare output of:: +Each container shares the root folder of the repository and the command +``utils/lxc.sh cmd`` **handles relative path names transparent**, compare output +of:: $ sudo -H ./utils/lxc.sh cmd -- ls -la Makefile ... @@ -66,6 +66,7 @@ If there comes the time you want to **get rid off all** the containers and $ sudo -H ./utils/lxc.sh remove $ sudo -H ./utils/lxc.sh remove images +.. _lxc.sh install suite: Install suite ============= From 01ee3a6eddceeea2027d1489d550809ced65ab38 Mon Sep 17 00:00:00 2001 From: Adam Tauber Date: Wed, 24 Jun 2020 14:43:33 +0200 Subject: [PATCH 17/30] [fix] remove redundant alt texts from images in Videos, Social Media and Music results - #350 --- searx/templates/oscar/result_templates/default.html | 2 +- searx/templates/oscar/result_templates/videos.html | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/searx/templates/oscar/result_templates/default.html b/searx/templates/oscar/result_templates/default.html index 885cbbfa..57365f34 100644 --- a/searx/templates/oscar/result_templates/default.html +++ b/searx/templates/oscar/result_templates/default.html @@ -16,7 +16,7 @@ {%- if result.img_src -%}
-{{ result.title|striptags }} + {% if result.content %}

{{ result.content|safe }}

{% endif -%}
diff --git a/searx/templates/oscar/result_templates/videos.html b/searx/templates/oscar/result_templates/videos.html index 51ed5a02..6d9101d2 100644 --- a/searx/templates/oscar/result_templates/videos.html +++ b/searx/templates/oscar/result_templates/videos.html @@ -15,7 +15,7 @@
- {{ result.title|striptags }} {{ result.engine }} + {% if result.author %}

{{ _('Author') }}: {{ result.author }}

{% endif %} {% if result.length %}

{{ _('Length') }}: {{ result.length }}

{% endif %} {% if result.content %}

{{ result.content|safe }}

{% endif %} From d418b593bb76b52f6ee4c52d27a1b557853cdd62 Mon Sep 17 00:00:00 2001 From: Adam Tauber Date: Wed, 24 Jun 2020 14:44:27 +0200 Subject: [PATCH 18/30] [fix] do not use h1 element for the logo --- searx/templates/oscar/index.html | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/searx/templates/oscar/index.html b/searx/templates/oscar/index.html index b941f5f2..48c49d64 100644 --- a/searx/templates/oscar/index.html +++ b/searx/templates/oscar/index.html @@ -6,10 +6,10 @@ {% if cookies['oscar-style'] == 'pointhi' %}

searx logosearx

{% else %} -

+

+
{% endif %}
From 5ca52869123f40ad3cf991e78ad9baeaff52c4c2 Mon Sep 17 00:00:00 2001 From: Adam Tauber Date: Wed, 24 Jun 2020 15:00:33 +0200 Subject: [PATCH 19/30] [fix] indicate too high max time by e.g. icon not only by colour --- searx/templates/oscar/preferences.html | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/searx/templates/oscar/preferences.html b/searx/templates/oscar/preferences.html index 0d96ff5f..cd4070d0 100644 --- a/searx/templates/oscar/preferences.html +++ b/searx/templates/oscar/preferences.html @@ -204,11 +204,11 @@ {{ support_toggle(stats[search_engine.name].supports_selected_language) }} {{ support_toggle(search_engine.safesearch==True) }} {{ support_toggle(search_engine.time_range_support==True) }} - {{ 'N/A' if stats[search_engine.name].time==None else stats[search_engine.name].time }} - {{ search_engine.timeout }} - {% else %} - {{ search_engine.timeout }} - {{ 'N/A' if stats[search_engine.name].time==None else stats[search_engine.name].time }} + {% if stats[search_engine.name]['warn_time'] %}{{ icon('exclamation-sign')}} {% endif %}{{ 'N/A' if stats[search_engine.name].time==None else stats[search_engine.name].time }} + {% if stats[search_engine.name]['warn_timeout'] %}{{ icon('exclamation-sign') }} {% endif %}{{ search_engine.timeout }} + {% else %} + {{ search_engine.timeout }}{% if stats[search_engine.name]['warn_time'] %} {{ icon('exclamation-sign')}}{% endif %} + {{ 'N/A' if stats[search_engine.name].time==None else stats[search_engine.name].time }}{% if stats[search_engine.name]['warn_time'] %} {{ icon('exclamation-sign')}}{% endif %} {{ support_toggle(search_engine.time_range_support==True) }} {{ support_toggle(search_engine.safesearch==True) }} {{ support_toggle(stats[search_engine.name].supports_selected_language) }} From fce2d496514f4ddfa725e9796c5b70e37a4dec74 Mon Sep 17 00:00:00 2001 From: Adam Tauber Date: Thu, 25 Jun 2020 15:51:42 +0200 Subject: [PATCH 20/30] [fix] use scope attribute to describe properties in the engine preferences table --- searx/templates/oscar/preferences.html | 36 +++++++++++++------------- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/searx/templates/oscar/preferences.html b/searx/templates/oscar/preferences.html index cd4070d0..3466df00 100644 --- a/searx/templates/oscar/preferences.html +++ b/searx/templates/oscar/preferences.html @@ -173,23 +173,23 @@ {% if not rtl %} - - - - - - - - + + + + + + + + {% else %} - - - - - - - - + + + + + + + + {% endif %} {% for search_engine in engines_by_category[categ] %} @@ -199,7 +199,7 @@ - + @@ -213,7 +213,7 @@ - + From 6ab569f5588e148bf73a7d80bc2f52bc737bf5e0 Mon Sep 17 00:00:00 2001 From: Adam Tauber Date: Thu, 25 Jun 2020 17:32:05 +0200 Subject: [PATCH 21/30] [fix] connect selects and inputs programmatically on the preferences page - #350 --- searx/templates/oscar/macros.html | 6 ++-- searx/templates/oscar/preferences.html | 48 +++++++++++++------------- 2 files changed, 27 insertions(+), 27 deletions(-) diff --git a/searx/templates/oscar/macros.html b/searx/templates/oscar/macros.html index c4b2a7dd..809c9f59 100644 --- a/searx/templates/oscar/macros.html +++ b/searx/templates/oscar/macros.html @@ -60,15 +60,15 @@ {%- endif %} {%- endmacro %} -{% macro preferences_item_header(info, label, rtl) -%} +{% macro preferences_item_header(info, label, rtl, id) -%} {% if rtl %}
- + {{ info }}
{% else %}
- +
{% endif %} {%- endmacro %} diff --git a/searx/templates/oscar/preferences.html b/searx/templates/oscar/preferences.html index 3466df00..8d37b8be 100644 --- a/searx/templates/oscar/preferences.html +++ b/searx/templates/oscar/preferences.html @@ -30,9 +30,9 @@
{% include 'oscar/categories.html' %}
- + {% else %} - +
{% include 'oscar/categories.html' %}
@@ -40,14 +40,14 @@
{% set language_label = _('Search language') %} {% set language_info = _('What language do you prefer for search?') %} - {{ preferences_item_header(language_info, language_label, rtl) }} + {{ preferences_item_header(language_info, language_label, rtl, 'language') }} {% include 'oscar/languages.html' %} {{ preferences_item_footer(language_info, language_label, rtl) }} {% set locale_label = _('Interface language') %} {% set locale_info = _('Change the language of the layout') %} - {{ preferences_item_header(locale_info, locale_label, rtl) }} - {% for locale_id,locale_name in locales.items() | sort %} {% endfor %} @@ -56,8 +56,8 @@ {% set autocomplete_label = _('Autocomplete') %} {% set autocomplete_info = _('Find stuff as you type') %} - {{ preferences_item_header(autocomplete_info, autocomplete_label, rtl) }} - {% for backend in autocomplete_backends %} @@ -67,8 +67,8 @@ {% set image_proxy_label = _('Image proxy') %} {% set image_proxy_info = _('Proxying image results through searx') %} - {{ preferences_item_header(image_proxy_info, image_proxy_label, rtl) }} - @@ -76,8 +76,8 @@ {% set method_label = _('Method') %} {% set method_info = _('Change how forms are submited, learn more about request methods') %} - {{ preferences_item_header(method_info, method_label, rtl) }} - @@ -85,8 +85,8 @@ {% set safesearch_label = _('SafeSearch') %} {% set safesearch_info = _('Filter content') %} - {{ preferences_item_header(safesearch_info, safesearch_label, rtl) }} - @@ -95,16 +95,16 @@ {% set theme_label = _('Themes') %} {% set theme_info = _('Change searx layout') %} - {{ preferences_item_header(theme_info, theme_label, rtl) }} - {% for name in themes %} {% endfor %} {{ preferences_item_footer(theme_info, theme_label, rtl) }} - {{ preferences_item_header(_('Choose style for this theme'), _('Style'), rtl) }} - @@ -113,8 +113,8 @@ {% set label = _('Results on new tabs') %} {% set info = _('Open result links on new browser tabs') %} - {{ preferences_item_header(info, label, rtl) }} - @@ -122,8 +122,8 @@ {% set label = _('Open Access DOI resolver') %} {% set info = _('Redirect to open-access versions of publications when available (plugin required)') %} - {{ preferences_item_header(info, label, rtl) }} - {% for doi_resolver_name,doi_resolver_url in doi_resolvers.items() %}
@@ -241,7 +241,7 @@

{{ _(plugin.name) }}

-
{{ _(plugin.description) }}
+
{{ checkbox_toggle('plugin_' + plugin.id, plugin.id not in allowed_plugins) }} From ec48447361045bf1e68d3634fcae862e77f85d92 Mon Sep 17 00:00:00 2001 From: Adam Tauber Date: Thu, 25 Jun 2020 17:43:37 +0200 Subject: [PATCH 22/30] [fix] change lang attribute of the html to match the current locale - #350 --- searx/templates/oscar/base.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/searx/templates/oscar/base.html b/searx/templates/oscar/base.html index 83961997..2ccadd70 100644 --- a/searx/templates/oscar/base.html +++ b/searx/templates/oscar/base.html @@ -1,6 +1,6 @@ {% from 'oscar/macros.html' import icon %} - + From cc164abdfcecef82eada0d4657475e4e55b65be8 Mon Sep 17 00:00:00 2001 From: Adam Tauber Date: Thu, 25 Jun 2020 18:12:28 +0200 Subject: [PATCH 23/30] [fix] make "Advanced settings" openable and usable with keyboard - #350 --- .../themes/oscar/css/logicodev-dark.css | 4 ++-- .../themes/oscar/css/logicodev-dark.min.css | Bin 13864 -> 13889 bytes searx/static/themes/oscar/css/logicodev.css | 4 ++-- .../static/themes/oscar/css/logicodev.min.css | Bin 10424 -> 10449 bytes .../themes/oscar/less/logicodev/advanced.less | 4 ++-- searx/templates/oscar/advanced.html | 2 +- 6 files changed, 7 insertions(+), 7 deletions(-) diff --git a/searx/static/themes/oscar/css/logicodev-dark.css b/searx/static/themes/oscar/css/logicodev-dark.css index 38b9dbaa..cf32e76b 100644 --- a/searx/static/themes/oscar/css/logicodev-dark.css +++ b/searx/static/themes/oscar/css/logicodev-dark.css @@ -435,8 +435,8 @@ Ny0yNFQxMToxNTowMCswMjowMP7RDgQAAAAZdEVYdFNvZnR3YXJlAHd3dy5pbmtzY2FwZS5vcmeb font-weight: bold; border-bottom: #01d7d4 5px solid; } -#check-advanced { - display: none; +#check-advanced:focus + label { + text-decoration: underline; } #check-advanced:checked ~ #advanced-search-container { display: block; diff --git a/searx/static/themes/oscar/css/logicodev-dark.min.css b/searx/static/themes/oscar/css/logicodev-dark.min.css index 6c7cf749a01ee88e35c7003fe2c28cacf896ccd6..b1eeba8a2baa5bf0d23600525562355187c80edd 100644 GIT binary patch delta 49 zcmZ3Hb1-MaENK<1wEX1KV(py7q|}`1lGKV4-IUbi{G!B?%=|p7(!7+^qMXdlOQoBo E00GeyA^-pY delta 24 fcmX?@vm$51ENSlQl+5CSoWx42y!_4kq?@GxiDL?5 diff --git a/searx/static/themes/oscar/css/logicodev.css b/searx/static/themes/oscar/css/logicodev.css index ad6d967b..76af5835 100644 --- a/searx/static/themes/oscar/css/logicodev.css +++ b/searx/static/themes/oscar/css/logicodev.css @@ -408,8 +408,8 @@ Ny0yNFQxMToxNTowMCswMjowMP7RDgQAAAAZdEVYdFNvZnR3YXJlAHd3dy5pbmtzY2FwZS5vcmeb font-weight: bold; border-bottom: #01d7d4 5px solid; } -#check-advanced { - display: none; +#check-advanced:focus + label { + text-decoration: underline; } #check-advanced:checked ~ #advanced-search-container { display: block; diff --git a/searx/static/themes/oscar/css/logicodev.min.css b/searx/static/themes/oscar/css/logicodev.min.css index a39cc36b927c6f7ccc11ecc2b00844f15b1460c3..88b9dae54b152fd9a0868b984a2b588cd1162b93 100644 GIT binary patch delta 49 zcmdlHcrkDTyR3>;T7GhAv35>kQff|hNoqxjZc1u$eo +
{{ _("Allow") }}{{ _("Engine name") }}{{ _("Shortcut") }}{{ _("Selected language") }}{{ _("SafeSearch") }}{{ _("Time range") }}{{ _("Avg. time") }}{{ _("Max time") }}{{ _("Allow") }}{{ _("Engine name") }}{{ _("Shortcut") }}{{ _("Selected language") }}{{ _("SafeSearch") }}{{ _("Time range") }}{{ _("Avg. time") }}{{ _("Max time") }}{{ _("Max time") }}{{ _("Avg. time") }}{{ _("Time range") }}{{ _("SafeSearch") }}{{ _("Selected language") }}{{ _("Shortcut") }}{{ _("Engine name") }}{{ _("Allow") }}{{ _("Max time") }}{{ _("Avg. time") }}{{ _("Time range") }}{{ _("SafeSearch") }}{{ _("Selected language") }}{{ _("Shortcut") }}{{ _("Engine name") }}{{ _("Allow") }}
{{ checkbox_toggle('engine_' + search_engine.name|replace(' ', '_') + '__' + categ|replace(' ', '_'), (search_engine.name, categ) in disabled_engines) }} {{ search_engine.name }}{{ search_engine.name }} {{ shortcuts[search_engine.name] }} {{ support_toggle(stats[search_engine.name].supports_selected_language) }} {{ support_toggle(search_engine.safesearch==True) }}{{ support_toggle(search_engine.safesearch==True) }} {{ support_toggle(stats[search_engine.name].supports_selected_language) }} {{ shortcuts[search_engine.name] }}{{ search_engine.name }}{{ search_engine.name }} {{ checkbox_toggle('engine_' + search_engine.name|replace(' ', '_') + '__' + categ|replace(' ', '_'), (search_engine.name, categ) in disabled_engines) }}