commit 3f8292d44eea41a5304a8786f5b94fff1c8d8abd Author: mrpaulblack Date: Mon Jan 23 22:08:58 2023 +0000 build from commit 522ba9a14b7872d95e3ae61bc3a652b50100cc3d diff --git a/.buildinfo b/.buildinfo new file mode 100644 index 00000000..1258950f --- /dev/null +++ b/.buildinfo @@ -0,0 +1,4 @@ +# Sphinx build info version 1 +# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. +config: 78ff65dd4ea6762d8a38b76abe50b87c +tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 00000000..e69de29b diff --git a/404.html b/404.html new file mode 100644 index 00000000..7b1c63a4 --- /dev/null +++ b/404.html @@ -0,0 +1,109 @@ + + + + + + + + + Page not found — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Page not found

+ +Unfortunately we couldn't find the content you were looking for. + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/CNAME b/CNAME new file mode 100644 index 00000000..5e170067 --- /dev/null +++ b/CNAME @@ -0,0 +1 @@ +docs.searxng.org \ No newline at end of file diff --git a/_downloads/ad0ebe55d6b53b1559e0ca8dee6f30b9/reST.rst b/_downloads/ad0ebe55d6b53b1559e0ca8dee6f30b9/reST.rst new file mode 100644 index 00000000..e5d49f67 --- /dev/null +++ b/_downloads/ad0ebe55d6b53b1559e0ca8dee6f30b9/reST.rst @@ -0,0 +1,1436 @@ +.. _reST primer: + +=========== +reST primer +=========== + +.. sidebar:: KISS_ and readability_ + + Instead of defining more and more roles, we at SearXNG encourage our + contributors to follow principles like KISS_ and readability_. + +We at SearXNG are using reStructuredText (aka reST_) markup for all kind of +documentation, with the builders from the Sphinx_ project a HTML output is +generated and deployed at :docs:`github.io <.>`. For build prerequisites read +:ref:`docs build`. + +The source files of Searx's documentation are located at :origin:`docs`. Sphinx +assumes source files to be encoded in UTF-8 by default. Run :ref:`make docs.live +` to build HTML while editing. + +.. sidebar:: Further reading + + - Sphinx-Primer_ + - `Sphinx markup constructs`_ + - reST_, docutils_, `docutils FAQ`_ + - Sphinx_, `sphinx-doc FAQ`_ + - `sphinx config`_, doctree_ + - `sphinx cross references`_ + - linuxdoc_ + - intersphinx_ + - sphinx-jinja_ + - `Sphinx's autodoc`_ + - `Sphinx's Python domain`_, `Sphinx's C domain`_ + - SVG_, ImageMagick_ + - DOT_, `Graphviz's dot`_, Graphviz_ + + +.. contents:: Contents + :depth: 3 + :local: + :backlinks: entry + +Sphinx_ and reST_ have their place in the python ecosystem. Over that reST is +used in popular projects, e.g the Linux kernel documentation `[kernel doc]`_. + +.. _[kernel doc]: https://www.kernel.org/doc/html/latest/doc-guide/sphinx.html + +.. sidebar:: Content matters + + The readability_ of the reST sources has its value, therefore we recommend to + make sparse usage of reST markup / .. content matters! + +**reST** is a plaintext markup language, its markup is *mostly* intuitive and +you will not need to learn much to produce well formed articles with. I use the +word *mostly*: like everything in live, reST has its advantages and +disadvantages, some markups feel a bit grumpy (especially if you are used to +other plaintext markups). + +Soft skills +=========== + +Before going any deeper into the markup let's face on some **soft skills** a +trained author brings with, to reach a well feedback from readers: + +- Documentation is dedicated to an audience and answers questions from the + audience point of view. +- Don't detail things which are general knowledge from the audience point of + view. +- Limit the subject, use cross links for any further reading. + +To be more concrete what a *point of view* means. In the (:origin:`docs`) +folder we have three sections (and the *blog* folder), each dedicate to a +different group of audience. + +User's POV: :origin:`docs/user` + A typical user knows about search engines and might have heard about + meta crawlers and privacy. + +Admin's POV: :origin:`docs/admin` + A typical Admin knows about setting up services on a linux system, but he does + not know all the pros and cons of a SearXNG setup. + +Developer's POV: :origin:`docs/dev` + Depending on the readability_ of code, a typical developer is able to read and + understand source code. Describe what a item aims to do (e.g. a function). + If the chronological order matters, describe it. Name the *out-of-limits + conditions* and all the side effects a external developer will not know. + +.. _reST inline markup: + +Basic inline markup +=================== + +.. sidebar:: Inline markup + + - :ref:`reST roles` + - :ref:`reST smart ref` + +Basic inline markup is done with asterisks and backquotes. If asterisks or +backquotes appear in running text and could be confused with inline markup +delimiters, they have to be escaped with a backslash (``\*pointer``). + +.. table:: basic inline markup + :widths: 4 3 7 + + ================================================ ==================== ======================== + description rendered markup + ================================================ ==================== ======================== + one asterisk for emphasis *italics* ``*italics*`` + two asterisks for strong emphasis **boldface** ``**boldface**`` + backquotes for code samples and literals ``foo()`` ````foo()```` + quote asterisks or backquotes \*foo is a pointer ``\*foo is a pointer`` + ================================================ ==================== ======================== + +.. _reST basic structure: + +Basic article structure +======================= + +The basic structure of an article makes use of heading adornments to markup +chapter, sections and subsections. + +.. _reST template: + +reST template +------------- + +reST template for an simple article: + +.. code:: reST + + .. _doc refname: + + ============== + Document title + ============== + + Lorem ipsum dolor sit amet, consectetur adipisici elit .. Further read + :ref:`chapter refname`. + + .. _chapter refname: + + Chapter + ======= + + Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut + aliquid ex ea commodi consequat ... + + .. _section refname: + + Section + ------- + + lorem .. + + .. _subsection refname: + + Subsection + ~~~~~~~~~~ + + lorem .. + + +Headings +-------- + +#. title - with overline for document title: + + .. code:: reST + + ============== + Document title + ============== + + +#. chapter - with anchor named ``anchor name``: + + .. code:: reST + + .. _anchor name: + + Chapter + ======= + +#. section + + .. code:: reST + + Section + ------- + +#. subsection + + .. code:: reST + + Subsection + ~~~~~~~~~~ + + + +Anchors & Links +=============== + +.. _reST anchor: + +Anchors +------- + +.. _ref role: + https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-ref + +To refer a point in the documentation a anchor is needed. The :ref:`reST +template ` shows an example where a chapter titled *"Chapters"* +gets an anchor named ``chapter title``. Another example from *this* document, +where the anchor named ``reST anchor``: + +.. code:: reST + + .. _reST anchor: + + Anchors + ------- + + To refer a point in the documentation a anchor is needed ... + +To refer anchors use the `ref role`_ markup: + +.. code:: reST + + Visit chapter :ref:`reST anchor`. Or set hyperlink text manually :ref:`foo + bar `. + +.. admonition:: ``:ref:`` role + :class: rst-example + + Visist chapter :ref:`reST anchor`. Or set hyperlink text manually :ref:`foo + bar `. + +.. _reST ordinary ref: + +Link ordinary URL +----------------- + +If you need to reference external URLs use *named* hyperlinks to maintain +readability of reST sources. Here is a example taken from *this* article: + +.. code:: reST + + .. _Sphinx Field Lists: + https://www.sphinx-doc.org/en/master/usage/restructuredtext/field-lists.html + + With the *named* hyperlink `Sphinx Field Lists`_, the raw text is much more + readable. + + And this shows the alternative (less readable) hyperlink markup `Sphinx Field + Lists + `__. + +.. admonition:: Named hyperlink + :class: rst-example + + With the *named* hyperlink `Sphinx Field Lists`_, the raw text is much more + readable. + + And this shows the alternative (less readable) hyperlink markup `Sphinx Field + Lists + `__. + + +.. _reST smart ref: + +Smart refs +---------- + +With the power of sphinx.ext.extlinks_ and intersphinx_ referencing external +content becomes smart. + +.. table:: smart refs with sphinx.ext.extlinks_ and intersphinx_ + :widths: 4 3 7 + + ========================== ================================== ==================================== + refer ... rendered example markup + ========================== ================================== ==================================== + :rst:role:`rfc` :rfc:`822` ``:rfc:`822``` + :rst:role:`pep` :pep:`8` ``:pep:`8``` + sphinx.ext.extlinks_ + -------------------------------------------------------------------------------------------------- + project's wiki article :wiki:`Offline-engines` ``:wiki:`Offline-engines``` + to docs public URL :docs:`dev/reST.html` ``:docs:`dev/reST.html``` + files & folders origin :origin:`docs/dev/reST.rst` ``:origin:`docs/dev/reST.rst``` + pull request :pull:`4` ``:pull:`4``` + patch :patch:`af2cae6` ``:patch:`af2cae6``` + PyPi package :pypi:`searx` ``:pypi:`searx``` + manual page man :man:`bash` ``:man:`bash``` + intersphinx_ + -------------------------------------------------------------------------------------------------- + external anchor :ref:`python:and` ``:ref:`python:and``` + external doc anchor :doc:`jinja:templates` ``:doc:`jinja:templates``` + python code object :py:obj:`datetime.datetime` ``:py:obj:`datetime.datetime``` + flask code object :py:obj:`flask.Flask` ``:py:obj:`flask.Flask``` + ========================== ================================== ==================================== + + +Intersphinx is configured in :origin:`docs/conf.py`: + +.. code:: python + + intersphinx_mapping = { + "python": ("https://docs.python.org/3/", None), + "flask": ("https://flask.palletsprojects.com/", None), + "jinja": ("https://jinja.palletsprojects.com/", None), + "linuxdoc" : ("https://return42.github.io/linuxdoc/", None), + "sphinx" : ("https://www.sphinx-doc.org/en/master/", None), + } + + +To list all anchors of the inventory (e.g. ``python``) use: + +.. code:: sh + + $ python -m sphinx.ext.intersphinx https://docs.python.org/3/objects.inv + ... + $ python -m sphinx.ext.intersphinx https://docs.searxng.org/objects.inv + ... + +Literal blocks +============== + +The simplest form of :duref:`literal-blocks` is a indented block introduced by +two colons (``::``). For highlighting use :dudir:`highlight` or :ref:`reST +code` directive. To include literals from external files use +:rst:dir:`literalinclude` or :ref:`kernel-include ` +directive (latter one expands environment variables in the path name). + +.. _reST literal: + +``::`` +------ + +.. code:: reST + + :: + + Literal block + + Lorem ipsum dolor:: + + Literal block + + Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy + eirmod tempor invidunt ut labore :: + + Literal block + +.. admonition:: Literal block + :class: rst-example + + :: + + Literal block + + Lorem ipsum dolor:: + + Literal block + + Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy + eirmod tempor invidunt ut labore :: + + Literal block + + +.. _reST code: + +``code-block`` +-------------- + +.. _pygments: https://pygments.org/languages/ + +.. sidebar:: Syntax highlighting + + is handled by pygments_. + +The :rst:dir:`code-block` directive is a variant of the :dudir:`code` directive +with additional options. To learn more about code literals visit +:ref:`sphinx:code-examples`. + +.. code-block:: reST + + The URL ``/stats`` handle is shown in :ref:`stats-handle` + + .. code-block:: Python + :caption: python code block + :name: stats-handle + + @app.route('/stats', methods=['GET']) + def stats(): + """Render engine statistics page.""" + stats = get_engines_stats() + return render( + 'stats.html' + , stats = stats ) + +.. code-block:: reST + +.. admonition:: Code block + :class: rst-example + + The URL ``/stats`` handle is shown in :ref:`stats-handle` + + .. code-block:: Python + :caption: python code block + :name: stats-handle + + @app.route('/stats', methods=['GET']) + def stats(): + """Render engine statistics page.""" + stats = get_engines_stats() + return render( + 'stats.html' + , stats = stats ) + +Unicode substitution +==================== + +The :dudir:`unicode directive ` converts Unicode +character codes (numerical values) to characters. This directive can only be +used within a substitution definition. + +.. code-block:: reST + + .. |copy| unicode:: 0xA9 .. copyright sign + .. |(TM)| unicode:: U+2122 + + Trademark |(TM)| and copyright |copy| glyphs. + +.. admonition:: Unicode + :class: rst-example + + .. |copy| unicode:: 0xA9 .. copyright sign + .. |(TM)| unicode:: U+2122 + + Trademark |(TM)| and copyright |copy| glyphs. + + +.. _reST roles: + +Roles +===== + +.. sidebar:: Further reading + + - `Sphinx Roles`_ + - :doc:`sphinx:usage/restructuredtext/domains` + + +A *custom interpreted text role* (:duref:`ref `) is an inline piece of +explicit markup. It signifies that that the enclosed text should be interpreted +in a specific way. + +The general markup is one of: + +.. code:: reST + + :rolename:`ref-name` + :rolename:`ref text ` + +.. table:: smart refs with sphinx.ext.extlinks_ and intersphinx_ + :widths: 4 3 7 + + ========================== ================================== ==================================== + role rendered example markup + ========================== ================================== ==================================== + :rst:role:`guilabel` :guilabel:`&Cancel` ``:guilabel:`&Cancel``` + :rst:role:`kbd` :kbd:`C-x C-f` ``:kbd:`C-x C-f``` + :rst:role:`menuselection` :menuselection:`Open --> File` ``:menuselection:`Open --> File``` + :rst:role:`download` :download:`this file ` ``:download:`this file ``` + math_ :math:`a^2 + b^2 = c^2` ``:math:`a^2 + b^2 = c^2``` + :rst:role:`ref` :ref:`svg image example` ``:ref:`svg image example``` + :rst:role:`command` :command:`ls -la` ``:command:`ls -la``` + :durole:`emphasis` :emphasis:`italic` ``:emphasis:`italic``` + :durole:`strong` :strong:`bold` ``:strong:`bold``` + :durole:`literal` :literal:`foo()` ``:literal:`foo()``` + :durole:`subscript` H\ :sub:`2`\ O ``H\ :sub:`2`\ O`` + :durole:`superscript` E = mc\ :sup:`2` ``E = mc\ :sup:`2``` + :durole:`title-reference` :title:`Time` ``:title:`Time``` + ========================== ================================== ==================================== + +Figures & Images +================ + +.. sidebar:: Image processing + + With the directives from :ref:`linuxdoc ` the build process + is flexible. To get best results in the generated output format, install + ImageMagick_ and Graphviz_. + +Searx's sphinx setup includes: :ref:`linuxdoc:kfigure`. Scalable here means; +scalable in sense of the build process. Normally in absence of a converter +tool, the build process will break. From the authors POV it’s annoying to care +about the build process when handling with images, especially since he has no +access to the build process. With :ref:`linuxdoc:kfigure` the build process +continues and scales output quality in dependence of installed image processors. + +If you want to add an image, you should use the ``kernel-figure`` (inheritance +of :dudir:`figure`) and ``kernel-image`` (inheritance of :dudir:`image`) +directives. E.g. to insert a figure with a scalable image format use SVG +(:ref:`svg image example`): + +.. code:: reST + + .. _svg image example: + + .. kernel-figure:: svg_image.svg + :alt: SVG image example + + Simple SVG image + + To refer the figure, a caption block is needed: :ref:`svg image example`. + +.. _svg image example: + +.. kernel-figure:: svg_image.svg + :alt: SVG image example + + Simple SVG image. + +To refer the figure, a caption block is needed: :ref:`svg image example`. + +DOT files (aka Graphviz) +------------------------ + +With :ref:`linuxdoc:kernel-figure` reST support for **DOT** formatted files is +given. + +- `Graphviz's dot`_ +- DOT_ +- Graphviz_ + +A simple example is shown in :ref:`dot file example`: + +.. code:: reST + + .. _dot file example: + + .. kernel-figure:: hello.dot + :alt: hello world + + DOT's hello world example + +.. admonition:: hello.dot + :class: rst-example + + .. _dot file example: + + .. kernel-figure:: hello.dot + :alt: hello world + + DOT's hello world example + +``kernel-render`` DOT +--------------------- + +Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the +:ref:`linuxdoc:kernel-render` directive. A simple example of embedded DOT_ is +shown in figure :ref:`dot render example`: + +.. code:: reST + + .. _dot render example: + + .. kernel-render:: DOT + :alt: digraph + :caption: Embedded DOT (Graphviz) code + + digraph foo { + "bar" -> "baz"; + } + + Attribute ``caption`` is needed, if you want to refer the figure: :ref:`dot + render example`. + +Please note :ref:`build tools `. If Graphviz_ is +installed, you will see an vector image. If not, the raw markup is inserted as +*literal-block*. + +.. admonition:: kernel-render DOT + :class: rst-example + + .. _dot render example: + + .. kernel-render:: DOT + :alt: digraph + :caption: Embedded DOT (Graphviz) code + + digraph foo { + "bar" -> "baz"; + } + + Attribute ``caption`` is needed, if you want to refer the figure: :ref:`dot + render example`. + +``kernel-render`` SVG +--------------------- + +A simple example of embedded SVG_ is shown in figure :ref:`svg render example`: + +.. code:: reST + + .. _svg render example: + + .. kernel-render:: SVG + :caption: Embedded **SVG** markup + :alt: so-nw-arrow +.. + + .. code:: xml + + + + + + + +.. admonition:: kernel-render SVG + :class: rst-example + + .. _svg render example: + + .. kernel-render:: SVG + :caption: Embedded **SVG** markup + :alt: so-nw-arrow + + + + + + + + + + +.. _reST lists: + +List markups +============ + +Bullet list +----------- + +List markup (:duref:`ref `) is simple: + +.. code:: reST + + - This is a bulleted list. + + 1. Nested lists are possible, but be aware that they must be separated from + the parent list items by blank line + 2. Second item of nested list + + - It has two items, the second + item uses two lines. + + #. This is a numbered list. + #. It has two items too. + +.. admonition:: bullet list + :class: rst-example + + - This is a bulleted list. + + 1. Nested lists are possible, but be aware that they must be separated from + the parent list items by blank line + 2. Second item of nested list + + - It has two items, the second + item uses two lines. + + #. This is a numbered list. + #. It has two items too. + + +Horizontal list +--------------- + +The :rst:dir:`.. hlist:: ` transforms a bullet list into a more compact +list. + +.. code:: reST + + .. hlist:: + + - first list item + - second list item + - third list item + ... + +.. admonition:: hlist + :class: rst-example + + .. hlist:: + + - first list item + - second list item + - third list item + - next list item + - next list item xxxx + - next list item yyyy + - next list item zzzz + + +Definition list +--------------- + +.. sidebar:: Note .. + + - the term cannot have more than one line of text + + - there is **no blank line between term and definition block** // this + distinguishes definition lists (:duref:`ref `) from block + quotes (:duref:`ref `). + +Each definition list (:duref:`ref `) item contains a term, +optional classifiers and a definition. A term is a simple one-line word or +phrase. Optional classifiers may follow the term on the same line, each after +an inline ' : ' (**space, colon, space**). A definition is a block indented +relative to the term, and may contain multiple paragraphs and other body +elements. There may be no blank line between a term line and a definition block +(*this distinguishes definition lists from block quotes*). Blank lines are +required before the first and after the last definition list item, but are +optional in-between. + +Definition lists are created as follows: + +.. code:: reST + + term 1 (up to a line of text) + Definition 1. + + See the typo : this line is not a term! + + And this is not term's definition. **There is a blank line** in between + the line above and this paragraph. That's why this paragraph is taken as + **block quote** (:duref:`ref `) and not as term's definition! + + term 2 + Definition 2, paragraph 1. + + Definition 2, paragraph 2. + + term 3 : classifier + Definition 3. + + term 4 : classifier one : classifier two + Definition 4. + +.. admonition:: definition list + :class: rst-example + + term 1 (up to a line of text) + Definition 1. + + See the typo : this line is not a term! + + And this is not term's definition. **There is a blank line** in between + the line above and this paragraph. That's why this paragraph is taken as + **block quote** (:duref:`ref `) and not as term's definition! + + + term 2 + Definition 2, paragraph 1. + + Definition 2, paragraph 2. + + term 3 : classifier + Definition 3. + + term 4 : classifier one : classifier two + + +Quoted paragraphs +----------------- + +Quoted paragraphs (:duref:`ref `) are created by just indenting +them more than the surrounding paragraphs. Line blocks (:duref:`ref +`) are a way of preserving line breaks: + +.. code:: reST + + normal paragraph ... + lorem ipsum. + + Quoted paragraph ... + lorem ipsum. + + | These lines are + | broken exactly like in + | the source file. + + +.. admonition:: Quoted paragraph and line block + :class: rst-example + + normal paragraph ... + lorem ipsum. + + Quoted paragraph ... + lorem ipsum. + + | These lines are + | broken exactly like in + | the source file. + + +.. _reST field list: + +Field Lists +----------- + +.. _Sphinx Field Lists: + https://www.sphinx-doc.org/en/master/usage/restructuredtext/field-lists.html + +.. sidebar:: bibliographic fields + + First lines fields are bibliographic fields, see `Sphinx Field Lists`_. + +Field lists are used as part of an extension syntax, such as options for +directives, or database-like records meant for further processing. Field lists +are mappings from field names to field bodies. They marked up like this: + +.. code:: reST + + :fieldname: Field content + :foo: first paragraph in field foo + + second paragraph in field foo + + :bar: Field content + +.. admonition:: Field List + :class: rst-example + + :fieldname: Field content + :foo: first paragraph in field foo + + second paragraph in field foo + + :bar: Field content + + +They are commonly used in Python documentation: + +.. code:: python + + def my_function(my_arg, my_other_arg): + """A function just for me. + + :param my_arg: The first of my arguments. + :param my_other_arg: The second of my arguments. + + :returns: A message (just for me, of course). + """ + +Further list blocks +------------------- + +- field lists (:duref:`ref `, with caveats noted in + :ref:`reST field list`) +- option lists (:duref:`ref `) +- quoted literal blocks (:duref:`ref `) +- doctest blocks (:duref:`ref `) + + +Admonitions +=========== + +Sidebar +------- + +Sidebar is an eye catcher, often used for admonitions pointing further stuff or +site effects. Here is the source of the sidebar :ref:`on top of this page `. + +.. code:: reST + + .. sidebar:: KISS_ and readability_ + + Instead of defining more and more roles, we at SearXNG encourage our + contributors to follow principles like KISS_ and readability_. + +Generic admonition +------------------ + +The generic :dudir:`admonition ` needs a title: + +.. code:: reST + + .. admonition:: generic admonition title + + lorem ipsum .. + + +.. admonition:: generic admonition title + + lorem ipsum .. + + +Specific admonitions +-------------------- + +Specific admonitions: :dudir:`hint`, :dudir:`note`, :dudir:`tip` :dudir:`attention`, +:dudir:`caution`, :dudir:`danger`, :dudir:`error`, , :dudir:`important`, and +:dudir:`warning` . + +.. code:: reST + + .. hint:: + + lorem ipsum .. + + .. note:: + + lorem ipsum .. + + .. warning:: + + lorem ipsum .. + + +.. hint:: + + lorem ipsum .. + +.. note:: + + lorem ipsum .. + +.. tip:: + + lorem ipsum .. + +.. attention:: + + lorem ipsum .. + +.. caution:: + + lorem ipsum .. + +.. danger:: + + lorem ipsum .. + +.. important:: + + lorem ipsum .. + +.. error:: + + lorem ipsum .. + +.. warning:: + + lorem ipsum .. + + +Tables +====== + +.. sidebar:: Nested tables + + Nested tables are ugly! Not all builder support nested tables, don't use + them! + +ASCII-art tables like :ref:`reST simple table` and :ref:`reST grid table` might +be comfortable for readers of the text-files, but they have huge disadvantages +in the creation and modifying. First, they are hard to edit. Think about +adding a row or a column to a ASCII-art table or adding a paragraph in a cell, +it is a nightmare on big tables. + + +.. sidebar:: List tables + + For meaningful patch and diff use :ref:`reST flat table`. + +Second the diff of modifying ASCII-art tables is not meaningful, e.g. widening a +cell generates a diff in which also changes are included, which are only +ascribable to the ASCII-art. Anyway, if you prefer ASCII-art for any reason, +here are some helpers: + +* `Emacs Table Mode`_ +* `Online Tables Generator`_ + +.. _reST simple table: + +Simple tables +------------- + +:duref:`Simple tables ` allow *colspan* but not *rowspan*. If +your table need some metadata (e.g. a title) you need to add the ``.. table:: +directive`` :dudir:`(ref) ` in front and place the table in its body: + +.. code:: reST + + .. table:: foo gate truth table + :widths: grid + :align: left + + ====== ====== ====== + Inputs Output + ------------- ------ + A B A or B + ====== ====== ====== + False + -------------------- + True + -------------------- + True False True + (foo) + ------ ------ ------ + False True + (foo) + ====== ============= + +.. admonition:: Simple ASCII table + :class: rst-example + + .. table:: foo gate truth table + :widths: grid + :align: left + + ====== ====== ====== + Inputs Output + ------------- ------ + A B A or B + ====== ====== ====== + False + -------------------- + True + -------------------- + True False True + (foo) + ------ ------ ------ + False True + (foo) + ====== ============= + + + +.. _reST grid table: + +Grid tables +----------- + +:duref:`Grid tables ` allow colspan *colspan* and *rowspan*: + +.. code:: reST + + .. table:: grid table example + :widths: 1 1 5 + + +------------+------------+-----------+ + | Header 1 | Header 2 | Header 3 | + +============+============+===========+ + | body row 1 | column 2 | column 3 | + +------------+------------+-----------+ + | body row 2 | Cells may span columns.| + +------------+------------+-----------+ + | body row 3 | Cells may | - Cells | + +------------+ span rows. | - contain | + | body row 4 | | - blocks. | + +------------+------------+-----------+ + +.. admonition:: ASCII grid table + :class: rst-example + + .. table:: grid table example + :widths: 1 1 5 + + +------------+------------+-----------+ + | Header 1 | Header 2 | Header 3 | + +============+============+===========+ + | body row 1 | column 2 | column 3 | + +------------+------------+-----------+ + | body row 2 | Cells may span columns.| + +------------+------------+-----------+ + | body row 3 | Cells may | - Cells | + +------------+ span rows. | - contain | + | body row 4 | | - blocks. | + +------------+------------+-----------+ + + +.. _reST flat table: + +flat-table +---------- + +The ``flat-table`` is a further developed variant of the :ref:`list tables +`. It is a double-stage list similar to the +:dudir:`list-table` with some additional features: + +column-span: ``cspan`` + with the role ``cspan`` a cell can be extended through additional columns + +row-span: ``rspan`` + with the role ``rspan`` a cell can be extended through additional rows + +auto-span: + spans rightmost cell of a table row over the missing cells on the right side + of that table-row. With Option ``:fill-cells:`` this behavior can changed + from *auto span* to *auto fill*, which automatically inserts (empty) cells + instead of spanning the last cell. + +options: + :header-rows: [int] count of header rows + :stub-columns: [int] count of stub columns + :widths: [[int] [int] ... ] widths of columns + :fill-cells: instead of auto-span missing cells, insert missing cells + +roles: + :cspan: [int] additional columns (*morecols*) + :rspan: [int] additional rows (*morerows*) + +The example below shows how to use this markup. The first level of the staged +list is the *table-row*. In the *table-row* there is only one markup allowed, +the list of the cells in this *table-row*. Exception are *comments* ( ``..`` ) +and *targets* (e.g. a ref to :ref:`row 2 of table's body `). + +.. code:: reST + + .. flat-table:: ``flat-table`` example + :header-rows: 2 + :stub-columns: 1 + :widths: 1 1 1 1 2 + + * - :rspan:`1` head / stub + - :cspan:`3` head 1.1-4 + + * - head 2.1 + - head 2.2 + - head 2.3 + - head 2.4 + + * .. row body 1 / this is a comment + + - row 1 + - :rspan:`2` cell 1-3.1 + - cell 1.2 + - cell 1.3 + - cell 1.4 + + * .. Comments and targets are allowed on *table-row* stage. + .. _`row body 2`: + + - row 2 + - cell 2.2 + - :rspan:`1` :cspan:`1` + cell 2.3 with a span over + + * col 3-4 & + * row 2-3 + + * - row 3 + - cell 3.2 + + * - row 4 + - cell 4.1 + - cell 4.2 + - cell 4.3 + - cell 4.4 + + * - row 5 + - cell 5.1 with automatic span to right end + + * - row 6 + - cell 6.1 + - .. + + +.. admonition:: List table + :class: rst-example + + .. flat-table:: ``flat-table`` example + :header-rows: 2 + :stub-columns: 1 + :widths: 1 1 1 1 2 + + * - :rspan:`1` head / stub + - :cspan:`3` head 1.1-4 + + * - head 2.1 + - head 2.2 + - head 2.3 + - head 2.4 + + * .. row body 1 / this is a comment + + - row 1 + - :rspan:`2` cell 1-3.1 + - cell 1.2 + - cell 1.3 + - cell 1.4 + + * .. Comments and targets are allowed on *table-row* stage. + .. _`row body 2`: + + - row 2 + - cell 2.2 + - :rspan:`1` :cspan:`1` + cell 2.3 with a span over + + * col 3-4 & + * row 2-3 + + * - row 3 + - cell 3.2 + + * - row 4 + - cell 4.1 + - cell 4.2 + - cell 4.3 + - cell 4.4 + + * - row 5 + - cell 5.1 with automatic span to right end + + * - row 6 + - cell 6.1 + - .. + + +CSV table +--------- + +CSV table might be the choice if you want to include CSV-data from a outstanding +(build) process into your documentation. + +.. code:: reST + + .. csv-table:: CSV table example + :header: .. , Column 1, Column 2 + :widths: 2 5 5 + :stub-columns: 1 + :file: csv_table.txt + +Content of file ``csv_table.txt``: + +.. literalinclude:: csv_table.txt + +.. admonition:: CSV table + :class: rst-example + + .. csv-table:: CSV table example + :header: .. , Column 1, Column 2 + :widths: 3 5 5 + :stub-columns: 1 + :file: csv_table.txt + +Templating +========== + +.. sidebar:: Build environment + + All *generic-doc* tasks are running in the :ref:`make install`. + +Templating is suitable for documentation which is created generic at the build +time. The sphinx-jinja_ extension evaluates jinja_ templates in the :ref:`make +install` (with SearXNG modules installed). We use this e.g. to build chapter: +:ref:`configured engines`. Below the jinja directive from the +:origin:`docs/admin/engines.rst` is shown: + +.. literalinclude:: ../admin/engines/configured_engines.rst + :language: reST + :start-after: .. _configured engines: + +The context for the template is selected in the line ``.. jinja:: searx``. In +sphinx's build configuration (:origin:`docs/conf.py`) the ``searx`` context +contains the ``engines`` and ``plugins``. + +.. code:: py + + import searx.search + import searx.engines + import searx.plugins + searx.search.initialize() + jinja_contexts = { + 'searx': { + 'engines': searx.engines.engines, + 'plugins': searx.plugins.plugins + }, + } + + +Tabbed views +============ + +.. _sphinx-tabs: https://github.com/djungelorm/sphinx-tabs +.. _basic-tabs: https://github.com/djungelorm/sphinx-tabs#basic-tabs +.. _group-tabs: https://github.com/djungelorm/sphinx-tabs#group-tabs +.. _code-tabs: https://github.com/djungelorm/sphinx-tabs#code-tabs + +With `sphinx-tabs`_ extension we have *tabbed views*. To provide installation +instructions with one tab per distribution we use the `group-tabs`_ directive, +others are basic-tabs_ and code-tabs_. Below a *group-tab* example from +:ref:`docs build` is shown: + +.. literalinclude:: ../admin/buildhosts.rst + :language: reST + :start-after: .. SNIP sh lint requirements + :end-before: .. SNAP sh lint requirements + +.. _math: + +Math equations +============== + +.. _Mathematics: https://en.wikibooks.org/wiki/LaTeX/Mathematics +.. _amsmath user guide: + http://vesta.informatik.rwth-aachen.de/ftp/pub/mirror/ctan/macros/latex/required/amsmath/amsldoc.pdf + +.. sidebar:: About LaTeX + + - `amsmath user guide`_ + - Mathematics_ + - :ref:`docs build` + +The input language for mathematics is LaTeX markup using the :ctan:`amsmath` +package. + +To embed LaTeX markup in reST documents, use role :rst:role:`:math: ` for +inline and directive :rst:dir:`.. math:: ` for block markup. + +.. code:: reST + + In :math:numref:`schroedinger general` the time-dependent Schrödinger equation + is shown. + + .. math:: + :label: schroedinger general + + \mathrm{i}\hbar\dfrac{\partial}{\partial t} |\,\psi (t) \rangle = + \hat{H} |\,\psi (t) \rangle. + +.. admonition:: LaTeX math equation + :class: rst-example + + In :math:numref:`schroedinger general` the time-dependent Schrödinger equation + is shown. + + .. math:: + :label: schroedinger general + + \mathrm{i}\hbar\dfrac{\partial}{\partial t} |\,\psi (t) \rangle = + \hat{H} |\,\psi (t) \rangle. + + +The next example shows the difference of ``\tfrac`` (*textstyle*) and ``\dfrac`` +(*displaystyle*) used in a inline markup or another fraction. + +.. code:: reST + + ``\tfrac`` **inline example** :math:`\tfrac{\tfrac{1}{x}+\tfrac{1}{y}}{y-z}` + ``\dfrac`` **inline example** :math:`\dfrac{\dfrac{1}{x}+\dfrac{1}{y}}{y-z}` + +.. admonition:: Line spacing + :class: rst-example + + Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy + eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam + voluptua. ... + ``\tfrac`` **inline example** :math:`\tfrac{\tfrac{1}{x}+\tfrac{1}{y}}{y-z}` + At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd + gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. + + Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy + eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam + voluptua. ... + ``\tfrac`` **inline example** :math:`\dfrac{\dfrac{1}{x}+\dfrac{1}{y}}{y-z}` + At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd + gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. + + +.. _KISS: https://en.wikipedia.org/wiki/KISS_principle + +.. _readability: https://docs.python-guide.org/writing/style/ +.. _Sphinx-Primer: + https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html +.. _reST: https://docutils.sourceforge.io/rst.html +.. _Sphinx Roles: + https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html +.. _Sphinx: https://www.sphinx-doc.org +.. _`sphinx-doc FAQ`: https://www.sphinx-doc.org/en/stable/faq.html +.. _Sphinx markup constructs: + https://www.sphinx-doc.org/en/stable/markup/index.html +.. _`sphinx cross references`: + https://www.sphinx-doc.org/en/stable/markup/inline.html#cross-referencing-arbitrary-locations +.. _sphinx.ext.extlinks: + https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html +.. _intersphinx: https://www.sphinx-doc.org/en/stable/ext/intersphinx.html +.. _sphinx config: https://www.sphinx-doc.org/en/stable/config.html +.. _Sphinx's autodoc: https://www.sphinx-doc.org/en/stable/ext/autodoc.html +.. _Sphinx's Python domain: + https://www.sphinx-doc.org/en/stable/domains.html#the-python-domain +.. _Sphinx's C domain: + https://www.sphinx-doc.org/en/stable/domains.html#cross-referencing-c-constructs +.. _doctree: + https://www.sphinx-doc.org/en/master/extdev/tutorial.html?highlight=doctree#build-phases +.. _docutils: http://docutils.sourceforge.net/docs/index.html +.. _docutils FAQ: http://docutils.sourceforge.net/FAQ.html +.. _linuxdoc: https://return42.github.io/linuxdoc +.. _jinja: https://jinja.palletsprojects.com/ +.. _sphinx-jinja: https://github.com/tardyp/sphinx-jinja +.. _SVG: https://www.w3.org/TR/SVG11/expanded-toc.html +.. _DOT: https://graphviz.gitlab.io/_pages/doc/info/lang.html +.. _`Graphviz's dot`: https://graphviz.gitlab.io/_pages/pdf/dotguide.pdf +.. _Graphviz: https://graphviz.gitlab.io +.. _ImageMagick: https://www.imagemagick.org + +.. _`Emacs Table Mode`: https://www.emacswiki.org/emacs/TableMode +.. _`Online Tables Generator`: https://www.tablesgenerator.com/text_tables +.. _`OASIS XML Exchange Table Model`: https://www.oasis-open.org/specs/tm9901.html diff --git a/_images/DOT-57a4a7f78690d0b6b884bc59f36e84cfb0b61f76.dot b/_images/DOT-57a4a7f78690d0b6b884bc59f36e84cfb0b61f76.dot new file mode 100644 index 00000000..eaec9f36 --- /dev/null +++ b/_images/DOT-57a4a7f78690d0b6b884bc59f36e84cfb0b61f76.dot @@ -0,0 +1,3 @@ +digraph foo { + "bar" -> "baz"; +} \ No newline at end of file diff --git a/_images/DOT-57a4a7f78690d0b6b884bc59f36e84cfb0b61f76.svg b/_images/DOT-57a4a7f78690d0b6b884bc59f36e84cfb0b61f76.svg new file mode 100644 index 00000000..a5b36733 --- /dev/null +++ b/_images/DOT-57a4a7f78690d0b6b884bc59f36e84cfb0b61f76.svg @@ -0,0 +1,31 @@ + + + + + + +foo + + + +bar + +bar + + + +baz + +baz + + + +bar->baz + + + + + diff --git a/_images/SVG-1fb7029fa2cc454a267bae271cccb2c591387416.svg b/_images/SVG-1fb7029fa2cc454a267bae271cccb2c591387416.svg new file mode 100644 index 00000000..783577ec --- /dev/null +++ b/_images/SVG-1fb7029fa2cc454a267bae271cccb2c591387416.svg @@ -0,0 +1,13 @@ + + + + + \ No newline at end of file diff --git a/_images/arch_public.dot b/_images/arch_public.dot new file mode 100644 index 00000000..526fb53d --- /dev/null +++ b/_images/arch_public.dot @@ -0,0 +1,30 @@ +digraph G { + + node [style=filled, shape=box, fillcolor="#ffffcc", fontname=Sans]; + edge [fontname="Sans"]; + + browser [label="browser", shape=tab, fillcolor=aliceblue]; + rp [label="reverse proxy"]; + static [label="static files", shape=folder, href="url to configure static files", fillcolor=lightgray]; + uwsgi [label="uwsgi", shape=parallelogram href="https://docs.searxng.org/utils/searx.sh.html"] + redis [label="redis DB", shape=cylinder]; + searxng1 [label="SearXNG #1", fontcolor=blue3]; + searxng2 [label="SearXNG #2", fontcolor=blue3]; + searxng3 [label="SearXNG #3", fontcolor=blue3]; + searxng4 [label="SearXNG #4", fontcolor=blue3]; + + browser -> rp [label="HTTPS"] + + subgraph cluster_searxng { + label = "SearXNG instance" fontname=Sans; + bgcolor="#fafafa"; + { rank=same; static rp }; + rp -> static [label="optional: reverse proxy serves static files", fillcolor=slategray, fontcolor=slategray]; + rp -> uwsgi [label="http:// (tcp) or unix:// (socket)"]; + uwsgi -> searxng1 -> redis; + uwsgi -> searxng2 -> redis; + uwsgi -> searxng3 -> redis; + uwsgi -> searxng4 -> redis; + } + +} diff --git a/_images/arch_public.svg b/_images/arch_public.svg new file mode 100644 index 00000000..0a0a79ae --- /dev/null +++ b/_images/arch_public.svg @@ -0,0 +1,149 @@ + + + + + + +G + + +cluster_searxng + +SearXNG instance + + + +browser + + +browser + + + +rp + +reverse proxy + + + +browser->rp + + +HTTPS + + + +static + + +static files + + + + + +rp->static + + +optional: reverse proxy serves static files + + + +uwsgi + + +uwsgi + + + + + +rp->uwsgi + + +http:// (tcp) or unix:// (socket) + + + +searxng1 + +SearXNG #1 + + + +uwsgi->searxng1 + + + + + +searxng2 + +SearXNG #2 + + + +uwsgi->searxng2 + + + + + +searxng3 + +SearXNG #3 + + + +uwsgi->searxng3 + + + + + +searxng4 + +SearXNG #4 + + + +uwsgi->searxng4 + + + + + +redis + + +redis DB + + + +searxng1->redis + + + + + +searxng2->redis + + + + + +searxng3->redis + + + + + +searxng4->redis + + + + + diff --git a/_images/hello.dot b/_images/hello.dot new file mode 100644 index 00000000..504621df --- /dev/null +++ b/_images/hello.dot @@ -0,0 +1,3 @@ +graph G { + Hello -- World +} diff --git a/_images/hello.svg b/_images/hello.svg new file mode 100644 index 00000000..7d7af8e2 --- /dev/null +++ b/_images/hello.svg @@ -0,0 +1,30 @@ + + + + + + +G + + + +Hello + +Hello + + + +World + +World + + + +Hello--World + + + + diff --git a/_images/math/07c9ff4251510b06013159f4e45ec9ab97044096.svg b/_images/math/07c9ff4251510b06013159f4e45ec9ab97044096.svg new file mode 100644 index 00000000..80edca1d --- /dev/null +++ b/_images/math/07c9ff4251510b06013159f4e45ec9ab97044096.svg @@ -0,0 +1,25 @@ + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_images/math/3b8127a8eed95247f9249ea6c85e8e86df1baa82.svg b/_images/math/3b8127a8eed95247f9249ea6c85e8e86df1baa82.svg new file mode 100644 index 00000000..7c11c8b4 --- /dev/null +++ b/_images/math/3b8127a8eed95247f9249ea6c85e8e86df1baa82.svg @@ -0,0 +1,25 @@ + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_images/math/6673b43f9fe29455c1fcd1164e5844698cc64d38.svg b/_images/math/6673b43f9fe29455c1fcd1164e5844698cc64d38.svg new file mode 100644 index 00000000..8be77010 --- /dev/null +++ b/_images/math/6673b43f9fe29455c1fcd1164e5844698cc64d38.svg @@ -0,0 +1,22 @@ + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_images/math/a6a994cb6e7278ec30eaebe7e636046d3deccb5b.svg b/_images/math/a6a994cb6e7278ec30eaebe7e636046d3deccb5b.svg new file mode 100644 index 00000000..27ae3941 --- /dev/null +++ b/_images/math/a6a994cb6e7278ec30eaebe7e636046d3deccb5b.svg @@ -0,0 +1,43 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/_images/svg_image.svg b/_images/svg_image.svg new file mode 100644 index 00000000..5405f85b --- /dev/null +++ b/_images/svg_image.svg @@ -0,0 +1,10 @@ + + + + + + + + diff --git a/_images/translation.svg b/_images/translation.svg new file mode 100644 index 00000000..71f4172b --- /dev/null +++ b/_images/translation.svg @@ -0,0 +1,47 @@ +master branchmaster branchtranslations_update branchtranslations_update branchtranslations branchtranslations branchweblate clone oftranslations branchweblate clone oftranslations branchweblatepending changesweblatepending changesfor each commit on master.github/workflows/integration.ymlmake weblate.push.translationspybabel extractextract messages, store messages.pot on translations branchalt[if there are some changes in messages.pot]wlc lockwlc pullwlc commitgit merge weblate/translationspybabel update (messages.po)git add searx/translationsgit commitgit pushwlc unlockevery Friday.github/workflows/translations-update.ymlmake weblate.translations.commitwlc lockwlc pullwlc commitgit merge weblate/translationspybabel compilecp searx/translationsgit addgit commitwlc unlockcreate or update pull request"Update translations"developper's reviewmerge pull request \ No newline at end of file diff --git a/_modules/index.html b/_modules/index.html new file mode 100644 index 00000000..a3b8521c --- /dev/null +++ b/_modules/index.html @@ -0,0 +1,129 @@ + + + + + + + + + Overview: module code — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

All modules for which code is available

+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/babel_extract.html b/_modules/searx/babel_extract.html new file mode 100644 index 00000000..20ed1725 --- /dev/null +++ b/_modules/searx/babel_extract.html @@ -0,0 +1,166 @@ + + + + + + + + + searx.babel_extract — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.babel_extract

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""This module implements the :origin:`searxng_msg <babel.cfg>` extractor to
+extract messages from:
+
+- :origin:`searx/searxng.msg`
+
+The ``searxng.msg`` files are selected by Babel_, see Babel's configuration in
+:origin:`babel.cfg`::
+
+    searxng_msg = searx.babel_extract.extract
+    ...
+    [searxng_msg: **/searxng.msg]
+
+A ``searxng.msg`` file is a python file that is *executed* by the
+:py:obj:`extract` function.  Additional ``searxng.msg`` files can be added by:
+
+1. Adding a ``searxng.msg`` file in one of the SearXNG python packages and
+2. implement a method in :py:obj:`extract` that yields messages from this file.
+
+.. _Babel: https://babel.pocoo.org/en/latest/index.html
+
+"""
+
+from os import path
+
+SEARXNG_MSG_FILE = "searxng.msg"
+_MSG_FILES = [path.join(path.dirname(__file__), SEARXNG_MSG_FILE)]
+
+
+
[docs]def extract(
+    # pylint: disable=unused-argument
+    fileobj,
+    keywords,
+    comment_tags,
+    options,
+):
+    """Extract messages from ``searxng.msg`` files by a custom extractor_.
+
+    .. _extractor:
+       https://babel.pocoo.org/en/latest/messages.html#writing-extraction-methods
+    """
+    if fileobj.name not in _MSG_FILES:
+        raise RuntimeError("don't know how to extract messages from %s" % fileobj.name)
+
+    namespace = {}
+    exec(fileobj.read(), {}, namespace)  # pylint: disable=exec-used
+
+    for name in namespace['__all__']:
+        for k, v in namespace[name].items():
+            yield 0, '_', v, ["%s['%s']" % (name, k)]

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/engines.html b/_modules/searx/engines.html new file mode 100644 index 00000000..fc5b4167 --- /dev/null +++ b/_modules/searx/engines.html @@ -0,0 +1,423 @@ + + + + + + + + + searx.engines — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.engines

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""This module implements the engine loader.
+
+Load and initialize the ``engines``, see :py:func:`load_engines` and register
+:py:obj:`engine_shortcuts`.
+
+usage::
+
+    load_engines( settings['engines'] )
+
+"""
+
+import sys
+import copy
+from typing import Dict, List, Optional
+
+from os.path import realpath, dirname
+from babel.localedata import locale_identifiers
+from searx import logger, settings
+from searx.data import ENGINES_LANGUAGES
+from searx.network import get
+from searx.utils import load_module, match_language, gen_useragent
+
+
+logger = logger.getChild('engines')
+ENGINE_DIR = dirname(realpath(__file__))
+BABEL_LANGS = [
+    lang_parts[0] + '-' + lang_parts[-1] if len(lang_parts) > 1 else lang_parts[0]
+    for lang_parts in (lang_code.split('_') for lang_code in locale_identifiers())
+]
+ENGINE_DEFAULT_ARGS = {
+    "engine_type": "online",
+    "inactive": False,
+    "disabled": False,
+    "timeout": settings["outgoing"]["request_timeout"],
+    "shortcut": "-",
+    "categories": ["general"],
+    "supported_languages": [],
+    "language_aliases": {},
+    "paging": False,
+    "safesearch": False,
+    "time_range_support": False,
+    "enable_http": False,
+    "using_tor_proxy": False,
+    "display_error_messages": True,
+    "send_accept_language_header": False,
+    "tokens": [],
+    "about": {},
+}
+# set automatically when an engine does not have any tab category
+OTHER_CATEGORY = 'other'
+
+
+
[docs]class Engine:  # pylint: disable=too-few-public-methods
+    """This class is currently never initialized and only used for type hinting."""
+
+    name: str
+    engine: str
+    shortcut: str
+    categories: List[str]
+    supported_languages: List[str]
+    about: dict
+    inactive: bool
+    disabled: bool
+    language_support: bool
+    paging: bool
+    safesearch: bool
+    time_range_support: bool
+    timeout: float

+
+
+# Defaults for the namespace of an engine module, see :py:func:`load_engine`
+
+categories = {'general': []}
+engines: Dict[str, Engine] = {}
+engine_shortcuts = {}
+"""Simple map of registered *shortcuts* to name of the engine (or ``None``).
+
+::
+
+    engine_shortcuts[engine.shortcut] = engine.name
+
+:meta hide-value:
+"""
+
+
+
[docs]def load_engine(engine_data: dict) -> Optional[Engine]:
+    """Load engine from ``engine_data``.
+
+    :param dict engine_data:  Attributes from YAML ``settings:engines/<engine>``
+    :return: initialized namespace of the ``<engine>``.
+
+    1. create a namespace and load module of the ``<engine>``
+    2. update namespace with the defaults from :py:obj:`ENGINE_DEFAULT_ARGS`
+    3. update namespace with values from ``engine_data``
+
+    If engine *is active*, return namespace of the engine, otherwise return
+    ``None``.
+
+    This function also returns ``None`` if initialization of the namespace fails
+    for one of the following reasons:
+
+    - engine name contains underscore
+    - engine name is not lowercase
+    - required attribute is not set :py:func:`is_missing_required_attributes`
+
+    """
+    # pylint: disable=too-many-return-statements
+
+    engine_name = engine_data.get('name')
+    if engine_name is None:
+        logger.error('An engine does not have a "name" field')
+        return None
+    if '_' in engine_name:
+        logger.error('Engine name contains underscore: "{}"'.format(engine_name))
+        return None
+
+    if engine_name.lower() != engine_name:
+        logger.warn('Engine name is not lowercase: "{}", converting to lowercase'.format(engine_name))
+        engine_name = engine_name.lower()
+        engine_data['name'] = engine_name
+
+    # load_module
+    engine_module = engine_data.get('engine')
+    if engine_module is None:
+        logger.error('The "engine" field is missing for the engine named "{}"'.format(engine_name))
+        return None
+    try:
+        engine = load_module(engine_module + '.py', ENGINE_DIR)
+    except (SyntaxError, KeyboardInterrupt, SystemExit, SystemError, ImportError, RuntimeError):
+        logger.exception('Fatal exception in engine "{}"'.format(engine_module))
+        sys.exit(1)
+    except BaseException:
+        logger.exception('Cannot load engine "{}"'.format(engine_module))
+        return None
+
+    update_engine_attributes(engine, engine_data)
+    set_language_attributes(engine)
+    update_attributes_for_tor(engine)
+
+    if not is_engine_active(engine):
+        return None
+
+    if is_missing_required_attributes(engine):
+        return None
+
+    set_loggers(engine, engine_name)
+
+    if not any(cat in settings['categories_as_tabs'] for cat in engine.categories):
+        engine.categories.append(OTHER_CATEGORY)
+
+    return engine

+
+
+def set_loggers(engine, engine_name):
+    # set the logger for engine
+    engine.logger = logger.getChild(engine_name)
+    # the engine may have load some other engines
+    # may sure the logger is initialized
+    # use sys.modules.copy() to avoid "RuntimeError: dictionary changed size during iteration"
+    # see https://github.com/python/cpython/issues/89516
+    # and https://docs.python.org/3.10/library/sys.html#sys.modules
+    modules = sys.modules.copy()
+    for module_name, module in modules.items():
+        if (
+            module_name.startswith("searx.engines")
+            and module_name != "searx.engines.__init__"
+            and not hasattr(module, "logger")
+        ):
+            module_engine_name = module_name.split(".")[-1]
+            module.logger = logger.getChild(module_engine_name)
+
+
+def update_engine_attributes(engine: Engine, engine_data):
+    # set engine attributes from engine_data
+    for param_name, param_value in engine_data.items():
+        if param_name == 'categories':
+            if isinstance(param_value, str):
+                param_value = list(map(str.strip, param_value.split(',')))
+            engine.categories = param_value
+        elif hasattr(engine, 'about') and param_name == 'about':
+            engine.about = {**engine.about, **engine_data['about']}
+        else:
+            setattr(engine, param_name, param_value)
+
+    # set default attributes
+    for arg_name, arg_value in ENGINE_DEFAULT_ARGS.items():
+        if not hasattr(engine, arg_name):
+            setattr(engine, arg_name, copy.deepcopy(arg_value))
+
+
+def set_language_attributes(engine: Engine):
+    # assign supported languages from json file
+    if engine.name in ENGINES_LANGUAGES:
+        engine.supported_languages = ENGINES_LANGUAGES[engine.name]
+
+    elif engine.engine in ENGINES_LANGUAGES:
+        # The key of the dictionary ENGINES_LANGUAGES is the *engine name*
+        # configured in settings.xml.  When multiple engines are configured in
+        # settings.yml to use the same origin engine (python module) these
+        # additional engines can use the languages from the origin engine.
+        # For this use the configured ``engine: ...`` from settings.yml
+        engine.supported_languages = ENGINES_LANGUAGES[engine.engine]
+
+    if hasattr(engine, 'language'):
+        # For an engine, when there is `language: ...` in the YAML settings, the
+        # engine supports only one language, in this case
+        # engine.supported_languages should contains this value defined in
+        # settings.yml
+        if engine.language not in engine.supported_languages:
+            raise ValueError(
+                "settings.yml - engine: '%s' / language: '%s' not supported" % (engine.name, engine.language)
+            )
+
+        if isinstance(engine.supported_languages, dict):
+            engine.supported_languages = {engine.language: engine.supported_languages[engine.language]}
+        else:
+            engine.supported_languages = [engine.language]
+
+    # find custom aliases for non standard language codes
+    for engine_lang in engine.supported_languages:
+        iso_lang = match_language(engine_lang, BABEL_LANGS, fallback=None)
+        if (
+            iso_lang
+            and iso_lang != engine_lang
+            and not engine_lang.startswith(iso_lang)
+            and iso_lang not in engine.supported_languages
+        ):
+            engine.language_aliases[iso_lang] = engine_lang
+
+    # language_support
+    engine.language_support = len(engine.supported_languages) > 0
+
+    # assign language fetching method if auxiliary method exists
+    if hasattr(engine, '_fetch_supported_languages'):
+        headers = {
+            'User-Agent': gen_useragent(),
+            'Accept-Language': "en-US,en;q=0.5",  # bing needs to set the English language
+        }
+        engine.fetch_supported_languages = (
+            # pylint: disable=protected-access
+            lambda: engine._fetch_supported_languages(get(engine.supported_languages_url, headers=headers))
+        )
+
+
+def update_attributes_for_tor(engine: Engine) -> bool:
+    if using_tor_proxy(engine) and hasattr(engine, 'onion_url'):
+        engine.search_url = engine.onion_url + getattr(engine, 'search_path', '')
+        engine.timeout += settings['outgoing'].get('extra_proxy_timeout', 0)
+
+
+
[docs]def is_missing_required_attributes(engine):
+    """An attribute is required when its name doesn't start with ``_`` (underline).
+    Required attributes must not be ``None``.
+
+    """
+    missing = False
+    for engine_attr in dir(engine):
+        if not engine_attr.startswith('_') and getattr(engine, engine_attr) is None:
+            logger.error('Missing engine config attribute: "{0}.{1}"'.format(engine.name, engine_attr))
+            missing = True
+    return missing

+
+
+
[docs]def using_tor_proxy(engine: Engine):
+    """Return True if the engine configuration declares to use Tor."""
+    return settings['outgoing'].get('using_tor_proxy') or getattr(engine, 'using_tor_proxy', False)

+
+
+def is_engine_active(engine: Engine):
+    # check if engine is inactive
+    if engine.inactive is True:
+        return False
+
+    # exclude onion engines if not using tor
+    if 'onions' in engine.categories and not using_tor_proxy(engine):
+        return False
+
+    return True
+
+
+def register_engine(engine: Engine):
+    if engine.name in engines:
+        logger.error('Engine config error: ambiguous name: {0}'.format(engine.name))
+        sys.exit(1)
+    engines[engine.name] = engine
+
+    if engine.shortcut in engine_shortcuts:
+        logger.error('Engine config error: ambiguous shortcut: {0}'.format(engine.shortcut))
+        sys.exit(1)
+    engine_shortcuts[engine.shortcut] = engine.name
+
+    for category_name in engine.categories:
+        categories.setdefault(category_name, []).append(engine)
+
+
+
[docs]def load_engines(engine_list):
+    """usage: ``engine_list = settings['engines']``"""
+    engines.clear()
+    engine_shortcuts.clear()
+    categories.clear()
+    categories['general'] = []
+    for engine_data in engine_list:
+        engine = load_engine(engine_data)
+        if engine:
+            register_engine(engine)
+    return engines

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/engines/demo_offline.html b/_modules/searx/engines/demo_offline.html new file mode 100644 index 00000000..46dd1126 --- /dev/null +++ b/_modules/searx/engines/demo_offline.html @@ -0,0 +1,192 @@ + + + + + + + + + searx.engines.demo_offline — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.engines.demo_offline

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""Within this module we implement a *demo offline engine*.  Do not look to
+close to the implementation, its just a simple example.  To get in use of this
+*demo* engine add the following entry to your engines list in ``settings.yml``:
+
+.. code:: yaml
+
+  - name: my offline engine
+    engine: demo_offline
+    shortcut: demo
+    disabled: false
+
+"""
+
+import json
+
+engine_type = 'offline'
+categories = ['general']
+disabled = True
+timeout = 2.0
+
+about = {
+    "wikidata_id": None,
+    "official_api_documentation": None,
+    "use_official_api": False,
+    "require_api_key": False,
+    "results": 'JSON',
+}
+
+# if there is a need for globals, use a leading underline
+_my_offline_engine = None
+
+
+
[docs]def init(engine_settings=None):
+    """Initialization of the (offline) engine.  The origin of this demo engine is a
+    simple json string which is loaded in this example while the engine is
+    initialized.
+
+    """
+    global _my_offline_engine  # pylint: disable=global-statement
+
+    _my_offline_engine = (
+        '[ {"value": "%s"}'
+        ', {"value":"first item"}'
+        ', {"value":"second item"}'
+        ', {"value":"third item"}'
+        ']' % engine_settings.get('name')
+    )

+
+
+
+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/engines/demo_online.html b/_modules/searx/engines/demo_online.html new file mode 100644 index 00000000..f0dd8a0e --- /dev/null +++ b/_modules/searx/engines/demo_online.html @@ -0,0 +1,219 @@ + + + + + + + + + searx.engines.demo_online — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.engines.demo_online

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""Within this module we implement a *demo online engine*.  Do not look to
+close to the implementation, its just a simple example which queries `The Art
+Institute of Chicago <https://www.artic.edu>`_
+
+To get in use of this *demo* engine add the following entry to your engines
+list in ``settings.yml``:
+
+.. code:: yaml
+
+  - name: my online engine
+    engine: demo_online
+    shortcut: demo
+    disabled: false
+
+"""
+
+from json import loads
+from urllib.parse import urlencode
+
+engine_type = 'online'
+send_accept_language_header = True
+categories = ['general']
+disabled = True
+timeout = 2.0
+categories = ['images']
+paging = True
+page_size = 20
+
+search_api = 'https://api.artic.edu/api/v1/artworks/search?'
+image_api = 'https://www.artic.edu/iiif/2/'
+
+about = {
+    "website": 'https://www.artic.edu',
+    "wikidata_id": 'Q239303',
+    "official_api_documentation": 'http://api.artic.edu/docs/',
+    "use_official_api": True,
+    "require_api_key": False,
+    "results": 'JSON',
+}
+
+
+# if there is a need for globals, use a leading underline
+_my_online_engine = None
+
+
+
[docs]def init(engine_settings):
+    """Initialization of the (online) engine.  If no initialization is needed, drop
+    this init function.
+
+    """
+    global _my_online_engine  # pylint: disable=global-statement
+    _my_online_engine = engine_settings.get('name')

+
+
+
[docs]def request(query, params):
+    """Build up the ``params`` for the online request.  In this example we build a
+    URL to fetch images from `artic.edu <https://artic.edu>`__
+
+    """
+    args = urlencode(
+        {
+            'q': query,
+            'page': params['pageno'],
+            'fields': 'id,title,artist_display,medium_display,image_id,date_display,dimensions,artist_titles',
+            'limit': page_size,
+        }
+    )
+    params['url'] = search_api + args
+    return params

+
+
+
[docs]def response(resp):
+    """Parse out the result items from the response.  In this example we parse the
+    response from `api.artic.edu <https://artic.edu>`__ and filter out all
+    images.
+
+    """
+    results = []
+    json_data = loads(resp.text)
+
+    for result in json_data['data']:
+
+        if not result['image_id']:
+            continue
+
+        results.append(
+            {
+                'url': 'https://artic.edu/artworks/%(id)s' % result,
+                'title': result['title'] + " (%(date_display)s) //  %(artist_display)s" % result,
+                'content': result['medium_display'],
+                'author': ', '.join(result['artist_titles']),
+                'img_src': image_api + '/%(image_id)s/full/843,/0/default.jpg' % result,
+                'img_format': result['dimensions'],
+                'template': 'images.html',
+            }
+        )
+
+    return results

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/engines/google.html b/_modules/searx/engines/google.html new file mode 100644 index 00000000..bb9eadd0 --- /dev/null +++ b/_modules/searx/engines/google.html @@ -0,0 +1,494 @@ + + + + + + + + + searx.engines.google — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.engines.google

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""This is the implementation of the google WEB engine.  Some of this
+implementations are shared by other engines:
+
+- :ref:`google images engine`
+- :ref:`google news engine`
+- :ref:`google videos engine`
+
+The google WEB engine itself has a special setup option:
+
+.. code:: yaml
+
+  - name: google
+    ...
+    use_mobile_ui: false
+
+``use_mobile_ui``: (default: ``false``)
+  Enables to use *mobile endpoint* to bypass the google blocking (see
+  :issue:`159`).  On the mobile UI of Google Search, the button :guilabel:`More
+  results` is not affected by Google rate limiting and we can still do requests
+  while actively blocked by the original Google search.  By activate
+  ``use_mobile_ui`` this behavior is simulated by adding the parameter
+  ``async=use_ac:true,_fmt:pc`` to the :py:func:`request`.
+
+"""
+
+from urllib.parse import urlencode
+from lxml import html
+from searx.utils import match_language, extract_text, eval_xpath, eval_xpath_list, eval_xpath_getindex
+from searx.exceptions import SearxEngineCaptchaException
+
+# about
+about = {
+    "website": 'https://www.google.com',
+    "wikidata_id": 'Q9366',
+    "official_api_documentation": 'https://developers.google.com/custom-search/',
+    "use_official_api": False,
+    "require_api_key": False,
+    "results": 'HTML',
+}
+
+# engine dependent config
+categories = ['general', 'web']
+paging = True
+time_range_support = True
+safesearch = True
+send_accept_language_header = True
+use_mobile_ui = False
+supported_languages_url = 'https://www.google.com/preferences?#languages'
+
+# based on https://en.wikipedia.org/wiki/List_of_Google_domains and tests
+google_domains = {
+    'BG': 'google.bg',  # Bulgaria
+    'CZ': 'google.cz',  # Czech Republic
+    'DE': 'google.de',  # Germany
+    'DK': 'google.dk',  # Denmark
+    'AT': 'google.at',  # Austria
+    'CH': 'google.ch',  # Switzerland
+    'GR': 'google.gr',  # Greece
+    'AU': 'google.com.au',  # Australia
+    'CA': 'google.ca',  # Canada
+    'GB': 'google.co.uk',  # United Kingdom
+    'ID': 'google.co.id',  # Indonesia
+    'IE': 'google.ie',  # Ireland
+    'IN': 'google.co.in',  # India
+    'MY': 'google.com.my',  # Malaysia
+    'NZ': 'google.co.nz',  # New Zealand
+    'PH': 'google.com.ph',  # Philippines
+    'SG': 'google.com.sg',  # Singapore
+    'US': 'google.com',  # United States (google.us) redirects to .com
+    'ZA': 'google.co.za',  # South Africa
+    'AR': 'google.com.ar',  # Argentina
+    'CL': 'google.cl',  # Chile
+    'ES': 'google.es',  # Spain
+    'MX': 'google.com.mx',  # Mexico
+    'EE': 'google.ee',  # Estonia
+    'FI': 'google.fi',  # Finland
+    'BE': 'google.be',  # Belgium
+    'FR': 'google.fr',  # France
+    'IL': 'google.co.il',  # Israel
+    'HR': 'google.hr',  # Croatia
+    'HU': 'google.hu',  # Hungary
+    'IT': 'google.it',  # Italy
+    'JP': 'google.co.jp',  # Japan
+    'KR': 'google.co.kr',  # South Korea
+    'LT': 'google.lt',  # Lithuania
+    'LV': 'google.lv',  # Latvia
+    'NO': 'google.no',  # Norway
+    'NL': 'google.nl',  # Netherlands
+    'PL': 'google.pl',  # Poland
+    'BR': 'google.com.br',  # Brazil
+    'PT': 'google.pt',  # Portugal
+    'RO': 'google.ro',  # Romania
+    'RU': 'google.ru',  # Russia
+    'SK': 'google.sk',  # Slovakia
+    'SI': 'google.si',  # Slovenia
+    'SE': 'google.se',  # Sweden
+    'TH': 'google.co.th',  # Thailand
+    'TR': 'google.com.tr',  # Turkey
+    'UA': 'google.com.ua',  # Ukraine
+    'CN': 'google.com.hk',  # There is no google.cn, we use .com.hk for zh-CN
+    'HK': 'google.com.hk',  # Hong Kong
+    'TW': 'google.com.tw',  # Taiwan
+}
+
+time_range_dict = {'day': 'd', 'week': 'w', 'month': 'm', 'year': 'y'}
+
+# Filter results. 0: None, 1: Moderate, 2: Strict
+filter_mapping = {0: 'off', 1: 'medium', 2: 'high'}
+
+# specific xpath variables
+# ------------------------
+
+results_xpath = './/div[@data-sokoban-container]'
+title_xpath = './/a/h3[1]'
+href_xpath = './/a[h3]/@href'
+content_xpath = './/div[@data-content-feature=1]'
+
+# google *sections* are no usual *results*, we ignore them
+g_section_with_header = './g-section-with-header'
+
+
+# Suggestions are links placed in a *card-section*, we extract only the text
+# from the links not the links itself.
+suggestion_xpath = '//div[contains(@class, "EIaa9b")]//a'
+
+
+
[docs]def get_lang_info(params, lang_list, custom_aliases, supported_any_language):
+    """Composing various language properties for the google engines.
+
+    This function is called by the various google engines (:ref:`google web
+    engine`, :ref:`google images engine`, :ref:`google news engine` and
+    :ref:`google videos engine`).
+
+    :param dict param: request parameters of the engine
+
+    :param list lang_list: list of supported languages of the engine
+        :py:obj:`ENGINES_LANGUAGES[engine-name] <searx.data.ENGINES_LANGUAGES>`
+
+    :param dict lang_list: custom aliases for non standard language codes
+        (used when calling :py:func:`searx.utils.match_language`)
+
+    :param bool supported_any_language: When a language is not specified, the
+        language interpretation is left up to Google to decide how the search
+        results should be delivered.  This argument is ``True`` for the google
+        engine and ``False`` for the other engines (google-images, -news,
+        -scholar, -videos).
+
+    :rtype: dict
+    :returns:
+        Py-Dictionary with the key/value pairs:
+
+        language:
+            Return value from :py:func:`searx.utils.match_language`
+
+        country:
+            The country code (e.g. US, AT, CA, FR, DE ..)
+
+        subdomain:
+            Google subdomain :py:obj:`google_domains` that fits to the country
+            code.
+
+        params:
+            Py-Dictionary with additional request arguments (can be passed to
+            :py:func:`urllib.parse.urlencode`).
+
+        headers:
+            Py-Dictionary with additional HTTP headers (can be passed to
+            request's headers)
+    """
+    ret_val = {
+        'language': None,
+        'country': None,
+        'subdomain': None,
+        'params': {},
+        'headers': {},
+    }
+
+    # language ...
+
+    _lang = params['language']
+    _any_language = _lang.lower() == 'all'
+    if _any_language:
+        _lang = 'en-US'
+    language = match_language(_lang, lang_list, custom_aliases)
+    ret_val['language'] = language
+
+    # country ...
+
+    _l = _lang.split('-')
+    if len(_l) == 2:
+        country = _l[1]
+    else:
+        country = _l[0].upper()
+        if country == 'EN':
+            country = 'US'
+    ret_val['country'] = country
+
+    # subdomain ...
+
+    ret_val['subdomain'] = 'www.' + google_domains.get(country.upper(), 'google.com')
+
+    # params & headers
+
+    lang_country = '%s-%s' % (language, country)  # (en-US, en-EN, de-DE, de-AU, fr-FR ..)
+
+    # hl parameter:
+    #   https://developers.google.com/custom-search/docs/xml_results#hlsp The
+    # Interface Language:
+    #   https://developers.google.com/custom-search/docs/xml_results_appendices#interfaceLanguages
+
+    ret_val['params']['hl'] = lang_list.get(lang_country, language)
+
+    # lr parameter:
+    #   The lr (language restrict) parameter restricts search results to
+    #   documents written in a particular language.
+    #   https://developers.google.com/custom-search/docs/xml_results#lrsp
+    #   Language Collection Values:
+    #   https://developers.google.com/custom-search/docs/xml_results_appendices#languageCollections
+
+    if _any_language and supported_any_language:
+
+        # interpretation is left up to Google (based on whoogle)
+        #
+        # - add parameter ``source=lnt``
+        # - don't use parameter ``lr``
+        # - don't add a ``Accept-Language`` HTTP header.
+
+        ret_val['params']['source'] = 'lnt'
+
+    else:
+
+        # restricts search results to documents written in a particular
+        # language.
+        ret_val['params']['lr'] = "lang_" + lang_list.get(lang_country, language)
+
+    return ret_val

+
+
+def detect_google_sorry(resp):
+    if resp.url.host == 'sorry.google.com' or resp.url.path.startswith('/sorry'):
+        raise SearxEngineCaptchaException()
+
+
+
[docs]def request(query, params):
+    """Google search request"""
+
+    offset = (params['pageno'] - 1) * 10
+
+    lang_info = get_lang_info(params, supported_languages, language_aliases, True)
+
+    additional_parameters = {}
+    if use_mobile_ui:
+        additional_parameters = {
+            'asearch': 'arc',
+            'async': 'use_ac:true,_fmt:prog',
+        }
+
+    # https://www.google.de/search?q=corona&hl=de&lr=lang_de&start=0&tbs=qdr%3Ad&safe=medium
+    query_url = (
+        'https://'
+        + lang_info['subdomain']
+        + '/search'
+        + "?"
+        + urlencode(
+            {
+                'q': query,
+                **lang_info['params'],
+                'ie': "utf8",
+                'oe': "utf8",
+                'start': offset,
+                'filter': '0',
+                **additional_parameters,
+            }
+        )
+    )
+
+    if params['time_range'] in time_range_dict:
+        query_url += '&' + urlencode({'tbs': 'qdr:' + time_range_dict[params['time_range']]})
+    if params['safesearch']:
+        query_url += '&' + urlencode({'safe': filter_mapping[params['safesearch']]})
+    params['url'] = query_url
+
+    params['cookies']['CONSENT'] = "YES+"
+    params['headers'].update(lang_info['headers'])
+    if use_mobile_ui:
+        params['headers']['Accept'] = '*/*'
+    else:
+        params['headers']['Accept'] = 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8'
+
+    return params

+
+
+
[docs]def response(resp):
+    """Get response from google's search request"""
+
+    detect_google_sorry(resp)
+
+    results = []
+
+    # convert the text to dom
+    dom = html.fromstring(resp.text)
+    # results --> answer
+    answer_list = eval_xpath(dom, '//div[contains(@class, "LGOjhe")]')
+    if answer_list:
+        answer_list = [_.xpath("normalize-space()") for _ in answer_list]
+        results.append({'answer': ' '.join(answer_list)})
+    else:
+        logger.debug("did not find 'answer'")
+
+        # results --> number_of_results
+        if not use_mobile_ui:
+            try:
+                _txt = eval_xpath_getindex(dom, '//div[@id="result-stats"]//text()', 0)
+                _digit = ''.join([n for n in _txt if n.isdigit()])
+                number_of_results = int(_digit)
+                results.append({'number_of_results': number_of_results})
+            except Exception as e:  # pylint: disable=broad-except
+                logger.debug("did not 'number_of_results'")
+                logger.error(e, exc_info=True)
+
+    # parse results
+
+    for result in eval_xpath_list(dom, results_xpath):
+
+        # google *sections*
+        if extract_text(eval_xpath(result, g_section_with_header)):
+            logger.debug("ignoring <g-section-with-header>")
+            continue
+
+        try:
+            title_tag = eval_xpath_getindex(result, title_xpath, 0, default=None)
+            if title_tag is None:
+                # this not one of the common google results *section*
+                logger.debug('ignoring item from the result_xpath list: missing title')
+                continue
+            title = extract_text(title_tag)
+            url = eval_xpath_getindex(result, href_xpath, 0, None)
+            if url is None:
+                continue
+            content = extract_text(eval_xpath_getindex(result, content_xpath, 0, default=None), allow_none=True)
+            if content is None:
+                logger.debug('ignoring item from the result_xpath list: missing content of title "%s"', title)
+                continue
+
+            logger.debug('add link to results: %s', title)
+            results.append({'url': url, 'title': title, 'content': content})
+
+        except Exception as e:  # pylint: disable=broad-except
+            logger.error(e, exc_info=True)
+            continue
+
+    # parse suggestion
+    for suggestion in eval_xpath_list(dom, suggestion_xpath):
+        # append suggestion
+        results.append({'suggestion': extract_text(suggestion)})
+
+    # return results
+    return results

+
+
+# get supported languages from their site
+def _fetch_supported_languages(resp):
+    ret_val = {}
+    dom = html.fromstring(resp.text)
+
+    radio_buttons = eval_xpath_list(dom, '//*[@id="langSec"]//input[@name="lr"]')
+
+    for x in radio_buttons:
+        name = x.get("data-name")
+        code = x.get("value").split('_')[-1]
+        ret_val[code] = {"name": name}
+
+    return ret_val
+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/engines/google_images.html b/_modules/searx/engines/google_images.html new file mode 100644 index 00000000..03a5732c --- /dev/null +++ b/_modules/searx/engines/google_images.html @@ -0,0 +1,241 @@ + + + + + + + + + searx.engines.google_images — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.engines.google_images

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""This is the implementation of the google images engine using the google
+internal API used the Google Go Android app.
+
+This internal API offer results in
+
+- JSON (_fmt:json)
+- Protobuf (_fmt:pb)
+- Protobuf compressed? (_fmt:pc)
+- HTML (_fmt:html)
+- Protobuf encoded in JSON (_fmt:jspb).
+
+"""
+
+from urllib.parse import urlencode
+from json import loads
+
+from searx.engines.google import (
+    get_lang_info,
+    time_range_dict,
+    detect_google_sorry,
+)
+
+# pylint: disable=unused-import
+from searx.engines.google import supported_languages_url, _fetch_supported_languages
+
+# pylint: enable=unused-import
+
+# about
+about = {
+    "website": 'https://images.google.com',
+    "wikidata_id": 'Q521550',
+    "official_api_documentation": 'https://developers.google.com/custom-search',
+    "use_official_api": False,
+    "require_api_key": False,
+    "results": 'JSON',
+}
+
+# engine dependent config
+categories = ['images', 'web']
+paging = True
+use_locale_domain = True
+time_range_support = True
+safesearch = True
+send_accept_language_header = True
+
+filter_mapping = {0: 'images', 1: 'active', 2: 'active'}
+
+
+
[docs]def request(query, params):
+    """Google-Image search request"""
+
+    lang_info = get_lang_info(params, supported_languages, language_aliases, False)
+
+    query_url = (
+        'https://'
+        + lang_info['subdomain']
+        + '/search'
+        + "?"
+        + urlencode(
+            {
+                'q': query,
+                'tbm': "isch",
+                **lang_info['params'],
+                'ie': "utf8",
+                'oe': "utf8",
+                'asearch': 'isch',
+                'async': '_fmt:json,p:1,ijn:' + str(params['pageno']),
+            }
+        )
+    )
+
+    if params['time_range'] in time_range_dict:
+        query_url += '&' + urlencode({'tbs': 'qdr:' + time_range_dict[params['time_range']]})
+    if params['safesearch']:
+        query_url += '&' + urlencode({'safe': filter_mapping[params['safesearch']]})
+    params['url'] = query_url
+
+    params['headers'].update(lang_info['headers'])
+    params['headers']['User-Agent'] = 'NSTN/3.60.474802233.release Dalvik/2.1.0 (Linux; U; Android 12; US) gzip'
+    params['headers']['Accept'] = '*/*'
+    return params

+
+
+
[docs]def response(resp):
+    """Get response from google's search request"""
+    results = []
+
+    detect_google_sorry(resp)
+
+    json_start = resp.text.find('{"ischj":')
+    json_data = loads(resp.text[json_start:])
+
+    for item in json_data["ischj"]["metadata"]:
+
+        result_item = {
+            'url': item["result"]["referrer_url"],
+            'title': item["result"]["page_title"],
+            'content': item["text_in_grid"]["snippet"],
+            'source': item["result"]["site_title"],
+            'img_format': f'{item["original_image"]["width"]} x {item["original_image"]["height"]}',
+            'img_src': item["original_image"]["url"],
+            'thumbnail_src': item["thumbnail"]["url"],
+            'template': 'images.html',
+        }
+
+        author = item["result"].get('iptc', {}).get('creator')
+        if author:
+            result_item['author'] = ', '.join(author)
+
+        copyright_notice = item["result"].get('iptc', {}).get('copyright_notice')
+        if copyright_notice:
+            result_item['source'] += ' / ' + copyright_notice
+
+        file_size = item.get('gsa', {}).get('file_size')
+        if file_size:
+            result_item['source'] += ' (%s)' % file_size
+
+        results.append(result_item)
+
+    return results

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/engines/google_news.html b/_modules/searx/engines/google_news.html new file mode 100644 index 00000000..0cfe7b29 --- /dev/null +++ b/_modules/searx/engines/google_news.html @@ -0,0 +1,295 @@ + + + + + + + + + searx.engines.google_news — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.engines.google_news

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""This is the implementation of the google news engine.  The google news API
+ignores some parameters from the common :ref:`google API`:
+
+- num_ : the number of search results is ignored
+- save_ : is ignored / Google-News results are always *SafeSearch*
+
+.. _num: https://developers.google.com/custom-search/docs/xml_results#numsp
+.. _save: https://developers.google.com/custom-search/docs/xml_results#safesp
+
+"""
+
+# pylint: disable=invalid-name
+
+import binascii
+import re
+from urllib.parse import urlencode
+from base64 import b64decode
+from lxml import html
+
+from searx.utils import (
+    eval_xpath,
+    eval_xpath_list,
+    eval_xpath_getindex,
+    extract_text,
+)
+
+# pylint: disable=unused-import
+from searx.engines.google import (
+    supported_languages_url,
+    _fetch_supported_languages,
+)
+
+# pylint: enable=unused-import
+
+from searx.engines.google import (
+    get_lang_info,
+    detect_google_sorry,
+)
+
+# about
+about = {
+    "website": 'https://news.google.com',
+    "wikidata_id": 'Q12020',
+    "official_api_documentation": 'https://developers.google.com/custom-search',
+    "use_official_api": False,
+    "require_api_key": False,
+    "results": 'HTML',
+}
+
+# compared to other google engines google-news has a different time range
+# support.  The time range is included in the search term.
+time_range_dict = {
+    'day': 'when:1d',
+    'week': 'when:7d',
+    'month': 'when:1m',
+    'year': 'when:1y',
+}
+
+# engine dependent config
+
+categories = ['news']
+paging = False
+use_locale_domain = True
+time_range_support = True
+
+# Google-News results are always *SafeSearch*. Option 'safesearch' is set to
+# False here, otherwise checker will report safesearch-errors::
+#
+#  safesearch : results are identitical for safesearch=0 and safesearch=2
+safesearch = False
+send_accept_language_header = True
+
+
+
[docs]def request(query, params):
+    """Google-News search request"""
+
+    lang_info = get_lang_info(params, supported_languages, language_aliases, False)
+
+    # google news has only one domain
+    lang_info['subdomain'] = 'news.google.com'
+
+    ceid = "%s:%s" % (lang_info['country'], lang_info['language'])
+
+    # google news redirects en to en-US
+    if lang_info['params']['hl'] == 'en':
+        lang_info['params']['hl'] = 'en-US'
+
+    # Very special to google-news compared to other google engines, the time
+    # range is included in the search term.
+    if params['time_range']:
+        query += ' ' + time_range_dict[params['time_range']]
+
+    query_url = (
+        'https://'
+        + lang_info['subdomain']
+        + '/search'
+        + "?"
+        + urlencode({'q': query, **lang_info['params'], 'ie': "utf8", 'oe': "utf8", 'gl': lang_info['country']})
+        + ('&ceid=%s' % ceid)
+    )  # ceid includes a ':' character which must not be urlencoded
+    params['url'] = query_url
+
+    params['cookies']['CONSENT'] = "YES+"
+    params['headers'].update(lang_info['headers'])
+    params['headers']['Accept'] = 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8'
+
+    return params

+
+
+
[docs]def response(resp):
+    """Get response from google's search request"""
+    results = []
+
+    detect_google_sorry(resp)
+
+    # convert the text to dom
+    dom = html.fromstring(resp.text)
+
+    for result in eval_xpath_list(dom, '//div[@class="xrnccd"]'):
+
+        # The first <a> tag in the <article> contains the link to the
+        # article The href attribute of the <a> is a google internal link,
+        # we can't use.  The real link is hidden in the jslog attribute:
+        #
+        #   <a ...
+        #      jslog="95014; 4:https://www.cnn.com/.../index.html; track:click"
+        #      href="./articles/CAIiENu3nGS...?hl=en-US&amp;gl=US&amp;ceid=US%3Aen"
+        #      ... />
+
+        jslog = eval_xpath_getindex(result, './article/a/@jslog', 0)
+        url = re.findall('http[^;]*', jslog)
+        if url:
+            url = url[0]
+        else:
+            # The real URL is base64 encoded in the json attribute:
+            # jslog="95014; 5:W251bGwsbnVsbCxudW...giXQ==; track:click"
+            jslog = jslog.split(";")[1].split(':')[1].strip()
+            try:
+                padding = (4 - (len(jslog) % 4)) * "="
+                jslog = b64decode(jslog + padding)
+            except binascii.Error:
+                # URL can't be read, skip this result
+                continue
+
+            # now we have : b'[null, ... null,"https://www.cnn.com/.../index.html"]'
+            url = re.findall('http[^;"]*', str(jslog))[0]
+
+        # the first <h3> tag in the <article> contains the title of the link
+        title = extract_text(eval_xpath(result, './article/h3[1]'))
+
+        # The pub_date is mostly a string like 'yesertday', not a real
+        # timezone date or time.  Therefore we can't use publishedDate.
+        pub_date = extract_text(eval_xpath(result, './article/div[1]/div[1]/time'))
+        pub_origin = extract_text(eval_xpath(result, './article/div[1]/div[1]/a'))
+
+        content = ' / '.join([x for x in [pub_origin, pub_date] if x])
+
+        # The image URL is located in a preceding sibling <img> tag, e.g.:
+        # "https://lh3.googleusercontent.com/DjhQh7DMszk.....z=-p-h100-w100"
+        # These URL are long but not personalized (double checked via tor).
+
+        img_src = extract_text(result.xpath('preceding-sibling::a/figure/img/@src'))
+
+        results.append(
+            {
+                'url': url,
+                'title': title,
+                'content': content,
+                'img_src': img_src,
+            }
+        )
+
+    # return results
+    return results

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/engines/google_videos.html b/_modules/searx/engines/google_videos.html new file mode 100644 index 00000000..1b08e334 --- /dev/null +++ b/_modules/searx/engines/google_videos.html @@ -0,0 +1,309 @@ + + + + + + + + + searx.engines.google_videos — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.engines.google_videos

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""This is the implementation of the google videos engine.
+
+.. admonition:: Content-Security-Policy (CSP)
+
+   This engine needs to allow images from the `data URLs`_ (prefixed with the
+   ``data:`` scheme)::
+
+     Header set Content-Security-Policy "img-src 'self' data: ;"
+
+.. _data URLs:
+   https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs
+
+"""
+
+# pylint: disable=invalid-name
+
+import re
+from urllib.parse import urlencode
+from lxml import html
+
+from searx.utils import (
+    eval_xpath,
+    eval_xpath_list,
+    eval_xpath_getindex,
+    extract_text,
+)
+
+from searx.engines.google import (
+    get_lang_info,
+    time_range_dict,
+    filter_mapping,
+    g_section_with_header,
+    title_xpath,
+    suggestion_xpath,
+    detect_google_sorry,
+)
+
+# pylint: disable=unused-import
+from searx.engines.google import supported_languages_url, _fetch_supported_languages
+
+# pylint: enable=unused-import
+
+# about
+about = {
+    "website": 'https://www.google.com',
+    "wikidata_id": 'Q219885',
+    "official_api_documentation": 'https://developers.google.com/custom-search',
+    "use_official_api": False,
+    "require_api_key": False,
+    "results": 'HTML',
+}
+
+# engine dependent config
+
+categories = ['videos', 'web']
+paging = False
+language_support = True
+use_locale_domain = True
+time_range_support = True
+safesearch = True
+send_accept_language_header = True
+
+RE_CACHE = {}
+
+
+def _re(regexpr):
+    """returns compiled regular expression"""
+    RE_CACHE[regexpr] = RE_CACHE.get(regexpr, re.compile(regexpr))
+    return RE_CACHE[regexpr]
+
+
+def scrap_out_thumbs_src(dom):
+    ret_val = {}
+    thumb_name = 'dimg_'
+    for script in eval_xpath_list(dom, '//script[contains(., "google.ldi={")]'):
+        _script = script.text
+        # "dimg_35":"https://i.ytimg.c....",
+        _dimurl = _re("s='([^']*)").findall(_script)
+        for k, v in _re('(' + thumb_name + '[0-9]*)":"(http[^"]*)').findall(_script):
+            v = v.replace(r'\u003d', '=')
+            v = v.replace(r'\u0026', '&')
+            ret_val[k] = v
+    logger.debug("found %s imgdata for: %s", thumb_name, ret_val.keys())
+    return ret_val
+
+
+
[docs]def scrap_out_thumbs(dom):
+    """Scrap out thumbnail data from <script> tags."""
+    ret_val = {}
+    thumb_name = 'dimg_'
+
+    for script in eval_xpath_list(dom, '//script[contains(., "_setImagesSrc")]'):
+        _script = script.text
+
+        # var s='data:image/jpeg;base64, ...'
+        _imgdata = _re("s='([^']*)").findall(_script)
+        if not _imgdata:
+            continue
+
+        # var ii=['dimg_17']
+        for _vidthumb in _re(r"(%s\d+)" % thumb_name).findall(_script):
+            # At least the equal sign in the URL needs to be decoded
+            ret_val[_vidthumb] = _imgdata[0].replace(r"\x3d", "=")
+
+    logger.debug("found %s imgdata for: %s", thumb_name, ret_val.keys())
+    return ret_val

+
+
+
[docs]def request(query, params):
+    """Google-Video search request"""
+
+    lang_info = get_lang_info(params, supported_languages, language_aliases, False)
+
+    query_url = (
+        'https://'
+        + lang_info['subdomain']
+        + '/search'
+        + "?"
+        + urlencode({'q': query, 'tbm': "vid", **lang_info['params'], 'ie': "utf8", 'oe': "utf8"})
+    )
+
+    if params['time_range'] in time_range_dict:
+        query_url += '&' + urlencode({'tbs': 'qdr:' + time_range_dict[params['time_range']]})
+    if params['safesearch']:
+        query_url += '&' + urlencode({'safe': filter_mapping[params['safesearch']]})
+    params['url'] = query_url
+
+    params['cookies']['CONSENT'] = "YES+"
+    params['headers'].update(lang_info['headers'])
+    params['headers']['Accept'] = 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8'
+    return params

+
+
+
[docs]def response(resp):
+    """Get response from google's search request"""
+    results = []
+
+    detect_google_sorry(resp)
+
+    # convert the text to dom
+    dom = html.fromstring(resp.text)
+    vidthumb_imgdata = scrap_out_thumbs(dom)
+    thumbs_src = scrap_out_thumbs_src(dom)
+    logger.debug(str(thumbs_src))
+
+    # parse results
+    for result in eval_xpath_list(dom, '//div[contains(@class, "g ")]'):
+
+        # ignore google *sections*
+        if extract_text(eval_xpath(result, g_section_with_header)):
+            logger.debug("ignoring <g-section-with-header>")
+            continue
+
+        # ingnore articles without an image id / e.g. news articles
+        img_id = eval_xpath_getindex(result, './/g-img/img/@id', 0, default=None)
+        if img_id is None:
+            logger.error("no img_id found in item %s (news article?)", len(results) + 1)
+            continue
+
+        img_src = vidthumb_imgdata.get(img_id, None)
+        if not img_src:
+            img_src = thumbs_src.get(img_id, "")
+
+        title = extract_text(eval_xpath_getindex(result, title_xpath, 0))
+        url = eval_xpath_getindex(result, './/div[@class="dXiKIc"]//a/@href', 0)
+        length = extract_text(eval_xpath(result, './/div[contains(@class, "P7xzyf")]/span/span'))
+        c_node = eval_xpath_getindex(result, './/div[@class="Uroaid"]', 0)
+        content = extract_text(c_node)
+        pub_info = extract_text(eval_xpath(result, './/div[@class="Zg1NU"]'))
+
+        results.append(
+            {
+                'url': url,
+                'title': title,
+                'content': content,
+                'length': length,
+                'author': pub_info,
+                'thumbnail': img_src,
+                'template': 'videos.html',
+            }
+        )
+
+    # parse suggestion
+    for suggestion in eval_xpath_list(dom, suggestion_xpath):
+        # append suggestion
+        results.append({'suggestion': extract_text(suggestion)})
+
+    return results

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/engines/tineye.html b/_modules/searx/engines/tineye.html new file mode 100644 index 00000000..f0481017 --- /dev/null +++ b/_modules/searx/engines/tineye.html @@ -0,0 +1,344 @@ + + + + + + + + + searx.engines.tineye — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.engines.tineye

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""This engine implements *Tineye - reverse image search*
+
+Using TinEye, you can search by image or perform what we call a reverse image
+search.  You can do that by uploading an image or searching by URL. You can also
+simply drag and drop your images to start your search.  TinEye constantly crawls
+the web and adds images to its index.  Today, the TinEye index is over 50.2
+billion images `[tineye.com] <https://tineye.com/how>`_.
+
+.. hint::
+
+   This SearXNG engine only supports *'searching by URL'* and it does not use
+   the official API `[api.tineye.com] <https://api.tineye.com/python/docs/>`_.
+
+"""
+
+from urllib.parse import urlencode
+from datetime import datetime
+from flask_babel import gettext
+
+about = {
+    "website": 'https://tineye.com',
+    "wikidata_id": 'Q2382535',
+    "official_api_documentation": 'https://api.tineye.com/python/docs/',
+    "use_official_api": False,
+    "require_api_key": False,
+    "results": 'JSON',
+}
+
+engine_type = 'online_url_search'
+""":py:obj:`searx.search.processors.online_url_search`"""
+
+categories = ['general']
+paging = True
+safesearch = False
+base_url = 'https://tineye.com'
+search_string = '/result_json/?page={page}&{query}'
+
+FORMAT_NOT_SUPPORTED = gettext(
+    "Could not read that image url. This may be due to an unsupported file"
+    " format. TinEye only supports images that are JPEG, PNG, GIF, BMP, TIFF or WebP."
+)
+"""TinEye error message"""
+
+NO_SIGNATURE_ERROR = gettext(
+    "The image is too simple to find matches. TinEye requires a basic level of"
+    " visual detail to successfully identify matches."
+)
+"""TinEye error message"""
+
+DOWNLOAD_ERROR = gettext("The image could not be downloaded.")
+"""TinEye error message"""
+
+
+
[docs]def request(query, params):
+    """Build TinEye HTTP request using ``search_urls`` of a :py:obj:`engine_type`."""
+
+    params['raise_for_httperror'] = False
+
+    if params['search_urls']['data:image']:
+        query = params['search_urls']['data:image']
+    elif params['search_urls']['http']:
+        query = params['search_urls']['http']
+
+    logger.debug("query URL: %s", query)
+    query = urlencode({'url': query})
+
+    # see https://github.com/TinEye/pytineye/blob/main/pytineye/api.py
+    params['url'] = base_url + search_string.format(query=query, page=params['pageno'])
+
+    params['headers'].update(
+        {
+            'Connection': 'keep-alive',
+            'Accept-Encoding': 'gzip, defalte, br',
+            'Host': 'tineye.com',
+            'DNT': '1',
+            'TE': 'trailers',
+        }
+    )
+    return params

+
+
+
[docs]def parse_tineye_match(match_json):
+    """Takes parsed JSON from the API server and turns it into a :py:obj:`dict`
+    object.
+
+    Attributes `(class Match) <https://github.com/TinEye/pytineye/blob/main/pytineye/api.py>`__
+
+    - `image_url`, link to the result image.
+    - `domain`, domain this result was found on.
+    - `score`, a number (0 to 100) that indicates how closely the images match.
+    - `width`, image width in pixels.
+    - `height`, image height in pixels.
+    - `size`, image area in pixels.
+    - `format`, image format.
+    - `filesize`, image size in bytes.
+    - `overlay`, overlay URL.
+    - `tags`, whether this match belongs to a collection or stock domain.
+
+    - `backlinks`, a list of Backlink objects pointing to the original websites
+      and image URLs. List items are instances of :py:obj:`dict`, (`Backlink
+      <https://github.com/TinEye/pytineye/blob/main/pytineye/api.py>`__):
+
+      - `url`, the image URL to the image.
+      - `backlink`, the original website URL.
+      - `crawl_date`, the date the image was crawled.
+
+    """
+
+    # HINT: there exists an alternative backlink dict in the domains list / e.g.::
+    #
+    #     match_json['domains'][0]['backlinks']
+
+    backlinks = []
+    if "backlinks" in match_json:
+
+        for backlink_json in match_json["backlinks"]:
+            if not isinstance(backlink_json, dict):
+                continue
+
+            crawl_date = backlink_json.get("crawl_date")
+            if crawl_date:
+                crawl_date = datetime.fromisoformat(crawl_date[:-3])
+            else:
+                crawl_date = datetime.min
+
+            backlinks.append(
+                {
+                    'url': backlink_json.get("url"),
+                    'backlink': backlink_json.get("backlink"),
+                    'crawl_date': crawl_date,
+                    'image_name': backlink_json.get("image_name"),
+                }
+            )
+
+    return {
+        'image_url': match_json.get("image_url"),
+        'domain': match_json.get("domain"),
+        'score': match_json.get("score"),
+        'width': match_json.get("width"),
+        'height': match_json.get("height"),
+        'size': match_json.get("size"),
+        'image_format': match_json.get("format"),
+        'filesize': match_json.get("filesize"),
+        'overlay': match_json.get("overlay"),
+        'tags': match_json.get("tags"),
+        'backlinks': backlinks,
+    }

+
+
+
[docs]def response(resp):
+    """Parse HTTP response from TinEye."""
+    results = []
+
+    try:
+        json_data = resp.json()
+    except Exception as exc:  # pylint: disable=broad-except
+        msg = "can't parse JSON response // %s" % exc
+        logger.error(msg)
+        json_data = {'error': msg}
+
+    # handle error codes from Tineye
+
+    if resp.is_error:
+        if resp.status_code in (400, 422):
+
+            message = 'HTTP status: %s' % resp.status_code
+            error = json_data.get('error')
+            s_key = json_data.get('suggestions', {}).get('key', '')
+
+            if error and s_key:
+                message = "%s (%s)" % (error, s_key)
+            elif error:
+                message = error
+
+            if s_key == "Invalid image URL":
+                # test https://docs.searxng.org/_static/searxng-wordmark.svg
+                message = FORMAT_NOT_SUPPORTED
+            elif s_key == 'NO_SIGNATURE_ERROR':
+                # test https://pngimg.com/uploads/dot/dot_PNG4.png
+                message = NO_SIGNATURE_ERROR
+            elif s_key == 'Download Error':
+                # test https://notexists
+                message = DOWNLOAD_ERROR
+
+            # see https://github.com/searxng/searxng/pull/1456#issuecomment-1193105023
+            # results.append({'answer': message})
+            logger.error(message)
+
+            return results
+
+        resp.raise_for_status()
+
+    # append results from matches
+
+    for match_json in json_data['matches']:
+
+        tineye_match = parse_tineye_match(match_json)
+        if not tineye_match['backlinks']:
+            continue
+
+        backlink = tineye_match['backlinks'][0]
+        results.append(
+            {
+                'template': 'images.html',
+                'url': backlink['backlink'],
+                'thumbnail_src': tineye_match['image_url'],
+                'source': backlink['url'],
+                'title': backlink['image_name'],
+                'img_src': backlink['url'],
+                'format': tineye_match['image_format'],
+                'widht': tineye_match['width'],
+                'height': tineye_match['height'],
+                'publishedDate': backlink['crawl_date'],
+            }
+        )
+
+    # append number of results
+
+    number_of_results = json_data.get('num_matches')
+    if number_of_results:
+        results.append({'number_of_results': number_of_results})
+
+    return results

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/engines/xpath.html b/_modules/searx/engines/xpath.html new file mode 100644 index 00000000..b127fd00 --- /dev/null +++ b/_modules/searx/engines/xpath.html @@ -0,0 +1,377 @@ + + + + + + + + + searx.engines.xpath — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.engines.xpath

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""The XPath engine is a *generic* engine with which it is possible to configure
+engines in the settings.
+
+Here is a simple example of a XPath engine configured in the
+:ref:`settings engine` section, further read :ref:`engines-dev`.
+
+.. code:: yaml
+
+  - name : bitbucket
+    engine : xpath
+    paging : True
+    search_url : https://bitbucket.org/repo/all/{pageno}?name={query}
+    url_xpath : //article[@class="repo-summary"]//a[@class="repo-link"]/@href
+    title_xpath : //article[@class="repo-summary"]//a[@class="repo-link"]
+    content_xpath : //article[@class="repo-summary"]/p
+
+"""
+
+from urllib.parse import urlencode
+
+from lxml import html
+from searx.utils import extract_text, extract_url, eval_xpath, eval_xpath_list
+from searx.network import raise_for_httperror
+
+search_url = None
+"""
+Search URL of the engine. Example::
+
+    https://example.org/?search={query}&page={pageno}{time_range}{safe_search}
+
+Replacements are:
+
+``{query}``:
+  Search terms from user.
+
+``{pageno}``:
+  Page number if engine supports pagging :py:obj:`paging`
+
+``{lang}``:
+  ISO 639-1 language code (en, de, fr ..)
+
+``{time_range}``:
+  :py:obj:`URL parameter <time_range_url>` if engine :py:obj:`supports time
+  range <time_range_support>`.  The value for the parameter is taken from
+  :py:obj:`time_range_map`.
+
+``{safe_search}``:
+  Safe-search :py:obj:`URL parameter <safe_search_map>` if engine
+  :py:obj:`supports safe-search <safe_search_support>`.  The ``{safe_search}``
+  replacement is taken from the :py:obj:`safes_search_map`.  Filter results::
+
+      0: none, 1: moderate, 2:strict
+
+  If not supported, the URL parameter is an empty string.
+
+"""
+
+lang_all = 'en'
+'''Replacement ``{lang}`` in :py:obj:`search_url` if language ``all`` is
+selected.
+'''
+
+no_result_for_http_status = []
+'''Return empty result for these HTTP status codes instead of throwing an error.
+
+.. code:: yaml
+
+    no_result_for_http_status: []
+'''
+
+soft_max_redirects = 0
+'''Maximum redirects, soft limit. Record an error but don't stop the engine'''
+
+results_xpath = ''
+'''XPath selector for the list of result items'''
+
+url_xpath = None
+'''XPath selector of result's ``url``.'''
+
+content_xpath = None
+'''XPath selector of result's ``content``.'''
+
+title_xpath = None
+'''XPath selector of result's ``title``.'''
+
+thumbnail_xpath = False
+'''XPath selector of result's ``img_src``.'''
+
+suggestion_xpath = ''
+'''XPath selector of result's ``suggestion``.'''
+
+cached_xpath = ''
+cached_url = ''
+
+cookies = {}
+headers = {}
+'''Some engines might offer different result based on cookies or headers.
+Possible use-case: To set safesearch cookie or header to moderate.'''
+
+paging = False
+'''Engine supports paging [True or False].'''
+
+page_size = 1
+'''Number of results on each page.  Only needed if the site requires not a page
+number, but an offset.'''
+
+first_page_num = 1
+'''Number of the first page (usually 0 or 1).'''
+
+time_range_support = False
+'''Engine supports search time range.'''
+
+time_range_url = '&hours={time_range_val}'
+'''Time range URL parameter in the in :py:obj:`search_url`.  If no time range is
+requested by the user, the URL parameter is an empty string.  The
+``{time_range_val}`` replacement is taken from the :py:obj:`time_range_map`.
+
+.. code:: yaml
+
+    time_range_url : '&days={time_range_val}'
+'''
+
+time_range_map = {
+    'day': 24,
+    'week': 24 * 7,
+    'month': 24 * 30,
+    'year': 24 * 365,
+}
+'''Maps time range value from user to ``{time_range_val}`` in
+:py:obj:`time_range_url`.
+
+.. code:: yaml
+
+    time_range_map:
+      day: 1
+      week: 7
+      month: 30
+      year: 365
+'''
+
+safe_search_support = False
+'''Engine supports safe-search.'''
+
+safe_search_map = {0: '&filter=none', 1: '&filter=moderate', 2: '&filter=strict'}
+'''Maps safe-search value to ``{safe_search}`` in :py:obj:`search_url`.
+
+.. code:: yaml
+
+    safesearch: true
+    safes_search_map:
+      0: '&filter=none'
+      1: '&filter=moderate'
+      2: '&filter=strict'
+
+'''
+
+
+
[docs]def request(query, params):
+    '''Build request parameters (see :ref:`engine request`).'''
+    lang = lang_all
+    if params['language'] != 'all':
+        lang = params['language'][:2]
+
+    time_range = ''
+    if params.get('time_range'):
+        time_range_val = time_range_map.get(params.get('time_range'))
+        time_range = time_range_url.format(time_range_val=time_range_val)
+
+    safe_search = ''
+    if params['safesearch']:
+        safe_search = safe_search_map[params['safesearch']]
+
+    fargs = {
+        'query': urlencode({'q': query})[2:],
+        'lang': lang,
+        'pageno': (params['pageno'] - 1) * page_size + first_page_num,
+        'time_range': time_range,
+        'safe_search': safe_search,
+    }
+
+    params['cookies'].update(cookies)
+    params['headers'].update(headers)
+
+    params['url'] = search_url.format(**fargs)
+    params['soft_max_redirects'] = soft_max_redirects
+
+    params['raise_for_httperror'] = False
+
+    return params

+
+
+
[docs]def response(resp):  # pylint: disable=too-many-branches
+    '''Scrap *results* from the response (see :ref:`engine results`).'''
+    if no_result_for_http_status and resp.status_code in no_result_for_http_status:
+        return []
+
+    raise_for_httperror(resp)
+
+    results = []
+    dom = html.fromstring(resp.text)
+    is_onion = 'onions' in categories
+
+    if results_xpath:
+        for result in eval_xpath_list(dom, results_xpath):
+
+            url = extract_url(eval_xpath_list(result, url_xpath, min_len=1), search_url)
+            title = extract_text(eval_xpath_list(result, title_xpath, min_len=1))
+            content = extract_text(eval_xpath_list(result, content_xpath))
+            tmp_result = {'url': url, 'title': title, 'content': content}
+
+            # add thumbnail if available
+            if thumbnail_xpath:
+                thumbnail_xpath_result = eval_xpath_list(result, thumbnail_xpath)
+                if len(thumbnail_xpath_result) > 0:
+                    tmp_result['img_src'] = extract_url(thumbnail_xpath_result, search_url)
+
+            # add alternative cached url if available
+            if cached_xpath:
+                tmp_result['cached_url'] = cached_url + extract_text(eval_xpath_list(result, cached_xpath, min_len=1))
+
+            if is_onion:
+                tmp_result['is_onion'] = True
+
+            results.append(tmp_result)
+
+    else:
+        if cached_xpath:
+            for url, title, content, cached in zip(
+                (extract_url(x, search_url) for x in eval_xpath_list(dom, url_xpath)),
+                map(extract_text, eval_xpath_list(dom, title_xpath)),
+                map(extract_text, eval_xpath_list(dom, content_xpath)),
+                map(extract_text, eval_xpath_list(dom, cached_xpath)),
+            ):
+                results.append(
+                    {
+                        'url': url,
+                        'title': title,
+                        'content': content,
+                        'cached_url': cached_url + cached,
+                        'is_onion': is_onion,
+                    }
+                )
+        else:
+            for url, title, content in zip(
+                (extract_url(x, search_url) for x in eval_xpath_list(dom, url_xpath)),
+                map(extract_text, eval_xpath_list(dom, title_xpath)),
+                map(extract_text, eval_xpath_list(dom, content_xpath)),
+            ):
+                results.append({'url': url, 'title': title, 'content': content, 'is_onion': is_onion})
+
+    if suggestion_xpath:
+        for suggestion in eval_xpath(dom, suggestion_xpath):
+            results.append({'suggestion': extract_text(suggestion)})
+
+    logger.debug("found %s results", len(results))
+    return results

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/engines/yahoo.html b/_modules/searx/engines/yahoo.html new file mode 100644 index 00000000..d93adc0e --- /dev/null +++ b/_modules/searx/engines/yahoo.html @@ -0,0 +1,285 @@ + + + + + + + + + searx.engines.yahoo — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.engines.yahoo

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""Yahoo Search (Web)
+
+Languages are supported by mapping the language to a domain.  If domain is not
+found in :py:obj:`lang2domain` URL ``<lang>.search.yahoo.com`` is used.
+
+"""
+
+from urllib.parse import (
+    unquote,
+    urlencode,
+)
+from lxml import html
+
+from searx.utils import (
+    eval_xpath_getindex,
+    eval_xpath_list,
+    extract_text,
+    match_language,
+)
+
+# about
+about = {
+    "website": 'https://search.yahoo.com/',
+    "wikidata_id": None,
+    "official_api_documentation": 'https://developer.yahoo.com/api/',
+    "use_official_api": False,
+    "require_api_key": False,
+    "results": 'HTML',
+}
+
+# engine dependent config
+categories = ['general', 'web']
+paging = True
+time_range_support = True
+supported_languages_url = 'https://search.yahoo.com/preferences/languages'
+"""Supported languages are read from Yahoo preference page."""
+
+time_range_dict = {
+    'day': ('1d', 'd'),
+    'week': ('1w', 'w'),
+    'month': ('1m', 'm'),
+}
+
+language_aliases = {
+    'zh-HK': 'zh_chs',
+    'zh-CN': 'zh_chs',  # dead since 2015 / routed to hk.search.yahoo.com
+    'zh-TW': 'zh_cht',
+}
+
+lang2domain = {
+    'zh_chs': 'hk.search.yahoo.com',
+    'zh_cht': 'tw.search.yahoo.com',
+    'en': 'search.yahoo.com',
+    'bg': 'search.yahoo.com',
+    'cs': 'search.yahoo.com',
+    'da': 'search.yahoo.com',
+    'el': 'search.yahoo.com',
+    'et': 'search.yahoo.com',
+    'he': 'search.yahoo.com',
+    'hr': 'search.yahoo.com',
+    'ja': 'search.yahoo.com',
+    'ko': 'search.yahoo.com',
+    'sk': 'search.yahoo.com',
+    'sl': 'search.yahoo.com',
+}
+"""Map language to domain"""
+
+
+def _get_language(params):
+
+    lang = language_aliases.get(params['language'])
+    if lang is None:
+        lang = match_language(params['language'], supported_languages, language_aliases)
+    lang = lang.split('-')[0]
+    logger.debug("params['language']: %s --> %s", params['language'], lang)
+    return lang
+
+
+
[docs]def request(query, params):
+    """build request"""
+    offset = (params['pageno'] - 1) * 7 + 1
+    lang = _get_language(params)
+    age, btf = time_range_dict.get(params['time_range'], ('', ''))
+
+    args = urlencode(
+        {
+            'p': query,
+            'ei': 'UTF-8',
+            'fl': 1,
+            'vl': 'lang_' + lang,
+            'btf': btf,
+            'fr2': 'time',
+            'age': age,
+            'b': offset,
+            'xargs': 0,
+        }
+    )
+
+    domain = lang2domain.get(lang, '%s.search.yahoo.com' % lang)
+    params['url'] = 'https://%s/search?%s' % (domain, args)
+    return params

+
+
+
[docs]def parse_url(url_string):
+    """remove yahoo-specific tracking-url"""
+
+    endings = ['/RS', '/RK']
+    endpositions = []
+    start = url_string.find('http', url_string.find('/RU=') + 1)
+
+    for ending in endings:
+        endpos = url_string.rfind(ending)
+        if endpos > -1:
+            endpositions.append(endpos)
+
+    if start == 0 or len(endpositions) == 0:
+        return url_string
+
+    end = min(endpositions)
+    return unquote(url_string[start:end])

+
+
+
[docs]def response(resp):
+    """parse response"""
+
+    results = []
+    dom = html.fromstring(resp.text)
+
+    # parse results
+    for result in eval_xpath_list(dom, '//div[contains(@class,"algo-sr")]'):
+        url = eval_xpath_getindex(result, './/h3/a/@href', 0, default=None)
+        if url is None:
+            continue
+        url = parse_url(url)
+
+        title = eval_xpath_getindex(result, './/h3/a', 0, default=None)
+        if title is None:
+            continue
+        offset = len(extract_text(title.xpath('span')))
+        title = extract_text(title)[offset:]
+
+        content = eval_xpath_getindex(result, './/div[contains(@class, "compText")]', 0, default='')
+        content = extract_text(content, allow_none=True)
+
+        # append result
+        results.append({'url': url, 'title': title, 'content': content})
+
+    for suggestion in eval_xpath_list(dom, '//div[contains(@class, "AlsoTry")]//table//a'):
+        # append suggestion
+        results.append({'suggestion': extract_text(suggestion)})
+
+    return results

+
+
+# get supported languages from their site
+def _fetch_supported_languages(resp):
+    supported_languages = []
+    dom = html.fromstring(resp.text)
+    offset = len('lang_')
+
+    for val in eval_xpath_list(dom, '//div[contains(@class, "lang-item")]/input/@value'):
+        supported_languages.append(val[offset:])
+
+    return supported_languages
+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/infopage.html b/_modules/searx/infopage.html new file mode 100644 index 00000000..1f22194a --- /dev/null +++ b/_modules/searx/infopage.html @@ -0,0 +1,302 @@ + + + + + + + + + searx.infopage — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.infopage

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+# pyright: basic
+"""Render SearXNG instance documentation.
+
+Usage in a Flask app route:
+
+.. code:: python
+
+  from searx import infopage
+
+  _INFO_PAGES = infopage.InfoPageSet(infopage.MistletoePage)
+
+  @app.route('/info/<pagename>', methods=['GET'])
+  def info(pagename):
+
+      locale = request.preferences.get_value('locale')
+      page = _INFO_PAGES.get_page(pagename, locale)
+
+"""
+
+__all__ = ['InfoPage', 'InfoPageSet']
+
+import os
+import os.path
+import logging
+import typing
+
+import urllib.parse
+import jinja2
+from flask.helpers import url_for
+from markdown_it import MarkdownIt
+
+from .. import get_setting
+from ..compat import cached_property
+from ..version import GIT_URL
+from ..locales import LOCALE_NAMES
+
+
+logger = logging.getLogger('searx.infopage')
+_INFO_FOLDER = os.path.abspath(os.path.dirname(__file__))
+
+
+
[docs]class InfoPage:
+    """A page of the :py:obj:`online documentation <InfoPageSet>`."""
+
+    def __init__(self, fname):
+        self.fname = fname
+
+    @cached_property
+    def raw_content(self):
+        """Raw content of the page (without any jinja rendering)"""
+        with open(self.fname, 'r', encoding='utf-8') as f:
+            return f.read()
+
+    @cached_property
+    def content(self):
+        """Content of the page (rendered in a Jinja conntext)"""
+        ctx = self.get_ctx()
+        template = jinja2.Environment().from_string(self.raw_content)
+        return template.render(**ctx)
+
+    @cached_property
+    def title(self):
+        """Title of the content (without any markup)"""
+        t = ""
+        for l in self.raw_content.split('\n'):
+            if l.startswith('# '):
+                t = l.strip('# ')
+        return t
+
+    @cached_property
+    def html(self):
+        """Render Markdown (CommonMark_) to HTML by using markdown-it-py_.
+
+        .. _CommonMark: https://commonmark.org/
+        .. _markdown-it-py: https://github.com/executablebooks/markdown-it-py
+
+        """
+        return (
+            MarkdownIt("commonmark", {"typographer": True}).enable(["replacements", "smartquotes"]).render(self.content)
+        )
+
+
[docs]    def get_ctx(self):
+        """Jinja context to render :py:obj:`InfoPage.content`"""
+
+        def _md_link(name, url):
+            url = url_for(url, _external=True)
+            return "[%s](%s)" % (name, url)
+
+        def _md_search(query):
+            url = '%s?q=%s' % (url_for('search', _external=True), urllib.parse.quote(query))
+            return '[%s](%s)' % (query, url)
+
+        ctx = {}
+        ctx['GIT_URL'] = GIT_URL
+        ctx['get_setting'] = get_setting
+        ctx['link'] = _md_link
+        ctx['search'] = _md_search
+
+        return ctx

+
+    def __repr__(self):
+        return f'<{self.__class__.__name__} fname={self.fname!r}>'

+
+
+
[docs]class InfoPageSet:  # pylint: disable=too-few-public-methods
+    """Cached rendering of the online documentation a SearXNG instance has.
+
+    :param page_class: render online documentation by :py:obj:`InfoPage` parser.
+    :type page_class: :py:obj:`InfoPage`
+
+    :param info_folder: information directory
+    :type info_folder: str
+    """
+
+    def __init__(
+        self, page_class: typing.Optional[typing.Type[InfoPage]] = None, info_folder: typing.Optional[str] = None
+    ):
+        self.page_class = page_class or InfoPage
+        self.folder: str = info_folder or _INFO_FOLDER
+        """location of the Markdwon files"""
+
+        self.CACHE: typing.Dict[tuple, typing.Optional[InfoPage]] = {}
+
+        self.locale_default: str = 'en'
+        """default language"""
+
+        self.locales: typing.List[str] = [
+            locale.replace('_', '-') for locale in os.listdir(_INFO_FOLDER) if locale.replace('_', '-') in LOCALE_NAMES
+        ]
+        """list of supported languages (aka locales)"""
+
+        self.toc: typing.List[str] = [
+            'search-syntax',
+            'about',
+            'donate',
+        ]
+        """list of articles in the online documentation"""
+
+
[docs]    def get_page(self, pagename: str, locale: typing.Optional[str] = None):
+        """Return ``pagename`` instance of :py:obj:`InfoPage`
+
+        :param pagename: name of the page, a value from :py:obj:`InfoPageSet.toc`
+        :type pagename: str
+
+        :param locale: language of the page, e.g. ``en``, ``zh_Hans_CN``
+                       (default: :py:obj:`InfoPageSet.i18n_origin`)
+        :type locale: str
+
+        """
+        locale = locale or self.locale_default
+
+        if pagename not in self.toc:
+            return None
+        if locale not in self.locales:
+            return None
+
+        cache_key = (pagename, locale)
+
+        if cache_key in self.CACHE:
+            return self.CACHE[cache_key]
+
+        # not yet instantiated
+
+        fname = os.path.join(self.folder, locale.replace('-', '_'), pagename) + '.md'
+        if not os.path.exists(fname):
+            logger.info('file %s does not exists', fname)
+            self.CACHE[cache_key] = None
+            return None
+
+        page = self.page_class(fname)
+        self.CACHE[cache_key] = page
+        return page

+
+
[docs]    def iter_pages(self, locale: typing.Optional[str] = None, fallback_to_default=False):
+        """Iterate over all pages of the TOC"""
+        locale = locale or self.locale_default
+        for page_name in self.toc:
+            page_locale = locale
+            page = self.get_page(page_name, locale)
+            if fallback_to_default and page is None:
+                page_locale = self.locale_default
+                page = self.get_page(page_name, self.locale_default)
+            if page is not None:
+                # page is None if the page was deleted by the administrator
+                yield page_name, page_locale, page

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/locales.html b/_modules/searx/locales.html new file mode 100644 index 00000000..6d48a850 --- /dev/null +++ b/_modules/searx/locales.html @@ -0,0 +1,420 @@ + + + + + + + + + searx.locales — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.locales

+# -*- coding: utf-8 -*-
+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""Initialize :py:obj:`LOCALE_NAMES`, :py:obj:`RTL_LOCALES`.
+"""
+
+from typing import Set
+import os
+import pathlib
+
+from babel import Locale
+from babel.support import Translations
+import babel.languages
+import babel.core
+import flask_babel
+import flask
+from flask.ctx import has_request_context
+from searx import logger
+
+logger = logger.getChild('locales')
+
+
+# safe before monkey patching flask_babel.get_translations
+_flask_babel_get_translations = flask_babel.get_translations
+
+LOCALE_NAMES = {}
+"""Mapping of locales and their description.  Locales e.g. 'fr' or 'pt-BR' (see
+:py:obj:`locales_initialize`).
+
+:meta hide-value:
+"""
+
+RTL_LOCALES: Set[str] = set()
+"""List of *Right-To-Left* locales e.g. 'he' or 'fa-IR' (see
+:py:obj:`locales_initialize`)."""
+
+ADDITIONAL_TRANSLATIONS = {
+    "dv": "ދިވެހި (Dhivehi)",
+    "oc": "Occitan",
+    "szl": "Ślōnski (Silesian)",
+    "pap": "Papiamento",
+}
+"""Additional languages SearXNG has translations for but not supported by
+python-babel (see :py:obj:`locales_initialize`)."""
+
+LOCALE_BEST_MATCH = {
+    "dv": "si",
+    "oc": 'fr-FR',
+    "szl": "pl",
+    "nl-BE": "nl",
+    "zh-HK": "zh-Hant-TW",
+    "pap": "pt-BR",
+}
+"""Map a locale we do not have a translations for to a locale we have a
+translation for. By example: use Taiwan version of the translation for Hong
+Kong."""
+
+
+def localeselector():
+    locale = 'en'
+    if has_request_context():
+        value = flask.request.preferences.get_value('locale')
+        if value:
+            locale = value
+
+    # first, set the language that is not supported by babel
+    if locale in ADDITIONAL_TRANSLATIONS:
+        flask.request.form['use-translation'] = locale
+
+    # second, map locale to a value python-babel supports
+    locale = LOCALE_BEST_MATCH.get(locale, locale)
+
+    if locale == '':
+        # if there is an error loading the preferences
+        # the locale is going to be ''
+        locale = 'en'
+
+    # babel uses underscore instead of hyphen.
+    locale = locale.replace('-', '_')
+    return locale
+
+
+
[docs]def get_translations():
+    """Monkey patch of :py:obj:`flask_babel.get_translations`"""
+    if has_request_context():
+        use_translation = flask.request.form.get('use-translation')
+        if use_translation in ADDITIONAL_TRANSLATIONS:
+            babel_ext = flask_babel.current_app.extensions['babel']
+            return Translations.load(next(babel_ext.translation_directories), use_translation)
+    return _flask_babel_get_translations()

+
+
+
[docs]def get_locale_descr(locale, locale_name):
+    """Get locale name e.g. 'Français - fr' or 'Português (Brasil) - pt-BR'
+
+    :param locale: instance of :py:class:`Locale`
+    :param locale_name: name e.g. 'fr'  or 'pt_BR' (delimiter is *underscore*)
+    """
+
+    native_language, native_territory = _get_locale_descr(locale, locale_name)
+    english_language, english_territory = _get_locale_descr(locale, 'en')
+
+    if native_territory == english_territory:
+        english_territory = None
+
+    if not native_territory and not english_territory:
+        if native_language == english_language:
+            return native_language
+        return native_language + ' (' + english_language + ')'
+
+    result = native_language + ', ' + native_territory + ' (' + english_language
+    if english_territory:
+        return result + ', ' + english_territory + ')'
+    return result + ')'

+
+
+def _get_locale_descr(locale, language_code):
+    language_name = locale.get_language_name(language_code).capitalize()
+    if language_name and ('a' <= language_name[0] <= 'z'):
+        language_name = language_name.capitalize()
+    terrirtory_name = locale.get_territory_name(language_code)
+    return language_name, terrirtory_name
+
+
+
[docs]def locales_initialize(directory=None):
+    """Initialize locales environment of the SearXNG session.
+
+    - monkey patch :py:obj:`flask_babel.get_translations` by :py:obj:`get_translations`
+    - init global names :py:obj:`LOCALE_NAMES`, :py:obj:`RTL_LOCALES`
+    """
+
+    directory = directory or pathlib.Path(__file__).parent / 'translations'
+    logger.debug("locales_initialize: %s", directory)
+    flask_babel.get_translations = get_translations
+
+    for tag, descr in ADDITIONAL_TRANSLATIONS.items():
+        locale = Locale.parse(LOCALE_BEST_MATCH[tag], sep='-')
+        LOCALE_NAMES[tag] = descr
+        if locale.text_direction == 'rtl':
+            RTL_LOCALES.add(tag)
+
+    for tag in LOCALE_BEST_MATCH:
+        descr = LOCALE_NAMES.get(tag)
+        if not descr:
+            locale = Locale.parse(tag, sep='-')
+            LOCALE_NAMES[tag] = get_locale_descr(locale, tag.replace('-', '_'))
+            if locale.text_direction == 'rtl':
+                RTL_LOCALES.add(tag)
+
+    for dirname in sorted(os.listdir(directory)):
+        # Based on https://flask-babel.tkte.ch/_modules/flask_babel.html#Babel.list_translations
+        if not os.path.isdir(os.path.join(directory, dirname, 'LC_MESSAGES')):
+            continue
+        tag = dirname.replace('_', '-')
+        descr = LOCALE_NAMES.get(tag)
+        if not descr:
+            locale = Locale.parse(dirname)
+            LOCALE_NAMES[tag] = get_locale_descr(locale, dirname)
+            if locale.text_direction == 'rtl':
+                RTL_LOCALES.add(tag)

+
+
+
[docs]def get_engine_locale(searxng_locale, engine_locales, default=None):
+    """Return engine's language (aka locale) string that best fits to argument
+    ``searxng_locale``.
+
+    Argument ``engine_locales`` is a python dict that maps *SearXNG locales* to
+    corresponding *engine locales*::
+
+      <engine>: {
+          # SearXNG string : engine-string
+          'ca-ES'          : 'ca_ES',
+          'fr-BE'          : 'fr_BE',
+          'fr-CA'          : 'fr_CA',
+          'fr-CH'          : 'fr_CH',
+          'fr'             : 'fr_FR',
+          ...
+          'pl-PL'          : 'pl_PL',
+          'pt-PT'          : 'pt_PT'
+      }
+
+    .. hint::
+
+       The *SearXNG locale* string has to be known by babel!
+
+    If there is no direct 1:1 mapping, this functions tries to narrow down
+    engine's language (locale).  If no value can be determined by these
+    approximation attempts the ``default`` value is returned.
+
+    Assumptions:
+
+    A. When user select a language the results should be optimized according to
+       the selected language.
+
+    B. When user select a language and a territory the results should be
+       optimized with first priority on terrirtory and second on language.
+
+    First approximation rule (*by territory*):
+
+      When the user selects a locale with terrirtory (and a language), the
+      territory has priority over the language.  If any of the offical languages
+      in the terrirtory is supported by the engine (``engine_locales``) it will
+      be used.
+
+    Second approximation rule (*by language*):
+
+      If "First approximation rule" brings no result or the user selects only a
+      language without a terrirtory.  Check in which territories the language
+      has an offical status and if one of these territories is supported by the
+      engine.
+
+    """
+    # pylint: disable=too-many-branches
+
+    engine_locale = engine_locales.get(searxng_locale)
+
+    if engine_locale is not None:
+        # There was a 1:1 mapping (e.g. "fr-BE --> fr_BE" or "fr --> fr_FR"), no
+        # need to narrow language nor territory.
+        return engine_locale
+
+    try:
+        locale = babel.Locale.parse(searxng_locale, sep='-')
+    except babel.core.UnknownLocaleError:
+        try:
+            locale = babel.Locale.parse(searxng_locale.split('-')[0])
+        except babel.core.UnknownLocaleError:
+            return default
+
+    # SearXNG's selected locale is not supported by the engine ..
+
+    if locale.territory:
+        # Try to narrow by *offical* languages in the territory (??-XX).
+
+        for official_language in babel.languages.get_official_languages(locale.territory, de_facto=True):
+            searxng_locale = official_language + '-' + locale.territory
+            engine_locale = engine_locales.get(searxng_locale)
+            if engine_locale is not None:
+                return engine_locale
+
+    # Engine does not support one of the offical languages in the territory or
+    # there is only a language selected without a territory.
+
+    # Now lets have a look if the searxng_lang (the language selected by the
+    # user) is a offical language in other territories.  If so, check if
+    # engine does support the searxng_lang in this other territory.
+
+    if locale.language:
+
+        searxng_lang = locale.language
+        if locale.script:
+            searxng_lang += '_' + locale.script
+
+        terr_lang_dict = {}
+        for territory, langs in babel.core.get_global("territory_languages").items():
+            if not langs.get(searxng_lang, {}).get('official_status'):
+                continue
+            terr_lang_dict[territory] = langs.get(searxng_lang)
+
+        # first: check fr-FR, de-DE .. is supported by the engine
+        # exception: 'en' --> 'en-US'
+
+        territory = locale.language.upper()
+        if territory == 'EN':
+            territory = 'US'
+
+        if terr_lang_dict.get(territory):
+            searxng_locale = locale.language + '-' + territory
+            engine_locale = engine_locales.get(searxng_locale)
+            if engine_locale is not None:
+                return engine_locale
+
+        # second: sort by population_percent and take first match
+
+        # drawback of "population percent": if there is a terrirtory with a
+        #   small number of people (e.g 100) but the majority speaks the
+        #   language, then the percentage migth be 100% (--> 100 people) but in
+        #   a different terrirtory with more people (e.g. 10.000) where only 10%
+        #   speak the language the total amount of speaker is higher (--> 200
+        #   people).
+        #
+        #   By example: The population of Saint-Martin is 33.000, of which 100%
+        #   speak French, but this is less than the 30% of the approximately 2.5
+        #   million Belgian citizens
+        #
+        #   - 'fr-MF', 'population_percent': 100.0, 'official_status': 'official'
+        #   - 'fr-BE', 'population_percent': 38.0, 'official_status': 'official'
+
+        terr_lang_list = []
+        for k, v in terr_lang_dict.items():
+            terr_lang_list.append((k, v))
+
+        for territory, _lang in sorted(terr_lang_list, key=lambda item: item[1]['population_percent'], reverse=True):
+            searxng_locale = locale.language + '-' + territory
+            engine_locale = engine_locales.get(searxng_locale)
+            if engine_locale is not None:
+                return engine_locale
+
+    # No luck: narrow by "language from territory" and "territory from language"
+    # does not fit to a locale supported by the engine.
+
+    if engine_locale is None:
+        engine_locale = default
+
+    return default

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/redislib.html b/_modules/searx/redislib.html new file mode 100644 index 00000000..b77726b5 --- /dev/null +++ b/_modules/searx/redislib.html @@ -0,0 +1,356 @@ + + + + + + + + + searx.redislib — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.redislib

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+"""A collection of convenient functions and redis/lua scripts.
+
+This code was partial inspired by the `Bullet-Proofing Lua Scripts in RedisPy`_
+article.
+
+.. _Bullet-Proofing Lua Scripts in RedisPy:
+   https://redis.com/blog/bullet-proofing-lua-scripts-in-redispy/
+
+"""
+
+import hmac
+
+from searx import get_setting
+
+LUA_SCRIPT_STORAGE = {}
+"""A global dictionary to cache client's ``Script`` objects, used by
+:py:obj:`lua_script_storage`"""
+
+
+
[docs]def lua_script_storage(client, script):
+    """Returns a redis :py:obj:`Script
+    <redis.commands.core.CoreCommands.register_script>` instance.
+
+    Due to performance reason the ``Script`` object is instantiated only once
+    for a client (``client.register_script(..)``) and is cached in
+    :py:obj:`LUA_SCRIPT_STORAGE`.
+
+    """
+
+    # redis connection can be closed, lets use the id() of the redis connector
+    # as key in the script-storage:
+    client_id = id(client)
+
+    if LUA_SCRIPT_STORAGE.get(client_id) is None:
+        LUA_SCRIPT_STORAGE[client_id] = {}
+
+    if LUA_SCRIPT_STORAGE[client_id].get(script) is None:
+        LUA_SCRIPT_STORAGE[client_id][script] = client.register_script(script)
+
+    return LUA_SCRIPT_STORAGE[client_id][script]

+
+
+PURGE_BY_PREFIX = """
+local prefix = tostring(ARGV[1])
+for i, name in ipairs(redis.call('KEYS', prefix .. '*')) do
+    redis.call('EXPIRE', name, 0)
+end
+"""
+
+
+
[docs]def purge_by_prefix(client, prefix: str = "SearXNG_"):
+    """Purge all keys with ``prefix`` from database.
+
+    Queries all keys in the database by the given prefix and set expire time to
+    zero.  The default prefix will drop all keys which has been set by SearXNG
+    (drops SearXNG schema entirely from database).
+
+    The implementation is the lua script from string :py:obj:`PURGE_BY_PREFIX`.
+    The lua script uses EXPIRE_ instead of DEL_: if there are a lot keys to
+    delete and/or their values are big, `DEL` could take more time and blocks
+    the command loop while `EXPIRE` turns back immediate.
+
+    :param prefix: prefix of the key to delete (default: ``SearXNG_``)
+    :type name: str
+
+    .. _EXPIRE: https://redis.io/commands/expire/
+    .. _DEL: https://redis.io/commands/del/
+
+    """
+    script = lua_script_storage(client, PURGE_BY_PREFIX)
+    script(args=[prefix])

+
+
+
[docs]def secret_hash(name: str):
+    """Creates a hash of the ``name``.
+
+    Combines argument ``name`` with the ``secret_key`` from :ref:`settings
+    server`.  This function can be used to get a more anonymised name of a Redis
+    KEY.
+
+    :param name: the name to create a secret hash for
+    :type name: str
+    """
+    m = hmac.new(bytes(name, encoding='utf-8'), digestmod='sha256')
+    m.update(bytes(get_setting('server.secret_key'), encoding='utf-8'))
+    return m.hexdigest()

+
+
+INCR_COUNTER = """
+local limit = tonumber(ARGV[1])
+local expire = tonumber(ARGV[2])
+local c_name = KEYS[1]
+
+local c = redis.call('GET', c_name)
+
+if not c then
+    c = redis.call('INCR', c_name)
+    if expire > 0 then
+        redis.call('EXPIRE', c_name, expire)
+    end
+else
+    c = tonumber(c)
+    if limit == 0 or c < limit then
+       c = redis.call('INCR', c_name)
+    end
+end
+return c
+"""
+
+
+
[docs]def incr_counter(client, name: str, limit: int = 0, expire: int = 0):
+    """Increment a counter and return the new value.
+
+    If counter with redis key ``SearXNG_counter_<name>`` does not exists it is
+    created with initial value 1 returned.  The replacement ``<name>`` is a
+    *secret hash* of the value from argument ``name`` (see
+    :py:func:`secret_hash`).
+
+    The implementation of the redis counter is the lua script from string
+    :py:obj:`INCR_COUNTER`.
+
+    :param name: name of the counter
+    :type name: str
+
+    :param expire: live-time of the counter in seconds (default ``None`` means
+      infinite).
+    :type expire: int / see EXPIRE_
+
+    :param limit: limit where the counter stops to increment (default ``None``)
+    :type limit: int / limit is 2^64 see INCR_
+
+    :return: value of the incremented counter
+    :type return: int
+
+    .. _EXPIRE: https://redis.io/commands/expire/
+    .. _INCR: https://redis.io/commands/incr/
+
+    A simple demo of a counter with expire time and limit::
+
+      >>> for i in range(6):
+      ...   i, incr_counter(client, "foo", 3, 5) # max 3, duration 5 sec
+      ...   time.sleep(1) # from the third call on max has been reached
+      ...
+      (0, 1)
+      (1, 2)
+      (2, 3)
+      (3, 3)
+      (4, 3)
+      (5, 1)
+
+    """
+    script = lua_script_storage(client, INCR_COUNTER)
+    name = "SearXNG_counter_" + secret_hash(name)
+    c = script(args=[limit, expire], keys=[name])
+    return c

+
+
+
[docs]def drop_counter(client, name):
+    """Drop counter with redis key ``SearXNG_counter_<name>``
+
+    The replacement ``<name>`` is a *secret hash* of the value from argument
+    ``name`` (see :py:func:`incr_counter` and :py:func:`incr_sliding_window`).
+    """
+    name = "SearXNG_counter_" + secret_hash(name)
+    client.delete(name)

+
+
+INCR_SLIDING_WINDOW = """
+local expire = tonumber(ARGV[1])
+local name = KEYS[1]
+local current_time = redis.call('TIME')
+
+redis.call('ZREMRANGEBYSCORE', name, 0, current_time[1] - expire)
+redis.call('ZADD', name, current_time[1], current_time[1] .. current_time[2])
+local result = redis.call('ZCOUNT', name, 0, current_time[1] + 1)
+redis.call('EXPIRE', name, expire)
+return result
+"""
+
+
+
[docs]def incr_sliding_window(client, name: str, duration: int):
+    """Increment a sliding-window counter and return the new value.
+
+    If counter with redis key ``SearXNG_counter_<name>`` does not exists it is
+    created with initial value 1 returned.  The replacement ``<name>`` is a
+    *secret hash* of the value from argument ``name`` (see
+    :py:func:`secret_hash`).
+
+    :param name: name of the counter
+    :type name: str
+
+    :param duration: live-time of the sliding window in seconds
+    :typeduration: int
+
+    :return: value of the incremented counter
+    :type return: int
+
+    The implementation of the redis counter is the lua script from string
+    :py:obj:`INCR_SLIDING_WINDOW`.  The lua script uses `sorted sets in Redis`_
+    to implement a sliding window for the redis key ``SearXNG_counter_<name>``
+    (ZADD_).  The current TIME_ is used to score the items in the sorted set and
+    the time window is moved by removing items with a score lower current time
+    minus *duration* time (ZREMRANGEBYSCORE_).
+
+    The EXPIRE_ time (the duration of the sliding window) is refreshed on each
+    call (incrementation) and if there is no call in this duration, the sorted
+    set expires from the redis DB.
+
+    The return value is the amount of items in the sorted set (ZCOUNT_), what
+    means the number of calls in the sliding window.
+
+    .. _Sorted sets in Redis:
+       https://redis.com/ebook/part-1-getting-started/chapter-1-getting-to-know-redis/1-2-what-redis-data-structures-look-like/1-2-5-sorted-sets-in-redis/
+    .. _TIME: https://redis.io/commands/time/
+    .. _ZADD: https://redis.io/commands/zadd/
+    .. _EXPIRE: https://redis.io/commands/expire/
+    .. _ZREMRANGEBYSCORE: https://redis.io/commands/zremrangebyscore/
+    .. _ZCOUNT: https://redis.io/commands/zcount/
+
+    A simple demo of the sliding window::
+
+      >>> for i in range(5):
+      ...   incr_sliding_window(client, "foo", 3) # duration 3 sec
+      ...   time.sleep(1) # from the third call (second) on the window is moved
+      ...
+      1
+      2
+      3
+      3
+      3
+      >>> time.sleep(3)  # wait until expire
+      >>> incr_sliding_window(client, "foo", 3)
+      1
+
+    """
+    script = lua_script_storage(client, INCR_SLIDING_WINDOW)
+    name = "SearXNG_counter_" + secret_hash(name)
+    c = script(args=[duration], keys=[name])
+    return c

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/search.html b/_modules/searx/search.html new file mode 100644 index 00000000..957a9cf4 --- /dev/null +++ b/_modules/searx/search.html @@ -0,0 +1,324 @@ + + + + + + + + + searx.search — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.search

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+# pylint: disable=missing-module-docstring, too-few-public-methods
+
+import threading
+from timeit import default_timer
+from uuid import uuid4
+
+import flask
+
+from searx import settings
+from searx.answerers import ask
+from searx.external_bang import get_bang_url
+from searx.results import ResultContainer
+from searx import logger
+from searx.plugins import plugins
+from searx.search.models import EngineRef, SearchQuery
+from searx.engines import load_engines
+from searx.network import initialize as initialize_network, check_network_configuration
+from searx.metrics import initialize as initialize_metrics, counter_inc, histogram_observe_time
+from searx.search.processors import PROCESSORS, initialize as initialize_processors
+from searx.search.checker import initialize as initialize_checker
+
+
+logger = logger.getChild('search')
+
+
+def initialize(settings_engines=None, enable_checker=False, check_network=False, enable_metrics=True):
+    settings_engines = settings_engines or settings['engines']
+    load_engines(settings_engines)
+    initialize_network(settings_engines, settings['outgoing'])
+    if check_network:
+        check_network_configuration()
+    initialize_metrics([engine['name'] for engine in settings_engines], enable_metrics)
+    initialize_processors(settings_engines)
+    if enable_checker:
+        initialize_checker()
+
+
+
+
+
+
[docs]class SearchWithPlugins(Search):
+    """Inherit from the Search class, add calls to the plugins."""
+
+    __slots__ = 'ordered_plugin_list', 'request'
+
+    def __init__(self, search_query: SearchQuery, ordered_plugin_list, request: flask.Request):
+        super().__init__(search_query)
+        self.ordered_plugin_list = ordered_plugin_list
+        self.result_container.on_result = self._on_result
+        # pylint: disable=line-too-long
+        # get the "real" request to use it outside the Flask context.
+        # see
+        # * https://github.com/pallets/flask/blob/d01d26e5210e3ee4cbbdef12f05c886e08e92852/src/flask/globals.py#L55
+        # * https://github.com/pallets/werkzeug/blob/3c5d3c9bd0d9ce64590f0af8997a38f3823b368d/src/werkzeug/local.py#L548-L559
+        # * https://werkzeug.palletsprojects.com/en/2.0.x/local/#werkzeug.local.LocalProxy._get_current_object
+        # pylint: enable=line-too-long
+        self.request = request._get_current_object()
+
+    def _on_result(self, result):
+        return plugins.call(self.ordered_plugin_list, 'on_result', self.request, self, result)
+
+
[docs]    def search(self) -> ResultContainer:
+        if plugins.call(self.ordered_plugin_list, 'pre_search', self.request, self):
+            super().search()
+
+        plugins.call(self.ordered_plugin_list, 'post_search', self.request, self)
+
+        self.result_container.close()
+
+        return self.result_container

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/search/models.html b/_modules/searx/search/models.html new file mode 100644 index 00000000..24e7e747 --- /dev/null +++ b/_modules/searx/search/models.html @@ -0,0 +1,230 @@ + + + + + + + + + searx.search.models — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.search.models

+# SPDX-License-Identifier: AGPL-3.0-or-later
+
+import typing
+import babel
+
+
+
[docs]class EngineRef:
+    """Reference by names to an engine and category"""
+
+    __slots__ = 'name', 'category'
+
+    def __init__(self, name: str, category: str):
+        self.name = name
+        self.category = category
+
+    def __repr__(self):
+        return "EngineRef({!r}, {!r})".format(self.name, self.category)
+
+    def __eq__(self, other):
+        return self.name == other.name and self.category == other.category
+
+    def __hash__(self):
+        return hash((self.name, self.category))

+
+
+
[docs]class SearchQuery:
+    """container for all the search parameters (query, language, etc...)"""
+
+    __slots__ = (
+        'query',
+        'engineref_list',
+        'lang',
+        'locale',
+        'safesearch',
+        'pageno',
+        'time_range',
+        'timeout_limit',
+        'external_bang',
+        'engine_data',
+    )
+
+    def __init__(
+        self,
+        query: str,
+        engineref_list: typing.List[EngineRef],
+        lang: str = 'all',
+        safesearch: int = 0,
+        pageno: int = 1,
+        time_range: typing.Optional[str] = None,
+        timeout_limit: typing.Optional[float] = None,
+        external_bang: typing.Optional[str] = None,
+        engine_data: typing.Optional[typing.Dict[str, str]] = None,
+    ):
+        self.query = query
+        self.engineref_list = engineref_list
+        self.lang = lang
+        self.safesearch = safesearch
+        self.pageno = pageno
+        self.time_range = time_range
+        self.timeout_limit = timeout_limit
+        self.external_bang = external_bang
+        self.engine_data = engine_data or {}
+
+        self.locale = None
+        if self.lang:
+            try:
+                self.locale = babel.Locale.parse(self.lang, sep='-')
+            except babel.core.UnknownLocaleError:
+                pass
+
+    @property
+    def categories(self):
+        return list(set(map(lambda engineref: engineref.category, self.engineref_list)))
+
+    def __repr__(self):
+        return "SearchQuery({!r}, {!r}, {!r}, {!r}, {!r}, {!r}, {!r}, {!r})".format(
+            self.query,
+            self.engineref_list,
+            self.lang,
+            self.safesearch,
+            self.pageno,
+            self.time_range,
+            self.timeout_limit,
+            self.external_bang,
+        )
+
+    def __eq__(self, other):
+        return (
+            self.query == other.query
+            and self.engineref_list == other.engineref_list
+            and self.lang == other.lang
+            and self.safesearch == other.safesearch
+            and self.pageno == other.pageno
+            and self.time_range == other.time_range
+            and self.timeout_limit == other.timeout_limit
+            and self.external_bang == other.external_bang
+        )
+
+    def __hash__(self):
+        return hash(
+            (
+                self.query,
+                tuple(self.engineref_list),
+                self.lang,
+                self.safesearch,
+                self.pageno,
+                self.time_range,
+                self.timeout_limit,
+                self.external_bang,
+            )
+        )

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searx/utils.html b/_modules/searx/utils.html new file mode 100644 index 00000000..2c733b28 --- /dev/null +++ b/_modules/searx/utils.html @@ -0,0 +1,762 @@ + + + + + + + + + searx.utils — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searx.utils

+# SPDX-License-Identifier: AGPL-3.0-or-later
+# lint: pylint
+# pyright: basic
+"""Utility functions for the engines
+
+"""
+import re
+import importlib
+import importlib.util
+import types
+
+from typing import Optional, Union, Any, Set, List, Dict, MutableMapping, Tuple, Callable
+from numbers import Number
+from os.path import splitext, join
+from random import choice
+from html.parser import HTMLParser
+from urllib.parse import urljoin, urlparse
+
+from lxml import html
+from lxml.etree import ElementBase, XPath, XPathError, XPathSyntaxError, _ElementStringResult, _ElementUnicodeResult
+from babel.core import get_global
+
+
+from searx import settings
+from searx.data import USER_AGENTS, data_dir
+from searx.version import VERSION_TAG
+from searx.languages import language_codes
+from searx.exceptions import SearxXPathSyntaxException, SearxEngineXPathException
+from searx import logger
+
+
+logger = logger.getChild('utils')
+
+XPathSpecType = Union[str, XPath]
+
+_BLOCKED_TAGS = ('script', 'style')
+
+_ECMA_UNESCAPE4_RE = re.compile(r'%u([0-9a-fA-F]{4})', re.UNICODE)
+_ECMA_UNESCAPE2_RE = re.compile(r'%([0-9a-fA-F]{2})', re.UNICODE)
+
+_STORAGE_UNIT_VALUE: Dict[str, int] = {
+    'TB': 1024 * 1024 * 1024 * 1024,
+    'GB': 1024 * 1024 * 1024,
+    'MB': 1024 * 1024,
+    'TiB': 1000 * 1000 * 1000 * 1000,
+    'MiB': 1000 * 1000,
+    'KiB': 1000,
+}
+
+_XPATH_CACHE: Dict[str, XPath] = {}
+_LANG_TO_LC_CACHE: Dict[str, Dict[str, str]] = {}
+
+_FASTTEXT_MODEL: Optional["fasttext.FastText._FastText"] = None
+"""fasttext model to predict laguage of a search term"""
+
+
+class _NotSetClass:  # pylint: disable=too-few-public-methods
+    """Internal class for this module, do not create instance of this class.
+    Replace the None value, allow explicitly pass None as a function argument"""
+
+
+_NOTSET = _NotSetClass()
+
+
+
[docs]def searx_useragent() -> str:
+    """Return the searx User Agent"""
+    return 'searx/{searx_version} {suffix}'.format(
+        searx_version=VERSION_TAG, suffix=settings['outgoing']['useragent_suffix']
+    ).strip()

+
+
+
[docs]def gen_useragent(os_string: Optional[str] = None) -> str:
+    """Return a random browser User Agent
+
+    See searx/data/useragents.json
+    """
+    return USER_AGENTS['ua'].format(os=os_string or choice(USER_AGENTS['os']), version=choice(USER_AGENTS['versions']))

+
+
+class _HTMLTextExtractorException(Exception):
+    """Internal exception raised when the HTML is invalid"""
+
+
+class _HTMLTextExtractor(HTMLParser):  # pylint: disable=W0223  # (see https://bugs.python.org/issue31844)
+    """Internal class to extract text from HTML"""
+
+    def __init__(self):
+        HTMLParser.__init__(self)
+        self.result = []
+        self.tags = []
+
+    def handle_starttag(self, tag, attrs):
+        self.tags.append(tag)
+        if tag == 'br':
+            self.result.append(' ')
+
+    def handle_endtag(self, tag):
+        if not self.tags:
+            return
+
+        if tag != self.tags[-1]:
+            raise _HTMLTextExtractorException()
+
+        self.tags.pop()
+
+    def is_valid_tag(self):
+        return not self.tags or self.tags[-1] not in _BLOCKED_TAGS
+
+    def handle_data(self, data):
+        if not self.is_valid_tag():
+            return
+        self.result.append(data)
+
+    def handle_charref(self, name):
+        if not self.is_valid_tag():
+            return
+        if name[0] in ('x', 'X'):
+            codepoint = int(name[1:], 16)
+        else:
+            codepoint = int(name)
+        self.result.append(chr(codepoint))
+
+    def handle_entityref(self, name):
+        if not self.is_valid_tag():
+            return
+        # codepoint = htmlentitydefs.name2codepoint[name]
+        # self.result.append(chr(codepoint))
+        self.result.append(name)
+
+    def get_text(self):
+        return ''.join(self.result).strip()
+
+
+
[docs]def html_to_text(html_str: str) -> str:
+    """Extract text from a HTML string
+
+    Args:
+        * html_str (str): string HTML
+
+    Returns:
+        * str: extracted text
+
+    Examples:
+        >>> html_to_text('Example <span id="42">#2</span>')
+        'Example #2'
+
+        >>> html_to_text('<style>.span { color: red; }</style><span>Example</span>')
+        'Example'
+    """
+    html_str = html_str.replace('\n', ' ').replace('\r', ' ')
+    html_str = ' '.join(html_str.split())
+    s = _HTMLTextExtractor()
+    try:
+        s.feed(html_str)
+    except _HTMLTextExtractorException:
+        logger.debug("HTMLTextExtractor: invalid HTML\n%s", html_str)
+    return s.get_text()

+
+
+
[docs]def extract_text(xpath_results, allow_none: bool = False) -> Optional[str]:
+    """Extract text from a lxml result
+
+    * if xpath_results is list, extract the text from each result and concat the list
+    * if xpath_results is a xml element, extract all the text node from it
+      ( text_content() method from lxml )
+    * if xpath_results is a string element, then it's already done
+    """
+    if isinstance(xpath_results, list):
+        # it's list of result : concat everything using recursive call
+        result = ''
+        for e in xpath_results:
+            result = result + (extract_text(e) or '')
+        return result.strip()
+    if isinstance(xpath_results, ElementBase):
+        # it's a element
+        text: str = html.tostring(xpath_results, encoding='unicode', method='text', with_tail=False)
+        text = text.strip().replace('\n', ' ')
+        return ' '.join(text.split())
+    if isinstance(xpath_results, (_ElementStringResult, _ElementUnicodeResult, str, Number, bool)):
+        return str(xpath_results)
+    if xpath_results is None and allow_none:
+        return None
+    if xpath_results is None and not allow_none:
+        raise ValueError('extract_text(None, allow_none=False)')
+    raise ValueError('unsupported type')

+
+
+
[docs]def normalize_url(url: str, base_url: str) -> str:
+    """Normalize URL: add protocol, join URL with base_url, add trailing slash if there is no path
+
+    Args:
+        * url (str): Relative URL
+        * base_url (str): Base URL, it must be an absolute URL.
+
+    Example:
+        >>> normalize_url('https://example.com', 'http://example.com/')
+        'https://example.com/'
+        >>> normalize_url('//example.com', 'http://example.com/')
+        'http://example.com/'
+        >>> normalize_url('//example.com', 'https://example.com/')
+        'https://example.com/'
+        >>> normalize_url('/path?a=1', 'https://example.com')
+        'https://example.com/path?a=1'
+        >>> normalize_url('', 'https://example.com')
+        'https://example.com/'
+        >>> normalize_url('/test', '/path')
+        raise ValueError
+
+    Raises:
+        * lxml.etree.ParserError
+
+    Returns:
+        * str: normalized URL
+    """
+    if url.startswith('//'):
+        # add http or https to this kind of url //example.com/
+        parsed_search_url = urlparse(base_url)
+        url = '{0}:{1}'.format(parsed_search_url.scheme or 'http', url)
+    elif url.startswith('/'):
+        # fix relative url to the search engine
+        url = urljoin(base_url, url)
+
+    # fix relative urls that fall through the crack
+    if '://' not in url:
+        url = urljoin(base_url, url)
+
+    parsed_url = urlparse(url)
+
+    # add a / at this end of the url if there is no path
+    if not parsed_url.netloc:
+        raise ValueError('Cannot parse url')
+    if not parsed_url.path:
+        url += '/'
+
+    return url

+
+
+
[docs]def extract_url(xpath_results, base_url) -> str:
+    """Extract and normalize URL from lxml Element
+
+    Args:
+        * xpath_results (Union[List[html.HtmlElement], html.HtmlElement]): lxml Element(s)
+        * base_url (str): Base URL
+
+    Example:
+        >>> def f(s, search_url):
+        >>>    return searx.utils.extract_url(html.fromstring(s), search_url)
+        >>> f('<span id="42">https://example.com</span>', 'http://example.com/')
+        'https://example.com/'
+        >>> f('https://example.com', 'http://example.com/')
+        'https://example.com/'
+        >>> f('//example.com', 'http://example.com/')
+        'http://example.com/'
+        >>> f('//example.com', 'https://example.com/')
+        'https://example.com/'
+        >>> f('/path?a=1', 'https://example.com')
+        'https://example.com/path?a=1'
+        >>> f('', 'https://example.com')
+        raise lxml.etree.ParserError
+        >>> searx.utils.extract_url([], 'https://example.com')
+        raise ValueError
+
+    Raises:
+        * ValueError
+        * lxml.etree.ParserError
+
+    Returns:
+        * str: normalized URL
+    """
+    if xpath_results == []:
+        raise ValueError('Empty url resultset')
+
+    url = extract_text(xpath_results)
+    if url:
+        return normalize_url(url, base_url)
+    raise ValueError('URL not found')

+
+
+
[docs]def dict_subset(dictionary: MutableMapping, properties: Set[str]) -> Dict:
+    """Extract a subset of a dict
+
+    Examples:
+        >>> dict_subset({'A': 'a', 'B': 'b', 'C': 'c'}, ['A', 'C'])
+        {'A': 'a', 'C': 'c'}
+        >>> >> dict_subset({'A': 'a', 'B': 'b', 'C': 'c'}, ['A', 'D'])
+        {'A': 'a'}
+    """
+    return {k: dictionary[k] for k in properties if k in dictionary}

+
+
+
[docs]def get_torrent_size(filesize: str, filesize_multiplier: str) -> Optional[int]:
+    """
+
+    Args:
+        * filesize (str): size
+        * filesize_multiplier (str): TB, GB, .... TiB, GiB...
+
+    Returns:
+        * int: number of bytes
+
+    Example:
+        >>> get_torrent_size('5', 'GB')
+        5368709120
+        >>> get_torrent_size('3.14', 'MiB')
+        3140000
+    """
+    try:
+        multiplier = _STORAGE_UNIT_VALUE.get(filesize_multiplier, 1)
+        return int(float(filesize) * multiplier)
+    except ValueError:
+        return None

+
+
+
[docs]def convert_str_to_int(number_str: str) -> int:
+    """Convert number_str to int or 0 if number_str is not a number."""
+    if number_str.isdigit():
+        return int(number_str)
+    return 0

+
+
+
[docs]def int_or_zero(num: Union[List[str], str]) -> int:
+    """Convert num to int or 0. num can be either a str or a list.
+    If num is a list, the first element is converted to int (or return 0 if the list is empty).
+    If num is a str, see convert_str_to_int
+    """
+    if isinstance(num, list):
+        if len(num) < 1:
+            return 0
+        num = num[0]
+    return convert_str_to_int(num)

+
+
+
[docs]def is_valid_lang(lang) -> Optional[Tuple[bool, str, str]]:
+    """Return language code and name if lang describe a language.
+
+    Examples:
+        >>> is_valid_lang('zz')
+        None
+        >>> is_valid_lang('uk')
+        (True, 'uk', 'ukrainian')
+        >>> is_valid_lang(b'uk')
+        (True, 'uk', 'ukrainian')
+        >>> is_valid_lang('en')
+        (True, 'en', 'english')
+        >>> searx.utils.is_valid_lang('Español')
+        (True, 'es', 'spanish')
+        >>> searx.utils.is_valid_lang('Spanish')
+        (True, 'es', 'spanish')
+    """
+    if isinstance(lang, bytes):
+        lang = lang.decode()
+    is_abbr = len(lang) == 2
+    lang = lang.lower()
+    if is_abbr:
+        for l in language_codes:
+            if l[0][:2] == lang:
+                return (True, l[0][:2], l[3].lower())
+        return None
+    for l in language_codes:
+        if l[1].lower() == lang or l[3].lower() == lang:
+            return (True, l[0][:2], l[3].lower())
+    return None

+
+
+def _get_lang_to_lc_dict(lang_list: List[str]) -> Dict[str, str]:
+    key = str(lang_list)
+    value = _LANG_TO_LC_CACHE.get(key, None)
+    if value is None:
+        value = {}
+        for lang in lang_list:
+            value.setdefault(lang.split('-')[0], lang)
+        _LANG_TO_LC_CACHE[key] = value
+    return value
+
+
+# babel's get_global contains all sorts of miscellaneous locale and territory related data
+# see get_global in: https://github.com/python-babel/babel/blob/master/babel/core.py
+def _get_from_babel(lang_code: str, key):
+    match = get_global(key).get(lang_code.replace('-', '_'))
+    # for some keys, such as territory_aliases, match may be a list
+    if isinstance(match, str):
+        return match.replace('_', '-')
+    return match
+
+
+def _match_language(lang_code: str, lang_list=[], custom_aliases={}) -> Optional[str]:  # pylint: disable=W0102
+    """auxiliary function to match lang_code in lang_list"""
+    # replace language code with a custom alias if necessary
+    if lang_code in custom_aliases:
+        lang_code = custom_aliases[lang_code]
+
+    if lang_code in lang_list:
+        return lang_code
+
+    # try to get the most likely country for this language
+    subtags = _get_from_babel(lang_code, 'likely_subtags')
+    if subtags:
+        if subtags in lang_list:
+            return subtags
+        subtag_parts = subtags.split('-')
+        new_code = subtag_parts[0] + '-' + subtag_parts[-1]
+        if new_code in custom_aliases:
+            new_code = custom_aliases[new_code]
+        if new_code in lang_list:
+            return new_code
+
+    # try to get the any supported country for this language
+    return _get_lang_to_lc_dict(lang_list).get(lang_code)
+
+
+
[docs]def match_language(  # pylint: disable=W0102
+    locale_code, lang_list=[], custom_aliases={}, fallback: Optional[str] = 'en-US'
+) -> Optional[str]:
+    """get the language code from lang_list that best matches locale_code"""
+    # try to get language from given locale_code
+    language = _match_language(locale_code, lang_list, custom_aliases)
+    if language:
+        return language
+
+    locale_parts = locale_code.split('-')
+    lang_code = locale_parts[0]
+
+    # if locale_code has script, try matching without it
+    if len(locale_parts) > 2:
+        language = _match_language(lang_code + '-' + locale_parts[-1], lang_list, custom_aliases)
+        if language:
+            return language
+
+    # try to get language using an equivalent country code
+    if len(locale_parts) > 1:
+        country_alias = _get_from_babel(locale_parts[-1], 'territory_aliases')
+        if country_alias:
+            language = _match_language(lang_code + '-' + country_alias[0], lang_list, custom_aliases)
+            if language:
+                return language
+
+    # try to get language using an equivalent language code
+    alias = _get_from_babel(lang_code, 'language_aliases')
+    if alias:
+        language = _match_language(alias, lang_list, custom_aliases)
+        if language:
+            return language
+
+    if lang_code != locale_code:
+        # try to get language from given language without giving the country
+        language = _match_language(lang_code, lang_list, custom_aliases)
+
+    return language or fallback

+
+
+def load_module(filename: str, module_dir: str) -> types.ModuleType:
+    modname = splitext(filename)[0]
+    modpath = join(module_dir, filename)
+    # and https://docs.python.org/3/library/importlib.html#importing-a-source-file-directly
+    spec = importlib.util.spec_from_file_location(modname, modpath)
+    if not spec:
+        raise ValueError(f"Error loading '{modpath}' module")
+    module = importlib.util.module_from_spec(spec)
+    if not spec.loader:
+        raise ValueError(f"Error loading '{modpath}' module")
+    spec.loader.exec_module(module)
+    return module
+
+
+
[docs]def to_string(obj: Any) -> str:
+    """Convert obj to its string representation."""
+    if isinstance(obj, str):
+        return obj
+    if hasattr(obj, '__str__'):
+        return str(obj)
+    return repr(obj)

+
+
+
[docs]def ecma_unescape(string: str) -> str:
+    """Python implementation of the unescape javascript function
+
+    https://www.ecma-international.org/ecma-262/6.0/#sec-unescape-string
+    https://developer.mozilla.org/fr/docs/Web/JavaScript/Reference/Objets_globaux/unescape
+
+    Examples:
+        >>> ecma_unescape('%u5409')
+        '吉'
+        >>> ecma_unescape('%20')
+        ' '
+        >>> ecma_unescape('%F3')
+        'ó'
+    """
+    # "%u5409" becomes "吉"
+    string = _ECMA_UNESCAPE4_RE.sub(lambda e: chr(int(e.group(1), 16)), string)
+    # "%20" becomes " ", "%F3" becomes "ó"
+    string = _ECMA_UNESCAPE2_RE.sub(lambda e: chr(int(e.group(1), 16)), string)
+    return string

+
+
+def get_string_replaces_function(replaces: Dict[str, str]) -> Callable[[str], str]:
+    rep = {re.escape(k): v for k, v in replaces.items()}
+    pattern = re.compile("|".join(rep.keys()))
+
+    def func(text):
+        return pattern.sub(lambda m: rep[re.escape(m.group(0))], text)
+
+    return func
+
+
+
[docs]def get_engine_from_settings(name: str) -> Dict:
+    """Return engine configuration from settings.yml of a given engine name"""
+
+    if 'engines' not in settings:
+        return {}
+
+    for engine in settings['engines']:
+        if 'name' not in engine:
+            continue
+        if name == engine['name']:
+            return engine
+
+    return {}

+
+
+
[docs]def get_xpath(xpath_spec: XPathSpecType) -> XPath:
+    """Return cached compiled XPath
+
+    There is no thread lock.
+    Worst case scenario, xpath_str is compiled more than one time.
+
+    Args:
+        * xpath_spec (str|lxml.etree.XPath): XPath as a str or lxml.etree.XPath
+
+    Returns:
+        * result (bool, float, list, str): Results.
+
+    Raises:
+        * TypeError: Raise when xpath_spec is neither a str nor a lxml.etree.XPath
+        * SearxXPathSyntaxException: Raise when there is a syntax error in the XPath
+    """
+    if isinstance(xpath_spec, str):
+        result = _XPATH_CACHE.get(xpath_spec, None)
+        if result is None:
+            try:
+                result = XPath(xpath_spec)
+            except XPathSyntaxError as e:
+                raise SearxXPathSyntaxException(xpath_spec, str(e.msg)) from e
+            _XPATH_CACHE[xpath_spec] = result
+        return result
+
+    if isinstance(xpath_spec, XPath):
+        return xpath_spec
+
+    raise TypeError('xpath_spec must be either a str or a lxml.etree.XPath')

+
+
+
[docs]def eval_xpath(element: ElementBase, xpath_spec: XPathSpecType):
+    """Equivalent of element.xpath(xpath_str) but compile xpath_str once for all.
+    See https://lxml.de/xpathxslt.html#xpath-return-values
+
+    Args:
+        * element (ElementBase): [description]
+        * xpath_spec (str|lxml.etree.XPath): XPath as a str or lxml.etree.XPath
+
+    Returns:
+        * result (bool, float, list, str): Results.
+
+    Raises:
+        * TypeError: Raise when xpath_spec is neither a str nor a lxml.etree.XPath
+        * SearxXPathSyntaxException: Raise when there is a syntax error in the XPath
+        * SearxEngineXPathException: Raise when the XPath can't be evaluated.
+    """
+    xpath = get_xpath(xpath_spec)
+    try:
+        return xpath(element)
+    except XPathError as e:
+        arg = ' '.join([str(i) for i in e.args])
+        raise SearxEngineXPathException(xpath_spec, arg) from e

+
+
+
[docs]def eval_xpath_list(element: ElementBase, xpath_spec: XPathSpecType, min_len: Optional[int] = None):
+    """Same as eval_xpath, check if the result is a list
+
+    Args:
+        * element (ElementBase): [description]
+        * xpath_spec (str|lxml.etree.XPath): XPath as a str or lxml.etree.XPath
+        * min_len (int, optional): [description]. Defaults to None.
+
+    Raises:
+        * TypeError: Raise when xpath_spec is neither a str nor a lxml.etree.XPath
+        * SearxXPathSyntaxException: Raise when there is a syntax error in the XPath
+        * SearxEngineXPathException: raise if the result is not a list
+
+    Returns:
+        * result (bool, float, list, str): Results.
+    """
+    result = eval_xpath(element, xpath_spec)
+    if not isinstance(result, list):
+        raise SearxEngineXPathException(xpath_spec, 'the result is not a list')
+    if min_len is not None and min_len > len(result):
+        raise SearxEngineXPathException(xpath_spec, 'len(xpath_str) < ' + str(min_len))
+    return result

+
+
+
[docs]def eval_xpath_getindex(elements: ElementBase, xpath_spec: XPathSpecType, index: int, default=_NOTSET):
+    """Call eval_xpath_list then get one element using the index parameter.
+    If the index does not exist, either aise an exception is default is not set,
+    other return the default value (can be None).
+
+    Args:
+        * elements (ElementBase): lxml element to apply the xpath.
+        * xpath_spec (str|lxml.etree.XPath): XPath as a str or lxml.etree.XPath.
+        * index (int): index to get
+        * default (Object, optional): Defaults if index doesn't exist.
+
+    Raises:
+        * TypeError: Raise when xpath_spec is neither a str nor a lxml.etree.XPath
+        * SearxXPathSyntaxException: Raise when there is a syntax error in the XPath
+        * SearxEngineXPathException: if the index is not found. Also see eval_xpath.
+
+    Returns:
+        * result (bool, float, list, str): Results.
+    """
+    result = eval_xpath_list(elements, xpath_spec)
+    if -len(result) <= index < len(result):
+        return result[index]
+    if default == _NOTSET:
+        # raise an SearxEngineXPathException instead of IndexError
+        # to record xpath_spec
+        raise SearxEngineXPathException(xpath_spec, 'index ' + str(index) + ' not found')
+    return default

+
+
+def _get_fasttext_model() -> "fasttext.FastText._FastText":
+    global _FASTTEXT_MODEL  # pylint: disable=global-statement
+    if _FASTTEXT_MODEL is None:
+        import fasttext  # pylint: disable=import-outside-toplevel
+
+        # Monkey patch: prevent fasttext from showing a (useless) warning when loading a model.
+        fasttext.FastText.eprint = lambda x: None
+        _FASTTEXT_MODEL = fasttext.load_model(str(data_dir / 'lid.176.ftz'))
+    return _FASTTEXT_MODEL
+
+
+
[docs]def detect_language(text: str, threshold: float = 0.3, min_probability: float = 0.5) -> Optional[str]:
+    """https://fasttext.cc/docs/en/language-identification.html"""
+    if not isinstance(text, str):
+        raise ValueError('text must a str')
+    r = _get_fasttext_model().predict(text.replace('\n', ' '), k=1, threshold=threshold)
+    if isinstance(r, tuple) and len(r) == 2 and len(r[0]) > 0 and len(r[1]) > 0 and r[1][0] > min_probability:
+        return r[0][0].split('__label__')[1]
+    return None

+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searxng_extra/standalone_searx.html b/_modules/searxng_extra/standalone_searx.html new file mode 100644 index 00000000..5004b600 --- /dev/null +++ b/_modules/searxng_extra/standalone_searx.html @@ -0,0 +1,329 @@ + + + + + + + + + searxng_extra.standalone_searx — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searxng_extra.standalone_searx

+#!/usr/bin/env python
+# lint: pylint
+
+# SPDX-License-Identifier: AGPL-3.0-or-later
+# (C) Copyright Contributors to the SearXNG project.
+# (C) Copyright Contributors to the searx project (2014 - 2021)
+
+"""Script to run SearXNG from terminal.
+
+Getting categories without initiate the engine will only return `['general']`
+
+>>> import searx.engines
+... list(searx.engines.categories.keys())
+['general']
+>>> import searx.search
+... searx.search.initialize()
+... list(searx.engines.categories.keys())
+['general', 'it', 'science', 'images', 'news', 'videos', 'music', 'files', 'social media', 'map']
+
+Example to use this script:
+
+.. code::  bash
+
+    $ python3 searxng_extra/standalone_searx.py rain
+
+.. danger::
+
+   Be warned, using the ``standalone_searx.py`` won't give you privacy!
+
+   On the contrary, this script behaves like a SearXNG server: your IP is
+   exposed and tracked by all active engines (google, bing, qwant, ... ), with
+   every query!
+
+Example to run it from python:
+
+>>> import importlib
+... import json
+... import sys
+... import searx.engines
+... import searx.search
+... search_query = 'rain'
+... # initialize engines
+... searx.search.initialize()
+... # load engines categories once instead of each time the function called
+... engine_cs = list(searx.engines.categories.keys())
+... # load module
+... spec = importlib.util.spec_from_file_location(
+...     'utils.standalone_searx', 'searxng_extra/standalone_searx.py')
+... sas = importlib.util.module_from_spec(spec)
+... spec.loader.exec_module(sas)
+... # use function from module
+... prog_args = sas.parse_argument([search_query], category_choices=engine_cs)
+... search_q = sas.get_search_query(prog_args, engine_categories=engine_cs)
+... res_dict = sas.to_dict(search_q)
+... sys.stdout.write(json.dumps(
+...     res_dict, sort_keys=True, indent=4, ensure_ascii=False,
+...     default=sas.json_serial))
+{
+    "answers": [],
+    "infoboxes": [ {...} ],
+    "paging": true,
+    "results": [... ],
+    "results_number": 820000000.0,
+    "search": {
+        "lang": "all",
+        "pageno": 1,
+        "q": "rain",
+        "safesearch": 0,
+        "timerange": null
+    },
+    "suggestions": [...]
+}
+
+"""  # pylint: disable=line-too-long
+
+import argparse
+import sys
+from datetime import datetime
+from json import dumps
+from typing import Any, Dict, List, Optional
+
+import searx
+import searx.preferences
+import searx.query
+import searx.search
+import searx.webadapter
+
+EngineCategoriesVar = Optional[List[str]]
+
+
+
[docs]def get_search_query(
+    args: argparse.Namespace, engine_categories: EngineCategoriesVar = None
+) -> searx.search.SearchQuery:
+    """Get  search results for the query"""
+    if engine_categories is None:
+        engine_categories = list(searx.engines.categories.keys())
+    try:
+        category = args.category.decode('utf-8')
+    except AttributeError:
+        category = args.category
+    form = {
+        "q": args.query,
+        "categories": category,
+        "pageno": str(args.pageno),
+        "language": args.lang,
+        "time_range": args.timerange,
+    }
+    preferences = searx.preferences.Preferences(['simple'], engine_categories, searx.engines.engines, [])
+    preferences.key_value_settings['safesearch'].parse(args.safesearch)
+
+    search_query = searx.webadapter.get_search_query_from_webapp(preferences, form)[0]
+    return search_query

+
+
+
[docs]def no_parsed_url(results: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+    """Remove parsed url from dict."""
+    for result in results:
+        del result['parsed_url']
+    return results

+
+
+
[docs]def json_serial(obj: Any) -> Any:
+    """JSON serializer for objects not serializable by default json code.
+
+    :raise TypeError: raised when **obj** is not serializable
+    """
+    if isinstance(obj, datetime):
+        serial = obj.isoformat()
+        return serial
+    if isinstance(obj, bytes):
+        return obj.decode('utf8')
+    if isinstance(obj, set):
+        return list(obj)
+    raise TypeError("Type ({}) not serializable".format(type(obj)))

+
+
+
[docs]def to_dict(search_query: searx.search.SearchQuery) -> Dict[str, Any]:
+    """Get result from parsed arguments."""
+    result_container = searx.search.Search(search_query).search()
+    result_container_json = {
+        "search": {
+            "q": search_query.query,
+            "pageno": search_query.pageno,
+            "lang": search_query.lang,
+            "safesearch": search_query.safesearch,
+            "timerange": search_query.time_range,
+        },
+        "results": no_parsed_url(result_container.get_ordered_results()),
+        "infoboxes": result_container.infoboxes,
+        "suggestions": list(result_container.suggestions),
+        "answers": list(result_container.answers),
+        "paging": result_container.paging,
+        "results_number": result_container.results_number(),
+    }
+    return result_container_json

+
+
+
[docs]def parse_argument(
+    args: Optional[List[str]] = None, category_choices: EngineCategoriesVar = None
+) -> argparse.Namespace:
+    """Parse command line.
+
+    :raise SystemExit: Query argument required on `args`
+
+    Examples:
+
+    >>> import importlib
+    ... # load module
+    ... spec = importlib.util.spec_from_file_location(
+    ...     'utils.standalone_searx', 'utils/standalone_searx.py')
+    ... sas = importlib.util.module_from_spec(spec)
+    ... spec.loader.exec_module(sas)
+    ... sas.parse_argument()
+    usage: ptipython [-h] [--category [{general}]] [--lang [LANG]] [--pageno [PAGENO]] [--safesearch [{0,1,2}]] [--timerange [{day,week,month,year}]]
+                     query
+    SystemExit: 2
+    >>> sas.parse_argument(['rain'])
+    Namespace(category='general', lang='all', pageno=1, query='rain', safesearch='0', timerange=None)
+    """  # noqa: E501
+    if not category_choices:
+        category_choices = list(searx.engines.categories.keys())
+    parser = argparse.ArgumentParser(description='Standalone searx.')
+    parser.add_argument('query', type=str, help='Text query')
+    parser.add_argument(
+        '--category', type=str, nargs='?', choices=category_choices, default='general', help='Search category'
+    )
+    parser.add_argument('--lang', type=str, nargs='?', default='all', help='Search language')
+    parser.add_argument('--pageno', type=int, nargs='?', default=1, help='Page number starting from 1')
+    parser.add_argument(
+        '--safesearch',
+        type=str,
+        nargs='?',
+        choices=['0', '1', '2'],
+        default='0',
+        help='Safe content filter from none to strict',
+    )
+    parser.add_argument(
+        '--timerange', type=str, nargs='?', choices=['day', 'week', 'month', 'year'], help='Filter by time range'
+    )
+    return parser.parse_args(args)

+
+
+if __name__ == '__main__':
+    settings_engines = searx.settings['engines']
+    searx.search.load_engines(settings_engines)
+    engine_cs = list(searx.engines.categories.keys())
+    prog_args = parse_argument(category_choices=engine_cs)
+    searx.search.initialize_network(settings_engines, searx.settings['outgoing'])
+    searx.search.check_network_configuration()
+    searx.search.initialize_metrics([engine['name'] for engine in settings_engines])
+    searx.search.initialize_processors(settings_engines)
+    search_q = get_search_query(prog_args, engine_categories=engine_cs)
+    res_dict = to_dict(search_q)
+    sys.stdout.write(dumps(res_dict, sort_keys=True, indent=4, ensure_ascii=False, default=json_serial))
+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searxng_extra/update/update_engine_descriptions.html b/_modules/searxng_extra/update/update_engine_descriptions.html new file mode 100644 index 00000000..d3d3c7b4 --- /dev/null +++ b/_modules/searxng_extra/update/update_engine_descriptions.html @@ -0,0 +1,420 @@ + + + + + + + + + searxng_extra.update.update_engine_descriptions — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searxng_extra.update.update_engine_descriptions

+#!/usr/bin/env python
+# lint: pylint
+# SPDX-License-Identifier: AGPL-3.0-or-later
+
+"""Fetch website description from websites and from
+:origin:`searx/engines/wikidata.py` engine.
+
+Output file: :origin:`searx/data/engine_descriptions.json`.
+
+"""
+
+# pylint: disable=invalid-name, global-statement
+
+import json
+from urllib.parse import urlparse
+from os.path import join
+
+from lxml.html import fromstring
+
+from searx.engines import wikidata, set_loggers
+from searx.utils import extract_text, match_language
+from searx.locales import LOCALE_NAMES, locales_initialize
+from searx import searx_dir
+from searx.utils import gen_useragent, detect_language
+import searx.search
+import searx.network
+
+set_loggers(wikidata, 'wikidata')
+locales_initialize()
+
+SPARQL_WIKIPEDIA_ARTICLE = """
+SELECT DISTINCT ?item ?name
+WHERE {
+  hint:Query hint:optimizer "None".
+  VALUES ?item { %IDS% }
+  ?article schema:about ?item ;
+              schema:inLanguage ?lang ;
+              schema:name ?name ;
+              schema:isPartOf [ wikibase:wikiGroup "wikipedia" ] .
+  FILTER(?lang in (%LANGUAGES_SPARQL%)) .
+  FILTER (!CONTAINS(?name, ':')) .
+}
+"""
+
+SPARQL_DESCRIPTION = """
+SELECT DISTINCT ?item ?itemDescription
+WHERE {
+  VALUES ?item { %IDS% }
+  ?item schema:description ?itemDescription .
+  FILTER (lang(?itemDescription) in (%LANGUAGES_SPARQL%))
+}
+ORDER BY ?itemLang
+"""
+
+NOT_A_DESCRIPTION = [
+    'web site',
+    'site web',
+    'komputa serĉilo',
+    'interreta serĉilo',
+    'bilaketa motor',
+    'web search engine',
+    'wikimedia täpsustuslehekülg',
+]
+
+SKIP_ENGINE_SOURCE = [
+    # fmt: off
+    ('gitlab', 'wikidata')
+    # descriptions are about wikipedia disambiguation pages
+    # fmt: on
+]
+
+LANGUAGES = LOCALE_NAMES.keys()
+WIKIPEDIA_LANGUAGES = {'language': 'wikipedia_language'}
+LANGUAGES_SPARQL = ''
+IDS = None
+
+descriptions = {}
+wd_to_engine_name = {}
+
+
+def normalize_description(description):
+    for c in [chr(c) for c in range(0, 31)]:
+        description = description.replace(c, ' ')
+    description = ' '.join(description.strip().split())
+    return description
+
+
+def update_description(engine_name, lang, description, source, replace=True):
+    if not isinstance(description, str):
+        return
+    description = normalize_description(description)
+    if description.lower() == engine_name.lower():
+        return
+    if description.lower() in NOT_A_DESCRIPTION:
+        return
+    if (engine_name, source) in SKIP_ENGINE_SOURCE:
+        return
+    if ' ' not in description:
+        # skip unique word description (like "website")
+        return
+    if replace or lang not in descriptions[engine_name]:
+        descriptions[engine_name][lang] = [description, source]
+
+
+def get_wikipedia_summary(lang, pageid):
+    params = {'language': lang.replace('_', '-'), 'headers': {}}
+    searx.engines.engines['wikipedia'].request(pageid, params)
+    try:
+        response = searx.network.get(params['url'], headers=params['headers'], timeout=10)
+        response.raise_for_status()
+        api_result = json.loads(response.text)
+        return api_result.get('extract')
+    except Exception:  # pylint: disable=broad-except
+        return None
+
+
+def get_website_description(url, lang1, lang2=None):
+    headers = {
+        'User-Agent': gen_useragent(),
+        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
+        'DNT': '1',
+        'Upgrade-Insecure-Requests': '1',
+        'Sec-GPC': '1',
+        'Cache-Control': 'max-age=0',
+    }
+    if lang1 is not None:
+        lang_list = [lang1]
+        if lang2 is not None:
+            lang_list.append(lang2)
+        headers['Accept-Language'] = f'{",".join(lang_list)};q=0.8'
+    try:
+        response = searx.network.get(url, headers=headers, timeout=10)
+        response.raise_for_status()
+    except Exception:  # pylint: disable=broad-except
+        return (None, None)
+
+    try:
+        html = fromstring(response.text)
+    except ValueError:
+        html = fromstring(response.content)
+
+    description = extract_text(html.xpath('/html/head/meta[@name="description"]/@content'))
+    if not description:
+        description = extract_text(html.xpath('/html/head/meta[@property="og:description"]/@content'))
+    if not description:
+        description = extract_text(html.xpath('/html/head/title'))
+    lang = extract_text(html.xpath('/html/@lang'))
+    if lang is None and len(lang1) > 0:
+        lang = lang1
+    lang = detect_language(description) or lang or 'en'
+    lang = lang.split('_')[0]
+    lang = lang.split('-')[0]
+    return (lang, description)
+
+
+def initialize():
+    global IDS, WIKIPEDIA_LANGUAGES, LANGUAGES_SPARQL
+    searx.search.initialize()
+    wikipedia_engine = searx.engines.engines['wikipedia']
+    WIKIPEDIA_LANGUAGES = {language: wikipedia_engine.url_lang(language.replace('_', '-')) for language in LANGUAGES}
+    WIKIPEDIA_LANGUAGES['nb_NO'] = 'no'
+    LANGUAGES_SPARQL = ', '.join(f"'{l}'" for l in set(WIKIPEDIA_LANGUAGES.values()))
+    for engine_name, engine in searx.engines.engines.items():
+        descriptions[engine_name] = {}
+        wikidata_id = getattr(engine, "about", {}).get('wikidata_id')
+        if wikidata_id is not None:
+            wd_to_engine_name.setdefault(wikidata_id, set()).add(engine_name)
+
+    IDS = ' '.join(list(map(lambda wd_id: 'wd:' + wd_id, wd_to_engine_name.keys())))
+
+
+def fetch_wikidata_descriptions():
+    searx.network.set_timeout_for_thread(60)
+    result = wikidata.send_wikidata_query(
+        SPARQL_DESCRIPTION.replace('%IDS%', IDS).replace('%LANGUAGES_SPARQL%', LANGUAGES_SPARQL)
+    )
+    if result is not None:
+        for binding in result['results']['bindings']:
+            wikidata_id = binding['item']['value'].replace('http://www.wikidata.org/entity/', '')
+            wikidata_lang = binding['itemDescription']['xml:lang']
+            description = binding['itemDescription']['value']
+            for engine_name in wd_to_engine_name[wikidata_id]:
+                for lang in LANGUAGES:
+                    if WIKIPEDIA_LANGUAGES[lang] == wikidata_lang:
+                        update_description(engine_name, lang, description, 'wikidata')
+
+
+def fetch_wikipedia_descriptions():
+    result = wikidata.send_wikidata_query(
+        SPARQL_WIKIPEDIA_ARTICLE.replace('%IDS%', IDS).replace('%LANGUAGES_SPARQL%', LANGUAGES_SPARQL)
+    )
+    if result is not None:
+        for binding in result['results']['bindings']:
+            wikidata_id = binding['item']['value'].replace('http://www.wikidata.org/entity/', '')
+            wikidata_lang = binding['name']['xml:lang']
+            pageid = binding['name']['value']
+            for engine_name in wd_to_engine_name[wikidata_id]:
+                for lang in LANGUAGES:
+                    if WIKIPEDIA_LANGUAGES[lang] == wikidata_lang:
+                        description = get_wikipedia_summary(lang, pageid)
+                        update_description(engine_name, lang, description, 'wikipedia')
+
+
+def normalize_url(url):
+    url = url.replace('{language}', 'en')
+    url = urlparse(url)._replace(path='/', params='', query='', fragment='').geturl()
+    url = url.replace('https://api.', 'https://')
+    return url
+
+
+def fetch_website_description(engine_name, website):
+    default_lang, default_description = get_website_description(website, None, None)
+    if default_lang is None or default_description is None:
+        # the front page can't be fetched: skip this engine
+        return
+
+    wikipedia_languages_r = {V: K for K, V in WIKIPEDIA_LANGUAGES.items()}
+    languages = ['en', 'es', 'pt', 'ru', 'tr', 'fr']
+    languages = languages + [l for l in LANGUAGES if l not in languages]
+
+    previous_matched_lang = None
+    previous_count = 0
+    for lang in languages:
+        if lang not in descriptions[engine_name]:
+            fetched_lang, desc = get_website_description(website, lang, WIKIPEDIA_LANGUAGES[lang])
+            if fetched_lang is None or desc is None:
+                continue
+            matched_lang = match_language(fetched_lang, LANGUAGES, fallback=None)
+            if matched_lang is None:
+                fetched_wikipedia_lang = match_language(fetched_lang, WIKIPEDIA_LANGUAGES.values(), fallback=None)
+                matched_lang = wikipedia_languages_r.get(fetched_wikipedia_lang)
+            if matched_lang is not None:
+                update_description(engine_name, matched_lang, desc, website, replace=False)
+            # check if desc changed with the different lang values
+            if matched_lang == previous_matched_lang:
+                previous_count += 1
+                if previous_count == 6:
+                    # the website has returned the same description for 6 different languages in Accept-Language header
+                    # stop now
+                    break
+            else:
+                previous_matched_lang = matched_lang
+                previous_count = 0
+
+
+def fetch_website_descriptions():
+    for engine_name, engine in searx.engines.engines.items():
+        website = getattr(engine, "about", {}).get('website')
+        if website is None and hasattr(engine, "search_url"):
+            website = normalize_url(getattr(engine, "search_url"))
+        if website is None and hasattr(engine, "base_url"):
+            website = normalize_url(getattr(engine, "base_url"))
+        if website is not None:
+            fetch_website_description(engine_name, website)
+
+
+def get_engine_descriptions_filename():
+    return join(join(searx_dir, "data"), "engine_descriptions.json")
+
+
+
[docs]def get_output():
+    """
+    From descriptions[engine][language] = [description, source]
+    To
+
+    * output[language][engine] = description_and_source
+    * description_and_source can be:
+       * [description, source]
+       * description (if source = "wikipedia")
+       * [f"engine:lang", "ref"] (reference to another existing description)
+    """
+    output = {locale: {} for locale in LOCALE_NAMES}
+
+    seen_descriptions = {}
+
+    for engine_name, lang_descriptions in descriptions.items():
+        for language, description in lang_descriptions.items():
+            if description[0] in seen_descriptions:
+                ref = seen_descriptions[description[0]]
+                description = [f'{ref[0]}:{ref[1]}', 'ref']
+            else:
+                seen_descriptions[description[0]] = (engine_name, language)
+                if description[1] == 'wikipedia':
+                    description = description[0]
+            output.setdefault(language, {}).setdefault(engine_name, description)
+
+    return output

+
+
+def main():
+    initialize()
+    print('Fetching wikidata descriptions')
+    fetch_wikidata_descriptions()
+    print('Fetching wikipedia descriptions')
+    fetch_wikipedia_descriptions()
+    print('Fetching website descriptions')
+    fetch_website_descriptions()
+
+    output = get_output()
+    with open(get_engine_descriptions_filename(), 'w', encoding='utf8') as f:
+        f.write(json.dumps(output, indent=1, separators=(',', ':'), ensure_ascii=False))
+
+
+if __name__ == "__main__":
+    main()
+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searxng_extra/update/update_external_bangs.html b/_modules/searxng_extra/update/update_external_bangs.html new file mode 100644 index 00000000..d3d02810 --- /dev/null +++ b/_modules/searxng_extra/update/update_external_bangs.html @@ -0,0 +1,278 @@ + + + + + + + + + searxng_extra.update.update_external_bangs — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searxng_extra.update.update_external_bangs

+#!/usr/bin/env python
+# lint: pylint
+# SPDX-License-Identifier: AGPL-3.0-or-later
+"""Update :origin:`searx/data/external_bangs.json` using the duckduckgo bangs
+(:origin:`CI Update data ... <.github/workflows/data-update.yml>`).
+
+https://duckduckgo.com/newbang loads:
+
+* a javascript which provides the bang version ( https://duckduckgo.com/bv1.js )
+* a JSON file which contains the bangs ( https://duckduckgo.com/bang.v260.js for example )
+
+This script loads the javascript, then the bangs.
+
+The javascript URL may change in the future ( for example
+https://duckduckgo.com/bv2.js ), but most probably it will requires to update
+RE_BANG_VERSION
+
+"""
+# pylint: disable=C0116
+
+import json
+import re
+from os.path import join
+
+import httpx
+
+from searx import searx_dir  # pylint: disable=E0401 C0413
+from searx.external_bang import LEAF_KEY
+
+# from https://duckduckgo.com/newbang
+URL_BV1 = 'https://duckduckgo.com/bv1.js'
+RE_BANG_VERSION = re.compile(r'\/bang\.v([0-9]+)\.js')
+HTTPS_COLON = 'https:'
+HTTP_COLON = 'http:'
+
+
+def get_bang_url():
+    response = httpx.get(URL_BV1)
+    response.raise_for_status()
+
+    r = RE_BANG_VERSION.findall(response.text)
+    return f'https://duckduckgo.com/bang.v{r[0]}.js', r[0]
+
+
+def fetch_ddg_bangs(url):
+    response = httpx.get(url)
+    response.raise_for_status()
+    return json.loads(response.content.decode())
+
+
+
[docs]def merge_when_no_leaf(node):
+    """Minimize the number of nodes
+
+    ``A -> B -> C``
+
+    - ``B`` is child of ``A``
+    - ``C`` is child of ``B``
+
+    If there are no ``C`` equals to ``<LEAF_KEY>``, then each ``C`` are merged
+    into ``A``.  For example (5 nodes)::
+
+      d -> d -> g -> <LEAF_KEY> (ddg)
+        -> i -> g -> <LEAF_KEY> (dig)
+
+    becomes (3 noodes)::
+
+      d -> dg -> <LEAF_KEY>
+        -> ig -> <LEAF_KEY>
+
+    """
+    restart = False
+    if not isinstance(node, dict):
+        return
+
+    # create a copy of the keys so node can be modified
+    keys = list(node.keys())
+
+    for key in keys:
+        if key == LEAF_KEY:
+            continue
+
+        value = node[key]
+        value_keys = list(value.keys())
+        if LEAF_KEY not in value_keys:
+            for value_key in value_keys:
+                node[key + value_key] = value[value_key]
+                merge_when_no_leaf(node[key + value_key])
+            del node[key]
+            restart = True
+        else:
+            merge_when_no_leaf(value)
+
+    if restart:
+        merge_when_no_leaf(node)

+
+
+def optimize_leaf(parent, parent_key, node):
+    if not isinstance(node, dict):
+        return
+
+    if len(node) == 1 and LEAF_KEY in node and parent is not None:
+        parent[parent_key] = node[LEAF_KEY]
+    else:
+        for key, value in node.items():
+            optimize_leaf(node, key, value)
+
+
+def parse_ddg_bangs(ddg_bangs):
+    bang_trie = {}
+    bang_urls = {}
+
+    for bang_definition in ddg_bangs:
+        # bang_list
+        bang_url = bang_definition['u']
+        if '{{{s}}}' not in bang_url:
+            # ignore invalid bang
+            continue
+
+        bang_url = bang_url.replace('{{{s}}}', chr(2))
+
+        # only for the https protocol: "https://example.com" becomes "//example.com"
+        if bang_url.startswith(HTTPS_COLON + '//'):
+            bang_url = bang_url[len(HTTPS_COLON) :]
+
+        #
+        if bang_url.startswith(HTTP_COLON + '//') and bang_url[len(HTTP_COLON) :] in bang_urls:
+            # if the bang_url uses the http:// protocol, and the same URL exists in https://
+            # then reuse the https:// bang definition. (written //example.com)
+            bang_def_output = bang_urls[bang_url[len(HTTP_COLON) :]]
+        else:
+            # normal use case : new http:// URL or https:// URL (without "https:", see above)
+            bang_rank = str(bang_definition['r'])
+            bang_def_output = bang_url + chr(1) + bang_rank
+            bang_def_output = bang_urls.setdefault(bang_url, bang_def_output)
+
+        bang_urls[bang_url] = bang_def_output
+
+        # bang name
+        bang = bang_definition['t']
+
+        # bang_trie
+        t = bang_trie
+        for bang_letter in bang:
+            t = t.setdefault(bang_letter, {})
+        t = t.setdefault(LEAF_KEY, bang_def_output)
+
+    # optimize the trie
+    merge_when_no_leaf(bang_trie)
+    optimize_leaf(None, None, bang_trie)
+
+    return bang_trie
+
+
+def get_bangs_filename():
+    return join(join(searx_dir, "data"), "external_bangs.json")
+
+
+if __name__ == '__main__':
+    bangs_url, bangs_version = get_bang_url()
+    print(f'fetch bangs from {bangs_url}')
+    output = {'version': bangs_version, 'trie': parse_ddg_bangs(fetch_ddg_bangs(bangs_url))}
+    with open(get_bangs_filename(), 'w', encoding="utf8") as fp:
+        json.dump(output, fp, ensure_ascii=False, indent=4)
+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_modules/searxng_extra/update/update_languages.html b/_modules/searxng_extra/update/update_languages.html new file mode 100644 index 00000000..320a9abf --- /dev/null +++ b/_modules/searxng_extra/update/update_languages.html @@ -0,0 +1,428 @@ + + + + + + + + + searxng_extra.update.update_languages — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +

Source code for searxng_extra.update.update_languages

+#!/usr/bin/env python
+# lint: pylint
+
+# SPDX-License-Identifier: AGPL-3.0-or-later
+"""This script generates languages.py from intersecting each engine's supported
+languages.
+
+Output files: :origin:`searx/data/engines_languages.json` and
+:origin:`searx/languages.py` (:origin:`CI Update data ...
+<.github/workflows/data-update.yml>`).
+
+"""
+
+# pylint: disable=invalid-name
+from unicodedata import lookup
+import json
+from pathlib import Path
+from pprint import pformat
+from babel import Locale, UnknownLocaleError
+from babel.languages import get_global
+from babel.core import parse_locale
+
+from searx import settings, searx_dir
+from searx.engines import load_engines, engines
+from searx.network import set_timeout_for_thread
+
+# Output files.
+engines_languages_file = Path(searx_dir) / 'data' / 'engines_languages.json'
+languages_file = Path(searx_dir) / 'languages.py'
+
+
+# Fetches supported languages for each engine and writes json file with those.
+def fetch_supported_languages():
+    set_timeout_for_thread(10.0)
+
+    engines_languages = {}
+    names = list(engines)
+    names.sort()
+
+    for engine_name in names:
+        if hasattr(engines[engine_name], 'fetch_supported_languages'):
+            engines_languages[engine_name] = engines[engine_name].fetch_supported_languages()
+            print("fetched %s languages from engine %s" % (len(engines_languages[engine_name]), engine_name))
+            if type(engines_languages[engine_name]) == list:  # pylint: disable=unidiomatic-typecheck
+                engines_languages[engine_name] = sorted(engines_languages[engine_name])
+
+    print("fetched languages from %s engines" % len(engines_languages))
+
+    # write json file
+    with open(engines_languages_file, 'w', encoding='utf-8') as f:
+        json.dump(engines_languages, f, indent=2, sort_keys=True)
+
+    return engines_languages
+
+
+# Get babel Locale object from lang_code if possible.
+def get_locale(lang_code):
+    try:
+        locale = Locale.parse(lang_code, sep='-')
+        return locale
+    except (UnknownLocaleError, ValueError):
+        return None
+
+
+lang2emoji = {
+    'ha': '\U0001F1F3\U0001F1EA',  # Hausa / Niger
+    'bs': '\U0001F1E7\U0001F1E6',  # Bosnian / Bosnia & Herzegovina
+    'jp': '\U0001F1EF\U0001F1F5',  # Japanese
+    'ua': '\U0001F1FA\U0001F1E6',  # Ukrainian
+    'he': '\U0001F1EE\U0001F1F7',  # Hebrew
+}
+
+
+
[docs]def get_unicode_flag(lang_code):
+    """Determine a unicode flag (emoji) that fits to the ``lang_code``"""
+
+    emoji = lang2emoji.get(lang_code.lower())
+    if emoji:
+        return emoji
+
+    if len(lang_code) == 2:
+        return '\U0001F310'
+
+    language = territory = script = variant = ''
+    try:
+        language, territory, script, variant = parse_locale(lang_code, '-')
+    except ValueError as exc:
+        print(exc)
+
+    # https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2
+    if not territory:
+        # https://www.unicode.org/emoji/charts/emoji-list.html#country-flag
+        emoji = lang2emoji.get(language)
+        if not emoji:
+            print(
+                "%s --> language: %s / territory: %s / script: %s / variant: %s"
+                % (lang_code, language, territory, script, variant)
+            )
+        return emoji
+
+    emoji = lang2emoji.get(territory.lower())
+    if emoji:
+        return emoji
+
+    try:
+        c1 = lookup('REGIONAL INDICATOR SYMBOL LETTER ' + territory[0])
+        c2 = lookup('REGIONAL INDICATOR SYMBOL LETTER ' + territory[1])
+        # print("%s --> territory: %s --> %s%s" %(lang_code, territory, c1, c2 ))
+    except KeyError as exc:
+        print("%s --> territory: %s --> %s" % (lang_code, territory, exc))
+        return None
+
+    return c1 + c2

+
+
+def get_territory_name(lang_code):
+    country_name = None
+    locale = get_locale(lang_code)
+    try:
+        if locale is not None:
+            country_name = locale.get_territory_name()
+    except FileNotFoundError as exc:
+        print("ERROR: %s --> %s" % (locale, exc))
+    return country_name
+
+
+# Join all language lists.
+def join_language_lists(engines_languages):
+    language_list = {}
+    for engine_name in engines_languages:
+        for lang_code in engines_languages[engine_name]:
+
+            # apply custom fixes if necessary
+            if lang_code in getattr(engines[engine_name], 'language_aliases', {}).values():
+                lang_code = next(
+                    lc for lc, alias in engines[engine_name].language_aliases.items() if lang_code == alias
+                )
+
+            locale = get_locale(lang_code)
+
+            # ensure that lang_code uses standard language and country codes
+            if locale and locale.territory:
+                lang_code = "{lang}-{country}".format(lang=locale.language, country=locale.territory)
+            short_code = lang_code.split('-')[0]
+
+            # add language without country if not in list
+            if short_code not in language_list:
+                if locale:
+                    # get language's data from babel's Locale object
+                    language_name = locale.get_language_name().title()
+                    english_name = locale.english_name.split(' (')[0]
+                elif short_code in engines_languages['wikipedia']:
+                    # get language's data from wikipedia if not known by babel
+                    language_name = engines_languages['wikipedia'][short_code]['name']
+                    english_name = engines_languages['wikipedia'][short_code]['english_name']
+                else:
+                    language_name = None
+                    english_name = None
+
+                # add language to list
+                language_list[short_code] = {
+                    'name': language_name,
+                    'english_name': english_name,
+                    'counter': set(),
+                    'countries': {},
+                }
+
+            # add language with country if not in list
+            if lang_code != short_code and lang_code not in language_list[short_code]['countries']:
+                country_name = ''
+                if locale:
+                    # get country name from babel's Locale object
+                    try:
+                        country_name = locale.get_territory_name()
+                    except FileNotFoundError as exc:
+                        print("ERROR: %s --> %s" % (locale, exc))
+                        locale = None
+
+                language_list[short_code]['countries'][lang_code] = {
+                    'country_name': country_name,
+                    'counter': set(),
+                }
+
+            # count engine for both language_country combination and language alone
+            language_list[short_code]['counter'].add(engine_name)
+            if lang_code != short_code:
+                language_list[short_code]['countries'][lang_code]['counter'].add(engine_name)
+
+    return language_list
+
+
+# Filter language list so it only includes the most supported languages and countries
+def filter_language_list(all_languages):
+    min_engines_per_lang = 12
+    min_engines_per_country = 7
+    # pylint: disable=consider-using-dict-items, consider-iterating-dictionary
+    main_engines = [
+        engine_name
+        for engine_name in engines.keys()
+        if 'general' in engines[engine_name].categories
+        and engines[engine_name].supported_languages
+        and not engines[engine_name].disabled
+    ]
+
+    # filter list to include only languages supported by most engines or all default general engines
+    filtered_languages = {
+        code: lang
+        for code, lang in all_languages.items()
+        if (
+            len(lang['counter']) >= min_engines_per_lang
+            or all(main_engine in lang['counter'] for main_engine in main_engines)
+        )
+    }
+
+    def _copy_lang_data(lang, country_name=None):
+        new_dict = {}
+        new_dict['name'] = all_languages[lang]['name']
+        new_dict['english_name'] = all_languages[lang]['english_name']
+        if country_name:
+            new_dict['country_name'] = country_name
+        return new_dict
+
+    # for each language get country codes supported by most engines or at least one country code
+    filtered_languages_with_countries = {}
+    for lang, lang_data in filtered_languages.items():
+        countries = lang_data['countries']
+        filtered_countries = {}
+
+        # get language's country codes with enough supported engines
+        for lang_country, country_data in countries.items():
+            if len(country_data['counter']) >= min_engines_per_country:
+                filtered_countries[lang_country] = _copy_lang_data(lang, country_data['country_name'])
+
+        # add language without countries too if there's more than one country to choose from
+        if len(filtered_countries) > 1:
+            filtered_countries[lang] = _copy_lang_data(lang, None)
+        elif len(filtered_countries) == 1:
+            lang_country = next(iter(filtered_countries))
+
+        # if no country has enough engines try to get most likely country code from babel
+        if not filtered_countries:
+            lang_country = None
+            subtags = get_global('likely_subtags').get(lang)
+            if subtags:
+                country_code = subtags.split('_')[-1]
+                if len(country_code) == 2:
+                    lang_country = "{lang}-{country}".format(lang=lang, country=country_code)
+
+            if lang_country:
+                filtered_countries[lang_country] = _copy_lang_data(lang, None)
+            else:
+                filtered_countries[lang] = _copy_lang_data(lang, None)
+
+        filtered_languages_with_countries.update(filtered_countries)
+
+    return filtered_languages_with_countries
+
+
+
[docs]class UnicodeEscape(str):
+    """Escape unicode string in :py:obj:`pprint.pformat`"""
+
+    def __repr__(self):
+        return "'" + "".join([chr(c) for c in self.encode('unicode-escape')]) + "'"

+
+
+# Write languages.py.
+def write_languages_file(languages):
+    file_headers = (
+        "# -*- coding: utf-8 -*-",
+        "# list of language codes",
+        "# this file is generated automatically by utils/fetch_languages.py",
+        "language_codes = (\n",
+    )
+
+    language_codes = []
+
+    for code in sorted(languages):
+
+        name = languages[code]['name']
+        if name is None:
+            print("ERROR: languages['%s'] --> %s" % (code, languages[code]))
+            continue
+
+        flag = get_unicode_flag(code) or ''
+        item = (
+            code,
+            languages[code]['name'].split(' (')[0],
+            get_territory_name(code) or '',
+            languages[code].get('english_name') or '',
+            UnicodeEscape(flag),
+        )
+
+        language_codes.append(item)
+
+    language_codes = tuple(language_codes)
+
+    with open(languages_file, 'w', encoding='utf-8') as new_file:
+        file_content = "{file_headers} {language_codes},\n)\n".format(
+            # fmt: off
+            file_headers = '\n'.join(file_headers),
+            language_codes = pformat(language_codes, indent=4)[1:-1]
+            # fmt: on
+        )
+        new_file.write(file_content)
+        new_file.close()
+
+
+if __name__ == "__main__":
+    load_engines(settings['engines'])
+    _engines_languages = fetch_supported_languages()
+    _all_languages = join_language_lists(_engines_languages)
+    _filtered_languages = filter_language_list(_all_languages)
+    write_languages_file(_filtered_languages)
+
+ +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/_sources/admin/api.rst.txt b/_sources/admin/api.rst.txt new file mode 100644 index 00000000..7bf46540 --- /dev/null +++ b/_sources/admin/api.rst.txt @@ -0,0 +1,96 @@ +.. _adminapi: + +================== +Administration API +================== + +Get configuration data +====================== + +.. code:: http + + GET /config HTTP/1.1 + +Sample response +--------------- + +.. code:: json + + { + "autocomplete": "", + "categories": [ + "map", + "it", + "images", + ], + "default_locale": "", + "default_theme": "simple", + "engines": [ + { + "categories": [ + "map" + ], + "enabled": true, + "name": "openstreetmap", + "shortcut": "osm" + }, + { + "categories": [ + "it" + ], + "enabled": true, + "name": "arch linux wiki", + "shortcut": "al" + }, + { + "categories": [ + "images" + ], + "enabled": true, + "name": "google images", + "shortcut": "goi" + }, + { + "categories": [ + "it" + ], + "enabled": false, + "name": "bitbucket", + "shortcut": "bb" + }, + ], + "instance_name": "searx", + "locales": { + "de": "Deutsch (German)", + "en": "English", + "eo": "Esperanto (Esperanto)", + }, + "plugins": [ + { + "enabled": true, + "name": "HTTPS rewrite" + }, + { + "enabled": false, + "name": "Vim-like hotkeys" + } + ], + "safe_search": 0 + } + + +Embed search bar +================ + +The search bar can be embedded into websites. Just paste the example into the +HTML of the site. URL of the SearXNG instance and values are customizable. + +.. code:: html + +
+ + + + + + diff --git a/_sources/admin/architecture.rst.txt b/_sources/admin/architecture.rst.txt new file mode 100644 index 00000000..d0d40715 --- /dev/null +++ b/_sources/admin/architecture.rst.txt @@ -0,0 +1,38 @@ +.. _architecture: + +============ +Architecture +============ + +.. sidebar:: Further reading + + - Reverse Proxy: :ref:`Apache ` & :ref:`nginx ` + - uWSGI: :ref:`searxng uwsgi` + - SearXNG: :ref:`installation basic` + +Herein you will find some hints and suggestions about typical architectures of +SearXNG infrastructures. + +.. _architecture uWSGI: + +uWSGI Setup +=========== + +We start with a *reference* setup for public SearXNG instances which can be build +up and maintained by the scripts from our :ref:`toolboxing`. + +.. _arch public: + +.. kernel-figure:: arch_public.dot + :alt: arch_public.dot + + Reference architecture of a public SearXNG setup. + +The reference installation activates ``server.limiter``, ``server.image_proxy`` +and ``ui.static_use_hash`` (:origin:`/etc/searxng/settings.yml +`) + +.. literalinclude:: ../../utils/templates/etc/searxng/settings.yml + :language: yaml + :end-before: # preferences: diff --git a/_sources/admin/buildhosts.rst.txt b/_sources/admin/buildhosts.rst.txt new file mode 100644 index 00000000..6926469e --- /dev/null +++ b/_sources/admin/buildhosts.rst.txt @@ -0,0 +1,155 @@ +.. _buildhosts: + +========== +Buildhosts +========== + +.. sidebar:: This article needs some work + + If you have any contribution send us your :pull:`PR <../pulls>`, see + :ref:`how to contribute`. + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +To get best results from build, its recommend to install additional packages +on build hosts (see :ref:`searxng.sh`).:: + + sudo -H ./utils/searxng.sh install buildhost + +This will install packages needed by searx: + +.. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START distro-packages + :end-before: END distro-packages + +and packages needed to build docuemtation and run tests: + +.. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START build-packages + :end-before: END build-packages + +.. _docs build: + +Build docs +========== + +.. _Graphviz: https://graphviz.gitlab.io +.. _ImageMagick: https://www.imagemagick.org +.. _XeTeX: https://tug.org/xetex/ +.. _dvisvgm: https://dvisvgm.de/ + +.. sidebar:: Sphinx build needs + + - ImageMagick_ + - Graphviz_ + - XeTeX_ + - dvisvgm_ + +Most of the sphinx requirements are installed from :origin:`setup.py` and the +docs can be build from scratch with ``make docs.html``. For better math and +image processing additional packages are needed. The XeTeX_ needed not only for +PDF creation, its also needed for :ref:`math` when HTML output is build. + +To be able to do :ref:`sphinx:math-support` without CDNs, the math are rendered +as images (``sphinx.ext.imgmath`` extension). + +Here is the extract from the :origin:`docs/conf.py` file, setting math renderer +to ``imgmath``: + +.. literalinclude:: ../conf.py + :language: python + :start-after: # sphinx.ext.imgmath setup + :end-before: # sphinx.ext.imgmath setup END + +If your docs build (``make docs.html``) shows warnings like this:: + + WARNING: dot(1) not found, for better output quality install \ + graphviz from https://www.graphviz.org + .. + WARNING: LaTeX command 'latex' cannot be run (needed for math \ + display), check the imgmath_latex setting + +you need to install additional packages on your build host, to get better HTML +output. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code-block:: sh + + $ sudo apt install graphviz imagemagick texlive-xetex librsvg2-bin + + .. group-tab:: Arch Linux + + .. code-block:: sh + + $ sudo pacman -S graphviz imagemagick texlive-bin extra/librsvg + + .. group-tab:: Fedora / RHEL + + .. code-block:: sh + + $ sudo dnf install graphviz graphviz-gd texlive-xetex-bin librsvg2-tools + + +For PDF output you also need: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + $ sudo apt texlive-latex-recommended texlive-extra-utils ttf-dejavu + + .. group-tab:: Arch Linux + + .. code:: sh + + $ sudo pacman -S texlive-core texlive-latexextra ttf-dejavu + + .. group-tab:: Fedora / RHEL + + .. code:: sh + + $ sudo dnf install \ + texlive-collection-fontsrecommended texlive-collection-latex \ + dejavu-sans-fonts dejavu-serif-fonts dejavu-sans-mono-fonts \ + ImageMagick + +.. _sh lint: + +Lint shell scripts +================== + +.. _ShellCheck: https://github.com/koalaman/shellcheck + +To lint shell scripts, we use ShellCheck_ - A shell script static analysis tool. + +.. SNIP sh lint requirements + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code-block:: sh + + $ sudo apt install shellcheck + + .. group-tab:: Arch Linux + + .. code-block:: sh + + $ sudo pacman -S shellcheck + + .. group-tab:: Fedora / RHEL + + .. code-block:: sh + + $ sudo dnf install ShellCheck + +.. SNAP sh lint requirements diff --git a/_sources/admin/engines/command-line-engines.rst.txt b/_sources/admin/engines/command-line-engines.rst.txt new file mode 100644 index 00000000..e9535e74 --- /dev/null +++ b/_sources/admin/engines/command-line-engines.rst.txt @@ -0,0 +1,79 @@ +.. _engine command: + +==================== +Command Line Engines +==================== + +.. sidebar:: info + + - :origin:`command.py ` + - :ref:`offline engines` + +With *command engines* administrators can run engines to integrate arbitrary +shell commands. + +When creating and enabling a ``command`` engine on a public instance, you must +be careful to avoid leaking private data. The easiest solution is to limit the +access by setting ``tokens`` as described in section :ref:`private engines`. + +The engine base is flexible. Only your imagination can limit the power of this +engine (and maybe security concerns). The following options are available: + +``command``: + A comma separated list of the elements of the command. A special token + ``{{QUERY}}`` tells where to put the search terms of the user. Example: + + .. code:: yaml + + ['ls', '-l', '-h', '{{QUERY}}'] + +``delimiter``: + A mapping containing a delimiter ``char`` and the *titles* of each element in + ``keys``. + +``parse_regex``: + A dict containing the regular expressions for each result key. + +``query_type``: + + The expected type of user search terms. Possible values: ``path`` and + ``enum``. + + ``path``: + Checks if the user provided path is inside the working directory. If not, + the query is not executed. + + ``enum``: + Is a list of allowed search terms. If the user submits something which is + not included in the list, the query returns an error. + +``query_enum``: + A list containing allowed search terms if ``query_type`` is set to ``enum``. + +``working_dir``: + + The directory where the command has to be executed. Default: ``./`` + +``result_separator``: + The character that separates results. Default: ``\n`` + +The example engine below can be used to find files with a specific name in the +configured working directory: + +.. code:: yaml + + - name: find + engine: command + command: ['find', '.', '-name', '{{QUERY}}'] + query_type: path + shortcut: fnd + delimiter: + chars: ' ' + keys: ['line'] + + +Acknowledgment +============== + +This development was sponsored by `Search and Discovery Fund +`_ of `NLnet Foundation `_. diff --git a/_sources/admin/engines/configured_engines.rst.txt b/_sources/admin/engines/configured_engines.rst.txt new file mode 100644 index 00000000..c7b6a1f5 --- /dev/null +++ b/_sources/admin/engines/configured_engines.rst.txt @@ -0,0 +1,75 @@ +.. _configured engines: + +================== +Configured Engines +================== + +.. sidebar:: Further reading .. + + - :ref:`engines-dev` + - :ref:`settings engine` + +Explanation of the :ref:`general engine configuration` shown in the table +:ref:`configured engines`. + +.. jinja:: searx + + SearXNG supports {{engines | length}} search engines (of which {{enabled_engine_count}} are enabled by default). + + {% for category, engines in categories_as_tabs.items() %} + + {{category}} search engines + --------------------------------------- + + {% for group, engines in engines | group_engines_in_tab %} + + {% if loop.length > 1 %} + {{group}} + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + {% endif %} + + .. flat-table:: + :header-rows: 2 + :stub-columns: 1 + + * - :cspan:`5` Engines configured by default (in :ref:`settings.yml `) + - :cspan:`3` :ref:`Supported features ` + + * - Name + - Shortcut + - Module + - Disabled + - Timeout + - Weight + - Paging + - Language + - Safe search + - Time range + + {% for mod in engines %} + + * - `{{mod.name}} <{{mod.about and mod.about.website}}>`_ + - ``!{{mod.shortcut}}`` + - {%- if 'searx.engines.' + mod.__name__ in documented_modules %} + :py:mod:`~searx.engines.{{mod.__name__}}` + {%- else %} + :origin:`{{mod.__name__}} ` + {%- endif %} + - {{(mod.disabled and "y") or ""}} + {%- if mod.about and mod.about.language %} + ({{mod.about.language | upper}}) + {%- endif %} + - {{mod.timeout}} + - {{mod.weight or 1 }} + {% if mod.engine_type == 'online' %} + - {{(mod.paging and "y") or ""}} + - {{(mod.language_support and "y") or ""}} + - {{(mod.safesearch and "y") or ""}} + - {{(mod.time_range_support and "y") or ""}} + {% else %} + - :cspan:`3` not applicable ({{mod.engine_type}}) + {% endif %} + + {% endfor %} + {% endfor %} + {% endfor %} diff --git a/_sources/admin/engines/index.rst.txt b/_sources/admin/engines/index.rst.txt new file mode 100644 index 00000000..f488731e --- /dev/null +++ b/_sources/admin/engines/index.rst.txt @@ -0,0 +1,23 @@ +.. _engines and settings: + +================== +Engines & Settings +================== + +.. sidebar:: Further reading .. + + - :ref:`settings engine` + - :ref:`engine settings` & :ref:`engine file` + +.. toctree:: + :maxdepth: 1 + + settings + configured_engines + private-engines + recoll + sql-engines + nosql-engines + search-indexer-engines + command-line-engines + searx.engines.xpath diff --git a/_sources/admin/engines/nosql-engines.rst.txt b/_sources/admin/engines/nosql-engines.rst.txt new file mode 100644 index 00000000..93f1dd58 --- /dev/null +++ b/_sources/admin/engines/nosql-engines.rst.txt @@ -0,0 +1,135 @@ +=============== +NoSQL databases +=============== + +.. sidebar:: further read + + - `NoSQL databases `_ + - `redis.io `_ + - `MongoDB `_ + +The following `NoSQL databases`_ are supported: + +- :ref:`engine redis_server` +- :ref:`engine mongodb` + +All of the engines above are just commented out in the :origin:`settings.yml +`, as you have to set various options and install +dependencies before using them. + +By default, the engines use the ``key-value`` template for displaying results / +see :origin:`simple ` +theme. If you are not satisfied with the original result layout, you can use +your own template, set ``result_template`` attribute to ``{template_name}`` and +place the templates at:: + + searx/templates/{theme_name}/result_templates/{template_name} + +Furthermore, if you do not wish to expose these engines on a public instance, you +can still add them and limit the access by setting ``tokens`` as described in +section :ref:`private engines`. + + +Configure the engines +===================== + +`NoSQL databases`_ are used for storing arbitrary data without first defining +their structure. + + +Extra Dependencies +------------------ + +For using :ref:`engine redis_server` or :ref:`engine mongodb` you need to +install additional packages in Python's Virtual Environment of your SearXNG +instance. To switch into the environment (:ref:`searxng-src`) you can use +:ref:`searxng.sh`:: + + $ sudo utils/searxng.sh instance cmd bash + (searxng-pyenv)$ pip install ... + + +.. _engine redis_server: + +Redis Server +------------ + +.. _redis: https://github.com/andymccurdy/redis-py#installation + +.. sidebar:: info + + - ``pip install`` redis_ + - redis.io_ + - :origin:`redis_server.py ` + + +Redis is an open source (BSD licensed), in-memory data structure (key value +based) store. Before configuring the ``redis_server`` engine, you must install +the dependency redis_. + +Select a database to search in and set its index in the option ``db``. You can +either look for exact matches or use partial keywords to find what you are +looking for by configuring ``exact_match_only``. You find an example +configuration below: + +.. code:: yaml + + # Required dependency: redis + + - name: myredis + shortcut : rds + engine: redis_server + exact_match_only: false + host: '127.0.0.1' + port: 6379 + enable_http: true + password: '' + db: 0 + +.. _engine mongodb: + +MongoDB +------- + +.. _pymongo: https://github.com/mongodb/mongo-python-driver#installation + +.. sidebar:: info + + - ``pip install`` pymongo_ + - MongoDB_ + - :origin:`mongodb.py ` + +MongoDB_ is a document based database program that handles JSON like data. +Before configuring the ``mongodb`` engine, you must install the dependency +redis_. + +In order to query MongoDB_, you have to select a ``database`` and a +``collection``. Furthermore, you have to select a ``key`` that is going to be +searched. MongoDB_ also supports the option ``exact_match_only``, so configure +it as you wish. Below is an example configuration for using a MongoDB +collection: + +.. code:: yaml + + # MongoDB engine + # Required dependency: pymongo + + - name: mymongo + engine: mongodb + shortcut: md + exact_match_only: false + host: '127.0.0.1' + port: 27017 + enable_http: true + results_per_page: 20 + database: 'business' + collection: 'reviews' # name of the db collection + key: 'name' # key in the collection to search for + + +Acknowledgment +============== + +This development was sponsored by `Search and Discovery Fund +`_ of `NLnet Foundation `_. + diff --git a/_sources/admin/engines/private-engines.rst.txt b/_sources/admin/engines/private-engines.rst.txt new file mode 100644 index 00000000..cc6ab256 --- /dev/null +++ b/_sources/admin/engines/private-engines.rst.txt @@ -0,0 +1,49 @@ +.. _private engines: + +============================ +Private Engines (``tokens``) +============================ + +Administrators might find themselves wanting to limit access to some of the +enabled engines on their instances. It might be because they do not want to +expose some private information through :ref:`offline engines`. Or they would +rather share engines only with their trusted friends or colleagues. + +To solve this issue the concept of *private engines* exists. + + +A new option was added to engines named `tokens`. It expects a list of +strings. If the user making a request presents one of the tokens of an engine, +they can access information about the engine and make search requests. + +Example configuration to restrict access to the Arch Linux Wiki engine: + +.. code:: yaml + + - name: arch linux wiki + engine: archlinux + shortcut: al + tokens: [ 'my-secret-token' ] + + +Unless a user has configured the right token, the engine is going +to be hidden from him/her. It is not going to be included in the +list of engines on the Preferences page and in the output of +`/config` REST API call. + +Tokens can be added to one's configuration on the Preferences page +under "Engine tokens". The input expects a comma separated list of +strings. + +The distribution of the tokens from the administrator to the users +is not carved in stone. As providing access to such engines +implies that the admin knows and trusts the user, we do not see +necessary to come up with a strict process. Instead, +we would like to add guidelines to the documentation of the feature. + + +Acknowledgment +============== + +This development was sponsored by `Search and Discovery Fund +`_ of `NLnet Foundation `_. diff --git a/_sources/admin/engines/recoll.rst.txt b/_sources/admin/engines/recoll.rst.txt new file mode 100644 index 00000000..ab853004 --- /dev/null +++ b/_sources/admin/engines/recoll.rst.txt @@ -0,0 +1,50 @@ +.. _engine recoll: + +============= +Recoll Engine +============= + +.. sidebar:: info + + - `Recoll `_ + - `recoll-webui `_ + - :origin:`searx/engines/recoll.py` + +Recoll_ is a desktop full-text search tool based on Xapian. By itself Recoll_ +does not offer WEB or API access, this can be achieved using recoll-webui_ + + +Configuration +============= + +You must configure the following settings: + +``base_url``: + Location where recoll-webui can be reached. + +``mount_prefix``: + Location where the file hierarchy is mounted on your *local* filesystem. + +``dl_prefix``: + Location where the file hierarchy as indexed by recoll can be reached. + +``search_dir``: + Part of the indexed file hierarchy to be search, if empty the full domain is + searched. + + +Example +======= + +Scenario: + +#. Recoll indexes a local filesystem mounted in ``/export/documents/reference``, +#. the Recoll search interface can be reached at https://recoll.example.org/ and +#. the contents of this filesystem can be reached though https://download.example.org/reference + +.. code:: yaml + + base_url: https://recoll.example.org/ + mount_prefix: /export/documents + dl_prefix: https://download.example.org + search_dir: '' diff --git a/_sources/admin/engines/search-indexer-engines.rst.txt b/_sources/admin/engines/search-indexer-engines.rst.txt new file mode 100644 index 00000000..51a66189 --- /dev/null +++ b/_sources/admin/engines/search-indexer-engines.rst.txt @@ -0,0 +1,136 @@ +==================== +Local Search Engines +==================== + +.. sidebar:: further read + + - `Comparison to alternatives + `_ + +Administrators might find themselves wanting to integrate locally running search +engines. The following ones are supported for now: + +* `Elasticsearch`_ +* `Meilisearch`_ +* `Solr`_ + +Each search engine is powerful, capable of full-text search. All of the engines +above are added to ``settings.yml`` just commented out, as you have to +``base_url`` for all them. + +Please note that if you are not using HTTPS to access these engines, you have to enable +HTTP requests by setting ``enable_http`` to ``True``. + +Furthermore, if you do not want to expose these engines on a public instance, you +can still add them and limit the access by setting ``tokens`` as described in +section :ref:`private engines`. + +.. _engine meilisearch: + +MeiliSearch +=========== + +.. sidebar:: info + + - :origin:`meilisearch.py ` + - `MeiliSearch `_ + - `MeiliSearch Documentation `_ + - `Install MeiliSearch + `_ + +MeiliSearch_ is aimed at individuals and small companies. It is designed for +small-scale (less than 10 million documents) data collections. E.g. it is great +for storing web pages you have visited and searching in the contents later. + +The engine supports faceted search, so you can search in a subset of documents +of the collection. Furthermore, you can search in MeiliSearch_ instances that +require authentication by setting ``auth_token``. + +Here is a simple example to query a Meilisearch instance: + +.. code:: yaml + + - name: meilisearch + engine: meilisearch + shortcut: mes + base_url: http://localhost:7700 + index: my-index + enable_http: true + + +.. _engine elasticsearch: + +Elasticsearch +============= + +.. sidebar:: info + + - :origin:`elasticsearch.py ` + - `Elasticsearch `_ + - `Elasticsearch Guide + `_ + - `Install Elasticsearch + `_ + +Elasticsearch_ supports numerous ways to query the data it is storing. At the +moment the engine supports the most popular search methods (``query_type``): + +- ``match``, +- ``simple_query_string``, +- ``term`` and +- ``terms``. + +If none of the methods fit your use case, you can select ``custom`` query type +and provide the JSON payload to submit to Elasticsearch in +``custom_query_json``. + +The following is an example configuration for an Elasticsearch_ instance with +authentication configured to read from ``my-index`` index. + +.. code:: yaml + + - name: elasticsearch + shortcut: es + engine: elasticsearch + base_url: http://localhost:9200 + username: elastic + password: changeme + index: my-index + query_type: match + # custom_query_json: '{ ... }' + enable_http: true + +.. _engine solr: + +Solr +==== + +.. sidebar:: info + + - :origin:`solr.py ` + - `Solr `_ + - `Solr Resources `_ + - `Install Solr `_ + +Solr_ is a popular search engine based on Lucene, just like Elasticsearch_. But +instead of searching in indices, you can search in collections. + +This is an example configuration for searching in the collection +``my-collection`` and get the results in ascending order. + +.. code:: yaml + + - name: solr + engine: solr + shortcut: slr + base_url: http://localhost:8983 + collection: my-collection + sort: asc + enable_http: true + + +Acknowledgment +============== + +This development was sponsored by `Search and Discovery Fund +`_ of `NLnet Foundation `_. diff --git a/_sources/admin/engines/searx.engines.xpath.rst.txt b/_sources/admin/engines/searx.engines.xpath.rst.txt new file mode 100644 index 00000000..695aa522 --- /dev/null +++ b/_sources/admin/engines/searx.engines.xpath.rst.txt @@ -0,0 +1,9 @@ +.. _xpath engine: + +============ +XPath Engine +============ + +.. automodule:: searx.engines.xpath + :members: + diff --git a/_sources/admin/engines/settings.rst.txt b/_sources/admin/engines/settings.rst.txt new file mode 100644 index 00000000..c747e3f4 --- /dev/null +++ b/_sources/admin/engines/settings.rst.txt @@ -0,0 +1,679 @@ +.. _settings.yml: + +================ +``settings.yml`` +================ + +This page describe the options possibilities of the :origin:`searx/settings.yml` +file. + +.. sidebar:: Further reading .. + + - :ref:`use_default_settings.yml` + - :ref:`search API` + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +.. _settings location: + +settings.yml location +===================== + +The initial ``settings.yml`` we be load from these locations: + +1. the full path specified in the ``SEARXNG_SETTINGS_PATH`` environment variable. +2. ``/etc/searxng/settings.yml`` + +If these files don't exist (or are empty or can't be read), SearXNG uses the +:origin:`searx/settings.yml` file. Read :ref:`settings use_default_settings` to +see how you can simplify your *user defined* ``settings.yml``. + + +.. _settings global: + +Global Settings +=============== + +.. _settings brand: + +``brand:`` +---------- + +.. code:: yaml + + brand: + issue_url: https://github.com/searxng/searxng/issues + docs_url: https://docs.searxng.org + public_instances: https://searx.space + wiki_url: https://github.com/searxng/searxng/wiki + +``issue_url`` : + If you host your own issue tracker change this URL. + +``docs_url`` : + If you host your own documentation change this URL. + +``public_instances`` : + If you host your own https://searx.space change this URL. + +``wiki_url`` : + Link to your wiki (or ``false``) + +.. _settings general: + +``general:`` +------------ + +.. code:: yaml + + general: + debug: false + instance_name: "SearXNG" + privacypolicy_url: false + donation_url: https://docs.searxng.org/donate.html + contact_url: false + enable_metrics: true + +``debug`` : ``$SEARXNG_DEBUG`` + Allow a more detailed log if you run SearXNG directly. Display *detailed* error + messages in the browser too, so this must be deactivated in production. + +``donation_url`` : + At default the donation link points to the `SearXNG project + `_. Set value to ``true`` to use your + own donation page written in the :ref:`searx/info/en/donate.md + ` and use ``false`` to disable the donation link altogether. + +``privacypolicy_url``: + Link to privacy policy. + +``contact_url``: + Contact ``mailto:`` address or WEB form. + +``enable_metrics``: + Enabled by default. Record various anonymous metrics availabled at ``/stats``, + ``/stats/errors`` and ``/preferences``. + +.. _settings search: + +``search:`` +----------- + +.. code:: yaml + + search: + safe_search: 0 + autocomplete: "" + default_lang: "" + ban_time_on_fail: 5 + max_ban_time_on_fail: 120 + formats: + - html + +``safe_search``: + Filter results. + + - ``0``: None + - ``1``: Moderate + - ``2``: Strict + +``autocomplete``: + Existing autocomplete backends, leave blank to turn it off. + + - ``dbpedia`` + - ``duckduckgo`` + - ``google`` + - ``startpage`` + - ``swisscows`` + - ``qwant`` + - ``wikipedia`` + +``default_lang``: + Default search language - leave blank to detect from browser information or + use codes from :origin:`searx/languages.py`. + +``languages``: + List of available languages - leave unset to use all codes from + :origin:`searx/languages.py`. Otherwise list codes of available languages. + The ``all`` value is shown as the ``Default language`` in the user interface + (in most cases, it is meant to send the query without a language parameter ; + in some cases, it means the English language) Example: + + .. code:: yaml + + languages: + - all + - en + - en-US + - de + - it-IT + - fr + - fr-BE + +``ban_time_on_fail``: + Ban time in seconds after engine errors. + +``max_ban_time_on_fail``: + Max ban time in seconds after engine errors. + +``formats``: + Result formats available from web, remove format to deny access (use lower + case). + + - ``html`` + - ``csv`` + - ``json`` + - ``rss`` + +.. _settings server: + +``server:`` +----------- + +.. code:: yaml + + server: + base_url: false # set custom base_url (or false) + port: 8888 + bind_address: "127.0.0.1" # address to listen on + secret_key: "ultrasecretkey" # change this! + limiter: false + image_proxy: false # proxying image results through SearXNG + default_http_headers: + X-Content-Type-Options : nosniff + X-XSS-Protection : 1; mode=block + X-Download-Options : noopen + X-Robots-Tag : noindex, nofollow + Referrer-Policy : no-referrer + +.. sidebar:: buildenv + + Changing a value tagged by :ref:`buildenv `, needs to + rebuild instance's environment :ref:`utils/brand.env `. + +``base_url`` : :ref:`buildenv SEARXNG_URL ` + The base URL where SearXNG is deployed. Used to create correct inbound links. + If you change the value, don't forget to rebuild instance's environment + (:ref:`utils/brand.env `) + +``port`` & ``bind_address``: :ref:`buildenv SEARXNG_PORT & SEARXNG_BIND_ADDRESS ` + Port number and *bind address* of the SearXNG web application if you run it + directly using ``python searx/webapp.py``. Doesn't apply to SearXNG running on + Apache or Nginx. + +``secret_key`` : ``$SEARXNG_SECRET`` + Used for cryptography purpose. + +.. _limiter: + +``limiter`` : + Rate limit the number of request on the instance, block some bots. The + :ref:`limiter plugin` requires a :ref:`settings redis` database. + +.. _image_proxy: + +``image_proxy`` : + Allow your instance of SearXNG of being able to proxy images. Uses memory space. + +.. _HTTP headers: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers + +``default_http_headers`` : + Set additional HTTP headers, see `#755 `__ + + +.. _settings ui: + +``ui:`` +------- + +.. _cache busting: + https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#caching_static_assets_with_cache_busting + +.. code:: yaml + + ui: + static_use_hash: false + default_locale: "" + query_in_title: false + infinite_scroll: false + center_alignment: false + cache_url: https://web.archive.org/web/ + default_theme: simple + theme_args: + simple_style: auto + +.. _static_use_hash: + +``static_use_hash`` : + Enables `cache busting`_ of static files. + +``default_locale`` : + SearXNG interface language. If blank, the locale is detected by using the + browser language. If it doesn't work, or you are deploying a language + specific instance of searx, a locale can be defined using an ISO language + code, like ``fr``, ``en``, ``de``. + +``query_in_title`` : + When true, the result page's titles contains the query it decreases the + privacy, since the browser can records the page titles. + +``infinite_scroll``: + When true, automatically loads the next page when scrolling to bottom of the current page. + +``center_alignment`` : default ``false`` + When enabled, the results are centered instead of being in the left (or RTL) + side of the screen. This setting only affects the *desktop layout* + (:origin:`min-width: @tablet `) + +.. cache_url: + +``cache_url`` : ``https://web.archive.org/web/`` + URL prefix of the internet archive or cache, don't forgett trailing slash (if + needed). The default is https://web.archive.org/web/ alternatives are: + + - https://webcache.googleusercontent.com/search?q=cache: + - https://archive.today/ + +``default_theme`` : + Name of the theme you want to use by default on your SearXNG instance. + +``theme_args.simple_style``: + Style of simple theme: ``auto``, ``light``, ``dark`` + +``results_on_new_tab``: + Open result links in a new tab by default. + + +.. _settings redis: + +``redis:`` +---------- + +.. _Redis.from_url(url): https://redis-py.readthedocs.io/en/stable/connections.html#redis.client.Redis.from_url + +A redis DB can be connected by an URL, in :py:obj:`searx.redisdb` you +will find a description to test your redis connection in SerXNG. When using +sockets, don't forget to check the access rights on the socket:: + + ls -la /usr/local/searxng-redis/run/redis.sock + srwxrwx--- 1 searxng-redis searxng-redis ... /usr/local/searxng-redis/run/redis.sock + +In this example read/write access is given to the *searxng-redis* group. To get +access rights to redis instance (the socket), your SearXNG (or even your +developer) account needs to be added to the *searxng-redis* group. + +``url`` + URL to connect redis database, see `Redis.from_url(url)`_ & :ref:`redis db`:: + + redis://[[username]:[password]]@localhost:6379/0 + rediss://[[username]:[password]]@localhost:6379/0 + unix://[[username]:[password]]@/path/to/socket.sock?db=0 + +.. admonition:: Tip for developers + + To set up a local redis instance, first set the socket path of the Redis DB + in your YAML setting: + + .. code:: yaml + + redis: + url: unix:///usr/local/searxng-redis/run/redis.sock?db=0 + + Then use the following commands to install the redis instance :: + + $ ./manage redis.build + $ sudo -H ./manage redis.install + $ sudo -H ./manage redis.addgrp "${USER}" + # don't forget to logout & login to get member of group + + +.. _settings outgoing: + +``outgoing:`` +------------- + +Communication with search engines. + +.. code:: yaml + + outgoing: + request_timeout: 2.0 # default timeout in seconds, can be override by engine + max_request_timeout: 10.0 # the maximum timeout in seconds + useragent_suffix: "" # information like an email address to the administrator + pool_connections: 100 # Maximum number of allowable connections, or null + # for no limits. The default is 100. + pool_maxsize: 10 # Number of allowable keep-alive connections, or null + # to always allow. The default is 10. + enable_http2: true # See https://www.python-httpx.org/http2/ + # uncomment below section if you want to use a custom server certificate + # see https://www.python-httpx.org/advanced/#changing-the-verification-defaults + # and https://www.python-httpx.org/compatibility/#ssl-configuration + # verify: ~/.mitmproxy/mitmproxy-ca-cert.cer + # + # uncomment below section if you want to use a proxyq see: SOCKS proxies + # https://2.python-requests.org/en/latest/user/advanced/#proxies + # are also supported: see + # https://2.python-requests.org/en/latest/user/advanced/#socks + # + # proxies: + # all://: + # - http://proxy1:8080 + # - http://proxy2:8080 + # + # using_tor_proxy: true + # + # Extra seconds to add in order to account for the time taken by the proxy + # + # extra_proxy_timeout: 10.0 + # + +``request_timeout`` : + Global timeout of the requests made to others engines in seconds. A bigger + timeout will allow to wait for answers from slow engines, but in consequence + will slow SearXNG reactivity (the result page may take the time specified in the + timeout to load). Can be override by :ref:`settings engine` + +``useragent_suffix`` : + Suffix to the user-agent SearXNG uses to send requests to others engines. If an + engine wish to block you, a contact info here may be useful to avoid that. + +``keepalive_expiry`` : + Number of seconds to keep a connection in the pool. By default 5.0 seconds. + +.. _httpx proxies: https://www.python-httpx.org/advanced/#http-proxying + +``proxies`` : + Define one or more proxies you wish to use, see `httpx proxies`_. + If there are more than one proxy for one protocol (http, https), + requests to the engines are distributed in a round-robin fashion. + +``source_ips`` : + If you use multiple network interfaces, define from which IP the requests must + be made. Example: + + * ``0.0.0.0`` any local IPv4 address. + * ``::`` any local IPv6 address. + * ``192.168.0.1`` + * ``[ 192.168.0.1, 192.168.0.2 ]`` these two specific IP addresses + * ``fe80::60a2:1691:e5a2:ee1f`` + * ``fe80::60a2:1691:e5a2:ee1f/126`` all IP addresses in this network. + * ``[ 192.168.0.1, fe80::/126 ]`` + +``retries`` : + Number of retry in case of an HTTP error. On each retry, SearXNG uses an + different proxy and source ip. + +``retry_on_http_error`` : + Retry request on some HTTP status code. + + Example: + + * ``true`` : on HTTP status code between 400 and 599. + * ``403`` : on HTTP status code 403. + * ``[403, 429]``: on HTTP status code 403 and 429. + +``enable_http2`` : + Enable by default. Set to ``false`` to disable HTTP/2. + +.. _httpx verification defaults: https://www.python-httpx.org/advanced/#changing-the-verification-defaults +.. _httpx ssl configuration: https://www.python-httpx.org/compatibility/#ssl-configuration + +``verify``: : ``$SSL_CERT_FILE``, ``$SSL_CERT_DIR`` + Allow to specify a path to certificate. + see `httpx verification defaults`_. + + In addition to ``verify``, SearXNG supports the ``$SSL_CERT_FILE`` (for a file) and + ``$SSL_CERT_DIR`` (for a directory) OpenSSL variables. + see `httpx ssl configuration`_. + +``max_redirects`` : + 30 by default. Maximum redirect before it is an error. + +``categories_as_tabs:`` +----------------------- + +A list of the categories that are displayed as tabs in the user interface. +Categories not listed here can still be searched with the :ref:`search-syntax`. + +.. code-block:: yaml + + categories_as_tabs: + general: + images: + videos: + news: + map: + music: + it: + science: + files: + social media: + +.. _settings engine: + +Engine settings +=============== + +.. sidebar:: Further reading .. + + - :ref:`configured engines` + - :ref:`engines-dev` + +In the code example below a *full fledged* example of a YAML setup from a dummy +engine is shown. Most of the options have a default value or even are optional. + +.. code:: yaml + + - name: example engine + engine: example + shortcut: demo + base_url: 'https://{language}.example.com/' + send_accept_language_header: false + categories: general + timeout: 3.0 + api_key: 'apikey' + disabled: false + language: en_US + tokens: [ 'my-secret-token' ] + weigth: 1 + display_error_messages: true + about: + website: https://example.com + wikidata_id: Q306656 + official_api_documentation: https://example.com/api-doc + use_official_api: true + require_api_key: true + results: HTML + enable_http: false + enable_http2: false + retries: 1 + retry_on_http_error: true # or 403 or [404, 429] + max_connections: 100 + max_keepalive_connections: 10 + keepalive_expiry: 5.0 + proxies: + http: + - http://proxy1:8080 + - http://proxy2:8080 + https: + - http://proxy1:8080 + - http://proxy2:8080 + - socks5://user:password@proxy3:1080 + - socks5h://user:password@proxy4:1080 + +``name`` : + Name that will be used across SearXNG to define this engine. In settings, on + the result page... + +``engine`` : + Name of the python file used to handle requests and responses to and from this + search engine. + +``shortcut`` : + Code used to execute bang requests (in this case using ``!bi``) + +``base_url`` : optional + Part of the URL that should be stable across every request. Can be useful to + use multiple sites using only one engine, or updating the site URL without + touching at the code. + +``send_accept_language_header`` : + Several engines that support languages (or regions) deal with the HTTP header + ``Accept-Language`` to build a response that fits to the locale. When this + option is activated, the language (locale) that is selected by the user is used + to build and send a ``Accept-Language`` header in the request to the origin + search engine. + +``categories`` : optional + Define in which categories this engine will be active. Most of the time, it is + defined in the code of the engine, but in a few cases it is useful, like when + describing multiple search engine using the same code. + +``timeout`` : optional + Timeout of the search with the current search engine. **Be careful, it will + modify the global timeout of SearXNG.** + +``api_key`` : optional + In a few cases, using an API needs the use of a secret key. How to obtain them + is described in the file. + +``disabled`` : optional + To disable by default the engine, but not deleting it. It will allow the user + to manually activate it in the settings. + +``language`` : optional + If you want to use another language for a specific engine, you can define it + by using the full ISO code of language and country, like ``fr_FR``, ``en_US``, + ``de_DE``. + +``tokens`` : optional + A list of secret tokens to make this engine *private*, more details see + :ref:`private engines`. + +``weigth`` : default ``1`` + Weighting of the results of this engine. + +``display_error_messages`` : default ``true`` + When an engine returns an error, the message is displayed on the user interface. + +``network`` : optional + Use the network configuration from another engine. + In addition, there are two default networks: + + - ``ipv4`` set ``local_addresses`` to ``0.0.0.0`` (use only IPv4 local addresses) + - ``ipv6`` set ``local_addresses`` to ``::`` (use only IPv6 local addresses) + +.. note:: + + A few more options are possible, but they are pretty specific to some + engines, and so won't be described here. + + +Example: Multilingual Search +---------------------------- + +SearXNG does not support true multilingual search. You have to use the language +prefix in your search query when searching in a different language. + +But there is a workaround: By adding a new search engine with a different +language, SearXNG will search in your default and other language. + +Example configuration in settings.yml for a German and English speaker: + +.. code-block:: yaml + + search: + default_lang : "de" + ... + + engines: + - name : google english + engine : google + language : en + ... + +When searching, the default google engine will return German results and +"google english" will return English results. + + +.. _settings use_default_settings: + +use_default_settings +==================== + +.. sidebar:: ``use_default_settings: true`` + + - :ref:`settings location` + - :ref:`use_default_settings.yml` + - :origin:`/etc/searxng/settings.yml ` + +The user defined ``settings.yml`` is loaded from the :ref:`settings location` +and can relied on the default configuration :origin:`searx/settings.yml` using: + + ``use_default_settings: true`` + +``server:`` + In the following example, the actual settings are the default settings defined + in :origin:`searx/settings.yml` with the exception of the ``secret_key`` and + the ``bind_address``: + + .. code-block:: yaml + + use_default_settings: true + server: + secret_key: "ultrasecretkey" # change this! + bind_address: "0.0.0.0" + +``engines:`` + With ``use_default_settings: true``, each settings can be override in a + similar way, the ``engines`` section is merged according to the engine + ``name``. In this example, SearXNG will load all the engine and the arch linux + wiki engine has a :ref:`token `: + + .. code-block:: yaml + + use_default_settings: true + server: + secret_key: "ultrasecretkey" # change this! + engines: + - name: arch linux wiki + tokens: ['$ecretValue'] + +``engines:`` / ``remove:`` + It is possible to remove some engines from the default settings. The following + example is similar to the above one, but SearXNG doesn't load the the google + engine: + + .. code-block:: yaml + + use_default_settings: + engines: + remove: + - google + server: + secret_key: "ultrasecretkey" # change this! + engines: + - name: arch linux wiki + tokens: ['$ecretValue'] + +``engines:`` / ``keep_only:`` + As an alternative, it is possible to specify the engines to keep. In the + following example, SearXNG has only two engines: + + .. code-block:: yaml + + use_default_settings: + engines: + keep_only: + - google + - duckduckgo + server: + secret_key: "ultrasecretkey" # change this! + engines: + - name: google + tokens: ['$ecretValue'] + - name: duckduckgo + tokens: ['$ecretValue'] diff --git a/_sources/admin/engines/sql-engines.rst.txt b/_sources/admin/engines/sql-engines.rst.txt new file mode 100644 index 00000000..3cd3c8f7 --- /dev/null +++ b/_sources/admin/engines/sql-engines.rst.txt @@ -0,0 +1,166 @@ +.. _sql engines: + +=========== +SQL Engines +=========== + +.. sidebar:: further read + + - `SQLite `_ + - `PostgreSQL `_ + - `MySQL `_ + +With the *SQL engines* you can bind SQL databases into SearXNG. The following +Relational Database Management System (RDBMS) are supported: + +- :ref:`engine sqlite` +- :ref:`engine postgresql` +- :ref:`engine mysql_server` + +All of the engines above are just commented out in the :origin:`settings.yml +`, as you have to set the required attributes for the +engines, e.g. ``database:`` ... + +.. code:: yaml + + - name: ... + engine: {sqlite|postgresql|mysql_server} + database: ... + result_template: {template_name} + query_str: ... + +By default, the engines use the ``key-value`` template for displaying results / +see :origin:`simple ` +theme. If you are not satisfied with the original result layout, you can use +your own template, set ``result_template`` attribute to ``{template_name}`` and +place the templates at:: + + searx/templates/{theme_name}/result_templates/{template_name} + +If you do not wish to expose these engines on a public instance, you can still +add them and limit the access by setting ``tokens`` as described in section +:ref:`private engines`. + + +Configure the engines +===================== + +The configuration of the new database engines are similar. You must put a valid +SQL-SELECT query in ``query_str``. At the moment you can only bind at most one +parameter in your query. By setting the attribute ``limit`` you can define how +many results you want from the SQL server. Basically, it is the same as the +``LIMIT`` keyword in SQL. + +Please, do not include ``LIMIT`` or ``OFFSET`` in your SQL query as the engines +rely on these keywords during paging. If you want to configure the number of +returned results use the option ``limit``. + +.. _engine sqlite: + +SQLite +------ + +.. sidebar:: info + + - :origin:`sqlite.py ` + +.. _MediathekView: https://mediathekview.de/ + +SQLite is a small, fast and reliable SQL database engine. It does not require +any extra dependency. To demonstrate the power of database engines, here is a +more complex example which reads from a MediathekView_ (DE) movie database. For +this example of the SQlite engine download the database: + +- https://liste.mediathekview.de/filmliste-v2.db.bz2 + +and unpack into ``searx/data/filmliste-v2.db``. To search the database use e.g +Query to test: ``!mediathekview concert`` + +.. code:: yaml + + - name: mediathekview + engine: sqlite + disabled: False + categories: general + result_template: default.html + database: searx/data/filmliste-v2.db + query_str: >- + SELECT title || ' (' || time(duration, 'unixepoch') || ')' AS title, + COALESCE( NULLIF(url_video_hd,''), NULLIF(url_video_sd,''), url_video) AS url, + description AS content + FROM film + WHERE title LIKE :wildcard OR description LIKE :wildcard + ORDER BY duration DESC + + +Extra Dependencies +------------------ + +For using :ref:`engine postgresql` or :ref:`engine mysql_server` you need to +install additional packages in Python's Virtual Environment of your SearXNG +instance. To switch into the environment (:ref:`searxng-src`) you can use +:ref:`searxng.sh`:: + + $ sudo utils/searxng.sh instance cmd bash + (searxng-pyenv)$ pip install ... + + +.. _engine postgresql: + +PostgreSQL +---------- + +.. _psycopg2: https://www.psycopg.org/install + +.. sidebar:: info + + - :origin:`postgresql.py ` + - ``pip install`` psycopg2_ + +PostgreSQL is a powerful and robust open source database. Before configuring +the PostgreSQL engine, you must install the dependency ``psychopg2``. You can +find an example configuration below: + +.. code:: yaml + + - name: my_database + engine: postgresql + database: my_database + username: searxng + password: password + query_str: 'SELECT * from my_table WHERE my_column = %(query)s' + +.. _engine mysql_server: + +MySQL +----- + +.. sidebar:: info + + - :origin:`mysql_server.py ` + - ``pip install`` :pypi:`mysql-connector-python ` + +MySQL is said to be the most popular open source database. Before enabling MySQL +engine, you must install the package ``mysql-connector-python``. + +The authentication plugin is configurable by setting ``auth_plugin`` in the +attributes. By default it is set to ``caching_sha2_password``. This is an +example configuration for querying a MySQL server: + +.. code:: yaml + + - name: my_database + engine: mysql_server + database: my_database + username: searxng + password: password + limit: 5 + query_str: 'SELECT * from my_table WHERE my_column=%(query)s' + + +Acknowledgment +============== + +This development was sponsored by `Search and Discovery Fund +`_ of `NLnet Foundation `_. + diff --git a/_sources/admin/index.rst.txt b/_sources/admin/index.rst.txt new file mode 100644 index 00000000..70516730 --- /dev/null +++ b/_sources/admin/index.rst.txt @@ -0,0 +1,21 @@ +=========================== +Administrator documentation +=========================== + +.. toctree:: + :maxdepth: 2 + :caption: Contents + + installation + installation-docker + installation-scripts + installation-searxng + installation-uwsgi + installation-nginx + installation-apache + update-searxng + engines/index + api + architecture + plugins + buildhosts diff --git a/_sources/admin/installation-apache.rst.txt b/_sources/admin/installation-apache.rst.txt new file mode 100644 index 00000000..673a37ee --- /dev/null +++ b/_sources/admin/installation-apache.rst.txt @@ -0,0 +1,388 @@ +.. _installation apache: + +====== +Apache +====== + +.. _Apache: https://httpd.apache.org/ +.. _Apache Debian: + https://cwiki.apache.org/confluence/display/HTTPD/DistrosDefaultLayout#DistrosDefaultLayout-Debian,Ubuntu(Apachehttpd2.x): +.. _apache2.README.Debian: + https://salsa.debian.org/apache-team/apache2/raw/master/debian/apache2.README.Debian +.. _Apache Arch Linux: + https://wiki.archlinux.org/index.php/Apache_HTTP_Server +.. _Apache Fedora: + https://docs.fedoraproject.org/en-US/quick-docs/getting-started-with-apache-http-server/index.html +.. _Apache directives: + https://httpd.apache.org/docs/trunk/mod/directives.html +.. _Getting Started: + https://httpd.apache.org/docs/current/en/getting-started.html +.. _Terms Used to Describe Directives: + https://httpd.apache.org/docs/current/en/mod/directive-dict.html +.. _Configuration Files: + https://httpd.apache.org/docs/current/en/configuring.html +.. _ProxyPreserveHost: https://httpd.apache.org/docs/trunk/mod/mod_proxy.html#proxypreservehost +.. _LoadModule: + https://httpd.apache.org/docs/mod/mod_so.html#loadmodule +.. _IncludeOptional: + https://httpd.apache.org/docs/mod/core.html#includeoptional +.. _DocumentRoot: + https://httpd.apache.org/docs/trunk/mod/core.html#documentroot +.. _Location: + https://httpd.apache.org/docs/trunk/mod/core.html#location +.. _uWSGI Apache support: + https://uwsgi-docs.readthedocs.io/en/latest/Apache.html +.. _mod_proxy_uwsgi: + https://uwsgi-docs.readthedocs.io/en/latest/Apache.html#mod-proxy-uwsgi +.. _mod_proxy_http: + https://httpd.apache.org/docs/current/mod/mod_proxy_http.html +.. _mod_proxy: + https://httpd.apache.org/docs/current/mod/mod_proxy.html + + +This section explains how to set up a SearXNG instance using the HTTP server Apache_. +If you did use the :ref:`installation scripts` and do not have any special preferences +you can install the :ref:`SearXNG site ` using +:ref:`searxng.sh `: + +.. code:: bash + + $ sudo -H ./utils/searxng.sh install apache + +If you have special interests or problems with setting up Apache, the following +section might give you some guidance. + + +.. sidebar:: further read + + - `Apache Arch Linux`_ + - `Apache Debian`_ + - `apache2.README.Debian`_ + - `Apache Fedora`_ + - `Apache directives`_ + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + + +The Apache HTTP server +====================== + +If Apache_ is not installed, install it now. If apache_ is new to you, the +`Getting Started`_, `Configuration Files`_ and `Terms Used to Describe +Directives`_ documentation gives first orientation. There is also a list of +`Apache directives`_ *to keep in the pocket*. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: bash + + sudo -H apt-get install apache2 + + .. group-tab:: Arch Linux + + .. code:: bash + + sudo -H pacman -S apache + sudo -H systemctl enable httpd + sudo -H systemctl start http + + .. group-tab:: Fedora / RHEL + + .. code:: bash + + sudo -H dnf install httpd + sudo -H systemctl enable httpd + sudo -H systemctl start httpd + +Now at http://localhost you should see some kind of *Welcome* or *Test* page. +How this default site is configured, depends on the linux distribution +(compare `Apache directives`_). + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: bash + + less /etc/apache2/sites-enabled/000-default.conf + + In this file, there is a line setting the `DocumentRoot`_ directive: + + .. code:: apache + + DocumentRoot /var/www/html + + And the *welcome* page is the HTML file at ``/var/www/html/index.html``. + + .. group-tab:: Arch Linux + + .. code:: bash + + less /etc/httpd/conf/httpd.conf + + In this file, there is a line setting the `DocumentRoot`_ directive: + + .. code:: apache + + DocumentRoot "/srv/http" + + Options Indexes FollowSymLinks + AllowOverride None + Require all granted + + + The *welcome* page of Arch Linux is a page showing the directory located + at ``DocumentRoot``. This *directory* page is generated by the Module + `mod_autoindex `_: + + .. code:: apache + + LoadModule autoindex_module modules/mod_autoindex.so + ... + Include conf/extra/httpd-autoindex.conf + + .. group-tab:: Fedora / RHEL + + .. code:: bash + + less /etc/httpd/conf/httpd.conf + + In this file, there is a line setting the ``DocumentRoot`` directive: + + .. code:: apache + + DocumentRoot "/var/www/html" + ... + + AllowOverride None + # Allow open access: + Require all granted + + + On fresh installations, the ``/var/www`` is empty and the *default + welcome page* is shown, the configuration is located at:: + + less /etc/httpd/conf.d/welcome.conf + + +.. _Debian's Apache layout: + +Debian's Apache layout +---------------------- + +Be aware, Debian's Apache layout is quite different from the standard Apache +configuration. For details look at the apache2.README.Debian_ +(``/usr/share/doc/apache2/README.Debian.gz``). Some commands you should know on +Debian: + +* :man:`apache2ctl`: Apache HTTP server control interface +* :man:`a2enmod`, :man:`a2dismod`: switch on/off modules +* :man:`a2enconf`, :man:`a2disconf`: switch on/off configurations +* :man:`a2ensite`, :man:`a2dissite`: switch on/off sites + +.. _apache modules: + +Apache modules +-------------- + +To load additional modules, in most distributions you have to un-comment the +lines with the corresponding LoadModule_ directive, except in :ref:`Debian's +Apache layout`. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + :ref:`Debian's Apache layout` uses :man:`a2enmod` and :man:`a2dismod` to + activate or disable modules: + + .. code:: bash + + sudo -H a2enmod ssl + sudo -H a2enmod headers + sudo -H a2enmod proxy + sudo -H a2enmod proxy_http + sudo -H a2enmod proxy_uwsgi + + .. group-tab:: Arch Linux + + In the ``/etc/httpd/conf/httpd.conf`` file, activate LoadModule_ + directives: + + .. code:: apache + + LoadModule ssl_module modules/mod_ssl.so + LoadModule headers_module modules/mod_headers.so + LoadModule proxy_module modules/mod_proxy.so + LoadModule proxy_http_module modules/mod_proxy_http.so + LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so + + .. group-tab:: Fedora / RHEL + + In the ``/etc/httpd/conf/httpd.conf`` file, activate LoadModule_ + directives: + + .. code:: apache + + LoadModule ssl_module modules/mod_ssl.so + LoadModule headers_module modules/mod_headers.so + LoadModule proxy_module modules/mod_proxy.so + LoadModule proxy_http_module modules/mod_proxy_http.so + LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so + + +.. _apache sites: + +Apache sites +------------ + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + In :ref:`Debian's Apache layout` you create a ``searxng.conf`` with the + ```` directive and save this file in the *sites + available* folder at ``/etc/apache2/sites-available``. To enable the + ``searxng.conf`` use :man:`a2ensite`: + + .. code:: bash + + sudo -H a2ensite searxng.conf + + .. group-tab:: Arch Linux + + In the ``/etc/httpd/conf/httpd.conf`` file add a IncludeOptional_ + directive: + + .. code:: apache + + IncludeOptional sites-enabled/*.conf + + Create two folders, one for the *available sites* and one for the *enabled sites*: + + .. code:: bash + + mkdir -p /etc/httpd/sites-available + mkdir -p /etc/httpd/sites-enabled + + Create configuration at ``/etc/httpd/sites-available`` and place a + symlink to ``sites-enabled``: + + .. code:: bash + + sudo -H ln -s /etc/httpd/sites-available/searxng.conf \ + /etc/httpd/sites-enabled/searxng.conf + + .. group-tab:: Fedora / RHEL + + In the ``/etc/httpd/conf/httpd.conf`` file add a IncludeOptional_ + directive: + + .. code:: apache + + IncludeOptional sites-enabled/*.conf + + Create two folders, one for the *available sites* and one for the *enabled sites*: + + .. code:: bash + + mkdir -p /etc/httpd/sites-available + mkdir -p /etc/httpd/sites-enabled + + Create configuration at ``/etc/httpd/sites-available`` and place a + symlink to ``sites-enabled``: + + .. code:: bash + + sudo -H ln -s /etc/httpd/sites-available/searxng.conf \ + /etc/httpd/sites-enabled/searxng.conf + + +.. _apache searxng site: + +Apache's SearXNG site +===================== + +.. _mod_uwsgi: https://uwsgi-docs.readthedocs.io/en/latest/Apache.html#mod-uwsgi + +.. sidebar:: uWSGI + + Use mod_proxy_uwsgi_ / don't use the old mod_uwsgi_ anymore. + +To proxy the incoming requests to the SearXNG instance Apache needs the +mod_proxy_ module (:ref:`apache modules`). + +.. sidebar:: HTTP headers + + With ProxyPreserveHost_ the incoming ``Host`` header is passed to the proxied + host. + +Depending on what your SearXNG installation is listening on, you need a http +mod_proxy_http_) or socket (mod_proxy_uwsgi_) communication to upstream. + +The :ref:`installation scripts` installs the :ref:`reference setup +` and a :ref:`uwsgi setup` that listens on a socket by default. +You can install and activate your own ``searxng.conf`` like shown in +:ref:`apache sites`. + +.. tabs:: + + .. group-tab:: socket + + .. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START apache socket + :end-before: END apache socket + + .. group-tab:: http + + .. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START apache http + :end-before: END apache http + +.. _restart apache: + +Restart service: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: bash + + sudo -H systemctl restart apache2 + sudo -H service uwsgi restart searxng + + .. group-tab:: Arch Linux + + .. code:: bash + + sudo -H systemctl restart httpd + sudo -H systemctl restart uwsgi@searxng + + .. group-tab:: Fedora / RHEL + + .. code:: bash + + sudo -H systemctl restart httpd + sudo -H touch /etc/uwsgi.d/searxng.ini + + +disable logs +============ + +For better privacy you can disable Apache logs. In the examples above activate +one of the lines and `restart apache`_: + +.. code:: apache + + SetEnvIf Request_URI "/searxng" dontlog + # CustomLog /dev/null combined env=dontlog + +The ``CustomLog`` directive disables logs for the entire (virtual) server, use it +when the URL of the service does not have a path component (``/searxng``), so when +SearXNG is located at root (``/``). diff --git a/_sources/admin/installation-docker.rst.txt b/_sources/admin/installation-docker.rst.txt new file mode 100644 index 00000000..1457d6a6 --- /dev/null +++ b/_sources/admin/installation-docker.rst.txt @@ -0,0 +1,194 @@ +.. _installation docker: + +================ +Docker Container +================ + +.. _ENTRYPOINT: https://docs.docker.com/engine/reference/builder/#entrypoint +.. _searxng/searxng @dockerhub: https://hub.docker.com/r/searxng/searxng +.. _searxng-docker: https://github.com/searxng/searxng-docker +.. _[caddy]: https://hub.docker.com/_/caddy +.. _Redis: https://redis.io/ + +---- + +.. sidebar:: info + + - `searxng/searxng @dockerhub`_ + - :origin:`Dockerfile` + - `Docker overview `_ + - `Docker Cheat Sheet `_ + - `Alpine Linux `_ + `(wiki) `__ + `apt packages `_ + - Alpine's ``/bin/sh`` is :man:`dash` + +**If you intend to create a public instance using Docker, use our well maintained +docker container** + +- `searxng/searxng @dockerhub`_. + +.. sidebar:: hint + + The rest of this article is of interest only to those who want to create and + maintain their own Docker images. + +The sources are hosted at searxng-docker_ and the container includes: + +- a HTTPS reverse proxy `[caddy]`_ and +- a Redis_ DB + +The `default SearXNG setup `_ +of this container: + +- enables :ref:`limiter ` to protect against bots +- enables :ref:`image proxy ` for better privacy +- enables :ref:`cache busting ` to save bandwith + +---- + + +Get Docker +========== + +If you plan to build and maintain a docker image by yourself, make sure you have +`Docker installed `_. On Linux don't +forget to add your user to the docker group (log out and log back in so that +your group membership is re-evaluated): + +.. code:: sh + + $ sudo usermod -a -G docker $USER + + +searxng/searxng +=============== + +.. sidebar:: ``docker run`` + + - `-\-rm `__ + automatically clean up when container exits + - `-d `__ start + detached container + - `-v `__ + mount volume ``HOST:CONTAINER`` + +The docker image is based on :origin:`Dockerfile` and available from +`searxng/searxng @dockerhub`_. Using the docker image is quite easy, for +instance you can pull the `searxng/searxng @dockerhub`_ image and deploy a local +instance using `docker run `_: + +.. code:: sh + + $ mkdir my-instance + $ cd my-instance + $ export PORT=8080 + $ docker pull searxng/searxng + $ docker run --rm \ + -d -p ${PORT}:8080 \ + -v "${PWD}/searxng:/etc/searxng" \ + -e "BASE_URL=http://localhost:$PORT/" \ + -e "INSTANCE_NAME=my-instance" \ + searxng/searxng + 2f998.... # container's ID + +Open your WEB browser and visit the URL: + +.. code:: sh + + $ xdg-open "http://localhost:$PORT" + +Inside ``${PWD}/searxng``, you will find ``settings.yml`` and ``uwsgi.ini``. You +can modify these files according to your needs and restart the Docker image. + +.. code:: sh + + $ docker container restart 2f998 + +Use command ``container ls`` to list running containers, add flag `-a +`__ to list +exited containers also. With ``container stop`` a running container can be +stoped. To get rid of a container use ``container rm``: + +.. code:: sh + + $ docker container ls + CONTAINER ID IMAGE COMMAND CREATED ... + 2f998d725993 searxng/searxng "/sbin/tini -- /usr/…" 7 minutes ago ... + + $ docker container stop 2f998 + $ docker container rm 2f998 + +.. sidebar:: Warning + + This might remove all docker items, not only those from SearXNG. + +If you won't use docker anymore and want to get rid of all conatiners & images +use the following *prune* command: + +.. code:: sh + + $ docker stop $(docker ps -aq) # stop all containers + $ docker system prune # make some housekeeping + $ docker rmi -f $(docker images -q) # drop all images + + +shell inside container +---------------------- + +.. sidebar:: Bashism + + - `A tale of two shells: bash or dash `_ + - `How to make bash scripts work in dash `_ + - `Checking for Bashisms `_ + +Like in many other distributions, Alpine's `/bin/sh +`__ is :man:`dash`. Dash is meant to be +`POSIX-compliant `__. +Compared to debian, in the Alpine image :man:`bash` is not installed. The +:origin:`dockerfiles/docker-entrypoint.sh` script is checked *against dash* +(``make tests.shell``). + +To open a shell inside the container: + +.. code:: sh + + $ docker exec -it 2f998 sh + + +Build the image +=============== + +It's also possible to build SearXNG from the embedded :origin:`Dockerfile`:: + + $ git clone https://github.com/searxng/searxng.git + $ cd searxng + $ make docker.build + ... + Successfully built 49586c016434 + Successfully tagged searxng/searxng:latest + Successfully tagged searxng/searxng:1.0.0-209-9c823800-dirty + + $ docker images + REPOSITORY TAG IMAGE ID CREATED SIZE + searxng/searxng 1.0.0-209-9c823800-dirty 49586c016434 13 minutes ago 308MB + searxng/searxng latest 49586c016434 13 minutes ago 308MB + alpine 3.13 6dbb9cc54074 3 weeks ago 5.61MB + + +Command line +============ + +.. sidebar:: docker run + + Use flags ``-it`` for `interactive processes + `__. + +In the :origin:`Dockerfile` the ENTRYPOINT_ is defined as +:origin:`dockerfiles/docker-entrypoint.sh` + +.. code:: sh + + docker run --rm -it searxng/searxng -h + +.. program-output:: ../dockerfiles/docker-entrypoint.sh -h diff --git a/_sources/admin/installation-nginx.rst.txt b/_sources/admin/installation-nginx.rst.txt new file mode 100644 index 00000000..8e529958 --- /dev/null +++ b/_sources/admin/installation-nginx.rst.txt @@ -0,0 +1,252 @@ +.. _installation nginx: + +===== +NGINX +===== + +.. _nginx: + https://docs.nginx.com/nginx/admin-guide/ +.. _nginx server configuration: + https://docs.nginx.com/nginx/admin-guide/web-server/web-server/#setting-up-virtual-servers +.. _nginx beginners guide: + https://nginx.org/en/docs/beginners_guide.html +.. _Getting Started wiki: + https://www.nginx.com/resources/wiki/start/ +.. _uWSGI support from nginx: + https://uwsgi-docs.readthedocs.io/en/latest/Nginx.html +.. _uwsgi_params: + https://uwsgi-docs.readthedocs.io/en/latest/Nginx.html#configuring-nginx +.. _SCRIPT_NAME: + https://werkzeug.palletsprojects.com/en/1.0.x/wsgi/#werkzeug.wsgi.get_script_name + +This section explains how to set up a SearXNG instance using the HTTP server nginx_. +If you have used the :ref:`installation scripts` and do not have any special preferences +you can install the :ref:`SearXNG site ` using +:ref:`searxng.sh `: + +.. code:: bash + + $ sudo -H ./utils/searxng.sh install nginx + +If you have special interests or problems with setting up nginx, the following +section might give you some guidance. + + +.. sidebar:: further reading + + - nginx_ + - `nginx beginners guide`_ + - `nginx server configuration`_ + - `Getting Started wiki`_ + - `uWSGI support from nginx`_ + + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + + +The nginx HTTP server +===================== + +If nginx_ is not installed, install it now. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: bash + + sudo -H apt-get install nginx + + .. group-tab:: Arch Linux + + .. code-block:: sh + + sudo -H pacman -S nginx-mainline + sudo -H systemctl enable nginx + sudo -H systemctl start nginx + + .. group-tab:: Fedora / RHEL + + .. code-block:: sh + + sudo -H dnf install nginx + sudo -H systemctl enable nginx + sudo -H systemctl start nginx + +Now at http://localhost you should see a *Welcome to nginx!* page, on Fedora you +see a *Fedora Webserver - Test Page*. The test page comes from the default +`nginx server configuration`_. How this default site is configured, +depends on the linux distribution: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: bash + + less /etc/nginx/nginx.conf + + There is one line that includes site configurations from: + + .. code:: nginx + + include /etc/nginx/sites-enabled/*; + + .. group-tab:: Arch Linux + + .. code-block:: sh + + less /etc/nginx/nginx.conf + + There is a configuration section named ``server``: + + .. code-block:: nginx + + server { + listen 80; + server_name localhost; + # ... + } + + .. group-tab:: Fedora / RHEL + + .. code-block:: sh + + less /etc/nginx/nginx.conf + + There is one line that includes site configurations from: + + .. code:: nginx + + include /etc/nginx/conf.d/*.conf; + + +.. _nginx searxng site: + +NGINX's SearXNG site +==================== + +Now you have to create a configuration file (``searxng.conf``) for the SearXNG +site. If nginx_ is new to you, the `nginx beginners guide`_ is a good starting +point and the `Getting Started wiki`_ is always a good resource *to keep in the +pocket*. + +Depending on what your SearXNG installation is listening on, you need a http or socket +communication to upstream. + +.. tabs:: + + .. group-tab:: socket + + .. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START nginx socket + :end-before: END nginx socket + + .. group-tab:: http + + .. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START nginx http + :end-before: END nginx http + +The :ref:`installation scripts` installs the :ref:`reference setup +` and a :ref:`uwsgi setup` that listens on a socket by default. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + Create configuration at ``/etc/nginx/sites-available/`` and place a + symlink to ``sites-enabled``: + + .. code:: bash + + sudo -H ln -s /etc/nginx/sites-available/searxng.conf \ + /etc/nginx/sites-enabled/searxng.conf + + .. group-tab:: Arch Linux + + In the ``/etc/nginx/nginx.conf`` file, in the ``server`` section add a + `include `_ + directive: + + .. code:: nginx + + server { + # ... + include /etc/nginx/default.d/*.conf; + # ... + } + + Create two folders, one for the *available sites* and one for the *enabled sites*: + + .. code:: bash + + mkdir -p /etc/nginx/default.d + mkdir -p /etc/nginx/default.apps-available + + Create configuration at ``/etc/nginx/default.apps-available`` and place a + symlink to ``default.d``: + + .. code:: bash + + sudo -H ln -s /etc/nginx/default.apps-available/searxng.conf \ + /etc/nginx/default.d/searxng.conf + + .. group-tab:: Fedora / RHEL + + Create a folder for the *available sites*: + + .. code:: bash + + mkdir -p /etc/nginx/default.apps-available + + Create configuration at ``/etc/nginx/default.apps-available`` and place a + symlink to ``conf.d``: + + .. code:: bash + + sudo -H ln -s /etc/nginx/default.apps-available/searxng.conf \ + /etc/nginx/conf.d/searxng.conf + +Restart services: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: bash + + sudo -H systemctl restart nginx + sudo -H service uwsgi restart searxng + + .. group-tab:: Arch Linux + + .. code:: bash + + sudo -H systemctl restart nginx + sudo -H systemctl restart uwsgi@searxng + + .. group-tab:: Fedora / RHEL + + .. code:: bash + + sudo -H systemctl restart nginx + sudo -H touch /etc/uwsgi.d/searxng.ini + + +Disable logs +============ + +For better privacy you can disable nginx logs in ``/etc/nginx/nginx.conf``. + +.. code:: nginx + + http { + # ... + access_log /dev/null; + error_log /dev/null; + # ... + } diff --git a/_sources/admin/installation-scripts.rst.txt b/_sources/admin/installation-scripts.rst.txt new file mode 100644 index 00000000..e256a246 --- /dev/null +++ b/_sources/admin/installation-scripts.rst.txt @@ -0,0 +1,62 @@ +.. _installation scripts: + +=================== +Installation Script +=================== + +.. sidebar:: Update the OS first! + + To avoid unwanted side effects, update your OS before installing SearXNG. + +The following will install a setup as shown in :ref:`the reference architecture +`. First you need to get a clone of the repository. The clone is only needed for +the installation procedure and some maintenance tasks. + +.. sidebar:: further read + + - :ref:`toolboxing` + +Jump to a folder that is readable by *others* and start to clone SearXNG, +alternatively you can create your own fork and clone from there. + +.. code:: bash + + $ cd ~/Downloads + $ git clone https://github.com/searxng/searxng.git searxng + $ cd searxng + +.. sidebar:: further read + + - :ref:`inspect searxng` + +To install a SearXNG :ref:`reference setup ` +including a :ref:`uWSGI setup ` as described in the +:ref:`installation basic` and in the :ref:`searxng uwsgi` section type: + +.. code:: bash + + $ sudo -H ./utils/searxng.sh install all + +.. attention:: + + For the installation procedure, use a *sudoer* login to run the scripts. If + you install from ``root``, take into account that the scripts are creating a + ``searxng`` user. In the installation procedure this new created user does + need read access to the cloned SearXNG repository, which is not the case if you clone + it into a folder below ``/root``! + +.. sidebar:: further read + + - :ref:`update searxng` + +.. _caddy: https://hub.docker.com/_/caddy + +When all services are installed and running fine, you can add SearXNG to your +HTTP server. We do not have any preferences for the HTTP server, you can use +whatever you prefer. + +We use caddy in our :ref:`docker image ` and we have +implemented installation procedures for: + +- :ref:`installation nginx` +- :ref:`installation apache` diff --git a/_sources/admin/installation-searxng.rst.txt b/_sources/admin/installation-searxng.rst.txt new file mode 100644 index 00000000..9152784f --- /dev/null +++ b/_sources/admin/installation-searxng.rst.txt @@ -0,0 +1,132 @@ +.. _installation basic: + +========================= +Step by step installation +========================= + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + + +In this section we show the setup of a SearXNG instance that will be installed +by the :ref:`installation scripts`. + +.. _install packages: + +Install packages +================ + +.. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START distro-packages + :end-before: END distro-packages + +.. hint:: + + This installs also the packages needed by :ref:`searxng uwsgi` + +.. _create searxng user: + +Create user +=========== + +.. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START create user + :end-before: END create user + +.. _searxng-src: + +Install SearXNG & dependencies +============================== + +Start a interactive shell from new created user and clone SearXNG: + +.. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START clone searxng + :end-before: END clone searxng + +In the same shell create *virtualenv*: + +.. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START create virtualenv + :end-before: END create virtualenv + +To install SearXNG's dependencies, exit the SearXNG *bash* session you opened above +and start a new one. Before installing, check if your *virtualenv* was sourced +from the login (*~/.profile*): + +.. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START manage.sh update_packages + :end-before: END manage.sh update_packages + +.. tip:: + + Open a second terminal for the configuration tasks and leave the ``(searx)$`` + terminal open for the tasks below. + + +.. _use_default_settings.yml: + +Configuration +============= + +.. sidebar:: ``use_default_settings: True`` + + - :ref:`settings global` + - :ref:`settings location` + - :ref:`settings use_default_settings` + - :origin:`/etc/searxng/settings.yml ` + +To create a initial ``/etc/searxng/settings.yml`` we recommend to start with a +copy of the file :origin:`utils/templates/etc/searxng/settings.yml`. This setup +:ref:`use default settings ` from +:origin:`searx/settings.yml` and is shown in the tab *"Use default settings"* +below. This setup: + +- enables :ref:`limiter ` to protect against bots +- enables :ref:`image proxy ` for better privacy +- enables :ref:`cache busting ` to save bandwith + +Modify the ``/etc/searxng/settings.yml`` to your needs: + +.. tabs:: + + .. group-tab:: Use default settings + + .. literalinclude:: ../../utils/templates/etc/searxng/settings.yml + :language: yaml + :end-before: # hostname_replace: + + To see the entire file jump to :origin:`utils/templates/etc/searxng/settings.yml` + + .. group-tab:: searx/settings.yml + + .. literalinclude:: ../../searx/settings.yml + :language: yaml + :end-before: # hostname_replace: + + To see the entire file jump to :origin:`searx/settings.yml` + +For a *minimal setup* you need to set ``server:secret_key``. + +.. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START searxng config + :end-before: END searxng config + + +Check +===== + +To check your SearXNG setup, optional enable debugging and start the *webapp*. +SearXNG looks at the exported environment ``$SEARXNG_SETTINGS_PATH`` for a +configuration file. + +.. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START check searxng installation + :end-before: END check searxng installation + +If everything works fine, hit ``[CTRL-C]`` to stop the *webapp* and disable the +debug option in ``settings.yml``. You can now exit SearXNG user bash session (enter exit +command twice). At this point SearXNG is not demonized; uwsgi allows this. + diff --git a/_sources/admin/installation-uwsgi.rst.txt b/_sources/admin/installation-uwsgi.rst.txt new file mode 100644 index 00000000..e888d067 --- /dev/null +++ b/_sources/admin/installation-uwsgi.rst.txt @@ -0,0 +1,268 @@ +.. _searxng uwsgi: + +===== +uWSGI +===== + +.. sidebar:: further reading + + - `systemd.unit`_ + - `uWSGI Emperor`_ + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + + +.. _systemd.unit: https://www.freedesktop.org/software/systemd/man/systemd.unit.html +.. _One service per app in systemd: + https://uwsgi-docs.readthedocs.io/en/latest/Systemd.html#one-service-per-app-in-systemd +.. _uWSGI Emperor: + https://uwsgi-docs.readthedocs.io/en/latest/Emperor.html +.. _uwsgi ini file: + https://uwsgi-docs.readthedocs.io/en/latest/Configuration.html#ini-files +.. _systemd unit template: + http://0pointer.de/blog/projects/instances.html + + +Origin uWSGI +============ + +.. _Tyrant mode: + https://uwsgi-docs.readthedocs.io/en/latest/Emperor.html#tyrant-mode-secure-multi-user-hosting + +How uWSGI is implemented by distributors varies. The uWSGI project itself +recommends two methods: + +1. `systemd.unit`_ template file as described here `One service per app in systemd`_: + + There is one `systemd unit template`_ on the system installed and one `uwsgi + ini file`_ per uWSGI-app placed at dedicated locations. Take archlinux and a + ``searxng.ini`` as example:: + + systemd template unit: /usr/lib/systemd/system/uwsgi@.service + contains: [Service] + ExecStart=/usr/bin/uwsgi --ini /etc/uwsgi/%I.ini + + SearXNG application: /etc/uwsgi/searxng.ini + links to: /etc/uwsgi/apps-available/searxng.ini + + The SearXNG app (template ``/etc/uwsgi/%I.ini``) can be maintained as known + from common systemd units: + + .. code:: sh + + $ systemctl enable uwsgi@searxng + $ systemctl start uwsgi@searxng + $ systemctl restart uwsgi@searxng + $ systemctl stop uwsgi@searxng + +2. The `uWSGI Emperor`_ which fits for maintaining a large range of uwsgi + apps and there is a `Tyrant mode`_ to secure multi-user hosting. + + The Emperor mode is a special uWSGI instance that will monitor specific + events. The Emperor mode (the service) is started by a (common, not template) + systemd unit. + + The Emperor service will scan specific directories for `uwsgi ini file`_\s + (also know as *vassals*). If a *vassal* is added, removed or the timestamp is + modified, a corresponding action takes place: a new uWSGI instance is started, + reload or stopped. Take Fedora and a ``searxng.ini`` as example:: + + to install & start SearXNG instance create --> /etc/uwsgi.d/searxng.ini + to reload the instance edit timestamp --> touch /etc/uwsgi.d/searxng.ini + to stop instance remove ini --> rm /etc/uwsgi.d/searxng.ini + + +Distributors +============ + +The `uWSGI Emperor`_ mode and `systemd unit template`_ is what the distributors +mostly offer their users, even if they differ in the way they implement both +modes and their defaults. Another point they might differ in is the packaging of +plugins (if so, compare :ref:`install packages`) and what the default python +interpreter is (python2 vs. python3). + +While archlinux does not start a uWSGI service by default, Fedora (RHEL) starts +a Emperor in `Tyrant mode`_ by default (you should have read :ref:`uWSGI Tyrant +mode pitfalls`). Worth to know; debian (ubuntu) follow a complete different +approach, read see :ref:`Debian's uWSGI layout`. + +.. _Debian's uWSGI layout: + +Debian's uWSGI layout +--------------------- + +.. _uwsgi.README.Debian: + https://salsa.debian.org/uwsgi-team/uwsgi/-/raw/debian/latest/debian/uwsgi.README.Debian + +Be aware, Debian's uWSGI layout is quite different from the standard uWSGI +configuration. Your are familiar with :ref:`Debian's Apache layout`? .. they do a +similar thing for the uWSGI infrastructure. The folders are:: + + /etc/uwsgi/apps-available/ + /etc/uwsgi/apps-enabled/ + +The `uwsgi ini file`_ is enabled by a symbolic link:: + + ln -s /etc/uwsgi/apps-available/searxng.ini /etc/uwsgi/apps-enabled/ + +More details can be found in the uwsgi.README.Debian_ +(``/usr/share/doc/uwsgi/README.Debian.gz``). Some commands you should know on +Debian: + +.. code:: none + + Commands recognized by init.d script + ==================================== + + You can issue to init.d script following commands: + * start | starts daemon + * stop | stops daemon + * reload | sends to daemon SIGHUP signal + * force-reload | sends to daemon SIGTERM signal + * restart | issues 'stop', then 'start' commands + * status | shows status of daemon instance (running/not running) + + 'status' command must be issued with exactly one argument: ''. + + Controlling specific instances of uWSGI + ======================================= + + You could control specific instance(s) by issuing: + + SYSTEMCTL_SKIP_REDIRECT=1 service uwsgi ... + + where: + * is one of 'start', 'stop' etc. + * is the name of configuration file (without extension) + + For example, this is how instance for /etc/uwsgi/apps-enabled/hello.xml is + started: + + SYSTEMCTL_SKIP_REDIRECT=1 service uwsgi start hello + + +.. _uWSGI maintenance: + +uWSGI maintenance +================= + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START searxng uwsgi-description ubuntu-20.04 + :end-before: END searxng uwsgi-description ubuntu-20.04 + + .. hotfix: a bug group-tab need this comment + + .. group-tab:: Arch Linux + + .. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START searxng uwsgi-description arch + :end-before: END searxng uwsgi-description arch + + .. hotfix: a bug group-tab need this comment + + .. group-tab:: Fedora / RHEL + + .. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START searxng uwsgi-description fedora + :end-before: END searxng uwsgi-description fedora + + +.. _uwsgi setup: + +uWSGI setup +=========== + +Create the configuration ini-file according to your distribution and restart the +uwsgi application. As shown below, the :ref:`installation scripts` installs by +default: + +- a uWSGI setup that listens on a socket and +- enables :ref:`cache busting `. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START searxng uwsgi-appini ubuntu-20.04 + :end-before: END searxng uwsgi-appini ubuntu-20.04 + + .. hotfix: a bug group-tab need this comment + + .. group-tab:: Arch Linux + + .. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START searxng uwsgi-appini arch + :end-before: END searxng uwsgi-appini arch + + .. hotfix: a bug group-tab need this comment + + .. group-tab:: Fedora / RHEL + + .. kernel-include:: $DOCS_BUILD/includes/searxng.rst + :start-after: START searxng uwsgi-appini fedora + :end-before: END searxng uwsgi-appini fedora + + +.. _uWSGI Tyrant mode pitfalls: + +Pitfalls of the Tyrant mode +=========================== + +The implementation of the process owners and groups in the `Tyrant mode`_ is +somewhat unusual and requires special consideration. In `Tyrant mode`_ mode the +Emperor will run the vassal using the UID/GID of the vassal configuration file +(user and group of the app ``.ini`` file). + +.. _#2099@uWSGI: https://github.com/unbit/uwsgi/issues/2099 +.. _#752@uWSGI: https://github.com/unbit/uwsgi/pull/752 +.. _#2425uWSGI: https://github.com/unbit/uwsgi/issues/2425 + +Without option ``emperor-tyrant-initgroups=true`` in ``/etc/uwsgi.ini`` the +process won't get the additional groups, but this option is not available in +2.0.x branch (see `#2099@uWSGI`_) the feature `#752@uWSGI`_ has been merged (on +Oct. 2014) to the master branch of uWSGI but had never been released; the last +major release is from Dec. 2013, since the there had been only bugfix releases +(see `#2425uWSGI`_). To shorten up: + + **In Tyrant mode, there is no way to get additional groups, and the uWSGI + process misses additional permissions that may be needed.** + +For example on Fedora (RHEL): If you try to install a redis DB with socket +communication and you want to connect to it from the SearXNG uWSGI, you will see a +*Permission denied* in the log of your instance:: + + ERROR:searx.redisdb: [searxng (993)] can't connect redis DB ... + ERROR:searx.redisdb: Error 13 connecting to unix socket: /usr/local/searxng-redis/run/redis.sock. Permission denied. + ERROR:searx.plugins.limiter: init limiter DB failed!!! + +Even if your *searxng* user of the uWSGI process is added to additional groups +to give access to the socket from the redis DB:: + + $ groups searxng + searxng : searxng searxng-redis + +To see the effective groups of the uwsgi process, you have to look at the status +of the process, by example:: + + $ ps -aef | grep '/usr/sbin/uwsgi --ini searxng.ini' + searxng 93 92 0 12:43 ? 00:00:00 /usr/sbin/uwsgi --ini searxng.ini + searxng 186 93 0 12:44 ? 00:00:01 /usr/sbin/uwsgi --ini searxng.ini + +Here you can see that the additional "Groups" of PID 186 are unset (missing gid +of ``searxng-redis``):: + + $ cat /proc/186/task/186/status + ... + Uid: 993 993 993 993 + Gid: 993 993 993 993 + FDSize: 128 + Groups: + ... diff --git a/_sources/admin/installation.rst.txt b/_sources/admin/installation.rst.txt new file mode 100644 index 00000000..cae51be6 --- /dev/null +++ b/_sources/admin/installation.rst.txt @@ -0,0 +1,22 @@ +.. _installation: + +============ +Installation +============ + +*You're spoilt for choice*, choose your preferred method of installation. + +- :ref:`installation docker` +- :ref:`installation scripts` +- :ref:`installation basic` + +The :ref:`installation basic` is an excellent illustration of *how a SearXNG +instance is build up* (see :ref:`architecture uWSGI`). If you do not have any +special preferences, its recommend to use the :ref:`installation docker` or the +:ref:`installation scripts`. + +.. attention:: + + SearXNG is growing rapidly, you should regularly read our :ref:`migrate and + stay tuned` section. If you want to upgrade an existing instance or migrate + from searx to SearXNG, you should read this section first! diff --git a/_sources/admin/plugins.rst.txt b/_sources/admin/plugins.rst.txt new file mode 100644 index 00000000..d97b3dad --- /dev/null +++ b/_sources/admin/plugins.rst.txt @@ -0,0 +1,39 @@ +.. _plugins generic: + +=============== +Plugins builtin +=============== + +.. sidebar:: Further reading .. + + - :ref:`dev plugin` + +Configuration defaults (at built time): + +:DO: Default on + +.. _configured plugins: + +.. jinja:: searx + + .. flat-table:: Plugins configured at built time (defaults) + :header-rows: 1 + :stub-columns: 1 + :widths: 3 1 9 + + * - Name + - DO + - Description + + JS & CSS dependencies + + {% for plgin in plugins %} + + * - {{plgin.name}} + - {{(plgin.default_on and "y") or ""}} + - {{plgin.description}} + + {% for dep in (plgin.js_dependencies + plgin.css_dependencies) %} + | ``{{dep}}`` {% endfor %} + + {% endfor %} diff --git a/_sources/admin/update-searxng.rst.txt b/_sources/admin/update-searxng.rst.txt new file mode 100644 index 00000000..3ddb41b5 --- /dev/null +++ b/_sources/admin/update-searxng.rst.txt @@ -0,0 +1,136 @@ +=================== +SearXNG maintenance +=================== + +.. sidebar:: further read + + - :ref:`toolboxing` + - :ref:`uWSGI maintenance` + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +.. _update searxng: + +How to update +============= + +How to update depends on the :ref:`installation` method. If you have used the +:ref:`installation scripts`, use the ``update`` command from the :ref:`searxng.sh` +script. + +.. code:: sh + + sudo -H ./utils/searxng.sh instance update + +.. _inspect searxng: + +How to inspect & debug +====================== + +How to debug depends on the :ref:`installation` method. If you have used the +:ref:`installation scripts`, use the ``inspect`` command from the :ref:`searxng.sh` +script. + +.. code:: sh + + sudo -H ./utils/searxng.sh instance inspect + +.. _migrate and stay tuned: + +Migrate and stay tuned! +======================= + +.. sidebar:: info + + - :pull:`1332` + - :pull:`456` + - :pull:`A comment about rolling release <446#issuecomment-954730358>` + +SearXNG is a *rolling release*; each commit to the master branch is a release. +SearXNG is growing rapidly, the services and opportunities are change every now +and then, to name just a few: + +- Bot protection has been switched from filtron to SearXNG's :ref:`limiter + `, this requires a :ref:`Redis ` database. + +- The image proxy morty is no longer needed, it has been replaced by the + :ref:`image proxy ` from SearXNG. + +- To save bandwith :ref:`cache busting ` has been implemented. + To get in use, the ``static-expires`` needs to be set in the :ref:`uwsgi + setup`. + +To stay tuned and get in use of the new features, instance maintainers have to +update the SearXNG code regularly (see :ref:`update searxng`). As the above +examples show, this is not always enough, sometimes services have to be set up +or reconfigured and sometimes services that are no longer needed should be +uninstalled. + +.. hint:: + + First of all: SearXNG is installed by the script :ref:`searxng.sh`. If you + have old filtron, morty or searx setup you should consider complete + uninstall/reinstall. + +Here you will find a list of changes that affect the infrastructure. Please +check to what extent it is necessary to update your installations: + +:pull:`1595`: ``[fix] uWSGI: increase buffer-size`` + Re-install uWSGI (:ref:`searxng.sh`) or fix your uWSGI ``searxng.ini`` + file manually. + + +remove obsolete services +------------------------ + +If your searx instance was installed *"Step by step"* or by the *"Installation +scripts"*, you need to undo the installation procedure completely. If you have +morty & filtron installed, it is recommended to uninstall these services also. +In case of scripts, to uninstall use the scripts from the origin you installed +searx from or try:: + + $ sudo -H ./utils/filtron.sh remove all + $ sudo -H ./utils/morty.sh remove all + $ sudo -H ./utils/searx.sh remove all + +.. hint:: + + If you are migrate from searx take into account that the ``.config.sh`` is no + longer used. + +If you upgrade from searx or from before :pull:`1332` has been merged and you +have filtron and/or morty installed, don't forget to remove HTTP sites. + +Apache:: + + $ sudo -H ./utils/filtron.sh apache remove + $ sudo -H ./utils/morty.sh apache remove + +nginx:: + + $ sudo -H ./utils/filtron.sh nginx remove + $ sudo -H ./utils/morty.sh nginx remove + + + +Check after Installation +------------------------ + +Once you have done your installation, you can run a SearXNG *check* procedure, +to see if there are some left overs. In this example there exists a *old* +``/etc/searx/settings.yml``:: + + $ sudo -H ./utils/searxng.sh instance check + + SearXNG checks + -------------- + ERROR: settings.yml in /etc/searx/ is deprecated, move file to folder /etc/searxng/ + INFO: [OK] (old) account 'searx' does not exists + INFO: [OK] (old) account 'filtron' does not exists + INFO: [OK] (old) account 'morty' does not exists + ... + INFO searx.redisdb : connecting to Redis db=0 path='/usr/local/searxng-redis/run/redis.sock' + INFO searx.redisdb : connected to Redis diff --git a/_sources/dev/contribution_guide.rst.txt b/_sources/dev/contribution_guide.rst.txt new file mode 100644 index 00000000..0a2f07e4 --- /dev/null +++ b/_sources/dev/contribution_guide.rst.txt @@ -0,0 +1,185 @@ +.. _how to contribute: + +================= +How to contribute +================= + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +Prime directives: Privacy, Hackability +====================================== + +SearXNG has two prime directives, **privacy-by-design and hackability** . The +hackability comes in three levels: + +- support of search engines +- plugins to alter search behaviour +- hacking SearXNG itself + +Note the lack of "world domination" among the directives. SearXNG has no +intention of wide mass-adoption, rounded corners, etc. The prime directive +"privacy" deserves a separate chapter, as it's quite uncommon unfortunately. + +Privacy-by-design +----------------- + +SearXNG was born out of the need for a **privacy-respecting** search tool which +can be extended easily to maximize both, its search and its privacy protecting +capabilities. + +A few widely used features work differently or turned off by default or not +implemented at all **as a consequence of privacy-by-design**. + +If a feature reduces the privacy preserving aspects of searx, it should be +switched off by default or should not implemented at all. There are plenty of +search engines already providing such features. If a feature reduces the +protection of searx, users must be informed about the effect of choosing to +enable it. Features that protect privacy but differ from the expectations of +the user should also be explained. + +Also, if you think that something works weird with searx, it's might be because +of the tool you use is designed in a way to interfere with the privacy respect. +Submitting a bugreport to the vendor of the tool that misbehaves might be a good +feedback to reconsider the disrespect to its customers (e.g. ``GET`` vs ``POST`` +requests in various browsers). + +Remember the other prime directive of SearXNG is to be hackable, so if the above +privacy concerns do not fancy you, simply fork it. + + *Happy hacking.* + +Code +==== + +.. _PEP8: https://www.python.org/dev/peps/pep-0008/ +.. _Conventional Commits: https://www.conventionalcommits.org/ +.. _Git Commit Good Practice: https://wiki.openstack.org/wiki/GitCommitMessages +.. _Structural split of changes: + https://wiki.openstack.org/wiki/GitCommitMessages#Structural_split_of_changes +.. _gitmoji: https://gitmoji.carloscuesta.me/ +.. _Semantic PR: https://github.com/zeke/semantic-pull-requests + +.. sidebar:: Create good commits! + + - `Structural split of changes`_ + - `Conventional Commits`_ + - `Git Commit Good Practice`_ + - some like to use: gitmoji_ + - not yet active: `Semantic PR`_ + +In order to submit a patch, please follow the steps below: + +- Follow coding conventions. + + - PEP8_ standards apply, except the convention of line length + - Maximum line length is 120 characters + +- The cardinal rule for creating good commits is to ensure there is only one + *logical change* per commit / read `Structural split of changes`_ + +- Check if your code breaks existing tests. If so, update the tests or fix your + code. + +- If your code can be unit-tested, add unit tests. + +- Add yourself to the :origin:`AUTHORS.rst` file. + +- Choose meaningful commit messages, read `Conventional Commits`_ + + .. code:: + + [optional scope]: + + [optional body] + + [optional footer(s)] + +- Create a pull request. + +For more help on getting started with SearXNG development, see :ref:`devquickstart`. + + +Translation +=========== + +Translation currently takes place on :ref:`weblate `. + + +.. _contrib docs: + +Documentation +============= + +.. _Sphinx: https://www.sphinx-doc.org +.. _reST: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html + +.. sidebar:: The reST sources + + has been moved from ``gh-branch`` into ``master`` (:origin:`docs`). + +The documentation is built using Sphinx_. So in order to be able to generate +the required files, you have to install it on your system. Much easier, use +our :ref:`makefile`. + +Here is an example which makes a complete rebuild: + +.. code:: sh + + $ make docs.clean docs.html + ... + The HTML pages are in dist/docs. + +.. _make docs.live: + +live build +---------- + +.. _sphinx-autobuild: + https://github.com/executablebooks/sphinx-autobuild/blob/master/README.md + +.. sidebar:: docs.clean + + It is recommended to assert a complete rebuild before deploying (use + ``docs.clean``). + +Live build is like WYSIWYG. If you want to edit the documentation, its +recommended to use. The Makefile target ``docs.live`` builds the docs, opens +URL in your favorite browser and rebuilds every time a reST file has been +changed. + +.. code:: sh + + $ make docs.live + ... + The HTML pages are in dist/docs. + ... Serving on http://0.0.0.0:8000 + ... Start watching changes + +Live builds are implemented by sphinx-autobuild_. Use environment +``$(SPHINXOPTS)`` to pass arguments to the sphinx-autobuild_ command. Except +option ``--host`` (which is always set to ``0.0.0.0``) you can pass any +argument. E.g to find and use a free port, use: + +.. code:: sh + + $ SPHINXOPTS="--port 0" make docs.live + ... + ... Serving on http://0.0.0.0:50593 + ... + + +.. _deploy on github.io: + +deploy on github.io +------------------- + +To deploy documentation at :docs:`github.io <.>` use Makefile target :ref:`make +docs.gh-pages`, which builds the documentation and runs all the needed git add, +commit and push: + +.. code:: sh + + $ make docs.clean docs.gh-pages diff --git a/_sources/dev/engine_overview.rst.txt b/_sources/dev/engine_overview.rst.txt new file mode 100644 index 00000000..95ed267e --- /dev/null +++ b/_sources/dev/engine_overview.rst.txt @@ -0,0 +1,402 @@ +.. _engines-dev: + +=============== +Engine Overview +=============== + +.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine + +.. sidebar:: Further reading .. + + - :ref:`configured engines` + - :ref:`settings engine` + +.. contents:: + :depth: 3 + :backlinks: entry + +SearXNG is a metasearch-engine_, so it uses different search engines to provide +better results. + +Because there is no general search API which could be used for every search +engine, an adapter has to be built between SearXNG and the external search +engines. Adapters are stored under the folder :origin:`searx/engines`. + +.. _general engine configuration: + +General Engine Configuration +============================ + +It is required to tell SearXNG the type of results the engine provides. The +arguments can be set in the engine file or in the settings file (normally +``settings.yml``). The arguments in the settings file override the ones in the +engine file. + +It does not matter if an option is stored in the engine file or in the settings. +However, the standard way is the following: + +.. _engine file: + +Engine File +----------- + +.. table:: Common options in the engine module + :width: 100% + + ======================= =========== ======================================================== + argument type information + ======================= =========== ======================================================== + categories list pages, in which the engine is working + paging boolean support multiple pages + time_range_support boolean support search time range + engine_type str - ``online`` :ref:`[ref] ` by + default, other possibles values are: + - ``offline`` :ref:`[ref] ` + - ``online_dictionary`` + - ``online_currency`` + ======================= =========== ======================================================== + +.. _engine settings: + +Engine ``settings.yml`` +----------------------- + +For a more detailed description, see :ref:`settings engine` in the :ref:`settings.yml`. + +.. table:: Common options in the engine setup (``settings.yml``) + :width: 100% + + ======================= =========== ================================================== + argument type information + ======================= =========== ================================================== + name string name of search-engine + engine string name of searxng-engine (file name without ``.py``) + enable_http bool enable HTTP (by default only HTTPS is enabled). + shortcut string shortcut of search-engine + timeout string specific timeout for search-engine + display_error_messages boolean display error messages on the web UI + proxies dict set proxies for a specific engine + (e.g. ``proxies : {http: socks5://proxy:port, + https: socks5://proxy:port}``) + ======================= =========== ================================================== + +.. _engine overrides: + +Overrides +--------- + +A few of the options have default values in the namespace of engine's python +modul, but are often overwritten by the settings. If ``None`` is assigned to an +option in the engine file, it has to be redefined in the settings, otherwise +SearXNG will not start with that engine (global names with a leading underline can +be ``None``). + +Here is an very simple example of the global names in the namespace of engine's +module: + +.. code:: python + + # engine dependent config + categories = ['general'] + paging = True + _non_overwritten_global = 'foo' + + +.. table:: The naming of overrides is arbitrary / recommended overrides are: + :width: 100% + + ======================= =========== =========================================== + argument type information + ======================= =========== =========================================== + base_url string base-url, can be overwritten to use same + engine on other URL + number_of_results int maximum number of results per request + language string ISO code of language and country like en_US + api_key string api-key if required by engine + ======================= =========== =========================================== + +.. _engine request: + +Making a Request +================ + +To perform a search an URL have to be specified. In addition to specifying an +URL, arguments can be passed to the query. + +.. _engine request arguments: + +Passed Arguments (request) +-------------------------- + +These arguments can be used to construct the search query. Furthermore, +parameters with default value can be redefined for special purposes. + + +.. table:: If the ``engine_type`` is ``online`` + :width: 100% + + ====================== ============== ======================================================================== + argument type default-value, information + ====================== ============== ======================================================================== + url str ``''`` + method str ``'GET'`` + headers set ``{}`` + data set ``{}`` + cookies set ``{}`` + verify bool ``True`` + headers.User-Agent str a random User-Agent + category str current category, like ``'general'`` + safesearch int ``0``, between ``0`` and ``2`` (normal, moderate, strict) + time_range Optional[str] ``None``, can be ``day``, ``week``, ``month``, ``year`` + pageno int current pagenumber + language str specific language code like ``'en_US'``, or ``'all'`` if unspecified + ====================== ============== ======================================================================== + + +.. table:: If the ``engine_type`` is ``online_dictionary``, in addition to the + ``online`` arguments: + :width: 100% + + ====================== ============== ======================================================================== + argument type default-value, information + ====================== ============== ======================================================================== + from_lang str specific language code like ``'en_US'`` + to_lang str specific language code like ``'en_US'`` + query str the text query without the languages + ====================== ============== ======================================================================== + +.. table:: If the ``engine_type`` is ``online_currency```, in addition to the + ``online`` arguments: + :width: 100% + + ====================== ============== ======================================================================== + argument type default-value, information + ====================== ============== ======================================================================== + amount float the amount to convert + from str ISO 4217 code + to str ISO 4217 code + from_name str currency name + to_name str currency name + ====================== ============== ======================================================================== + + +Specify Request +--------------- + +The function :py:func:`def request(query, params): +` always returns the ``params`` variable, the +following parameters can be used to specify a search request: + +.. table:: + :width: 100% + + =================== =========== ========================================================================== + argument type information + =================== =========== ========================================================================== + url str requested url + method str HTTP request method + headers set HTTP header information + data set HTTP data information + cookies set HTTP cookies + verify bool Performing SSL-Validity check + allow_redirects bool Follow redirects + max_redirects int maximum redirects, hard limit + soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine + raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300 + =================== =========== ========================================================================== + + +.. _engine results: +.. _engine media types: + +Media Types +=========== + +Each result item of an engine can be of different media-types. Currently the +following media-types are supported. To set another media-type as ``default``, +the parameter ``template`` must be set to the desired type. + +.. table:: Parameter of the **default** media type: + :width: 100% + + ========================= ===================================================== + result-parameter information + ========================= ===================================================== + url string, url of the result + title string, title of the result + content string, general result-text + publishedDate :py:class:`datetime.datetime`, time of publish + ========================= ===================================================== + + +.. table:: Parameter of the **images** media type: + :width: 100% + + ========================= ===================================================== + result-parameter information + ------------------------- ----------------------------------------------------- + template is set to ``images.html`` + ========================= ===================================================== + url string, url to the result site + title string, title of the result *(partly implemented)* + content *(partly implemented)* + publishedDate :py:class:`datetime.datetime`, + time of publish *(partly implemented)* + img\_src string, url to the result image + thumbnail\_src string, url to a small-preview image + ========================= ===================================================== + + +.. table:: Parameter of the **videos** media type: + :width: 100% + + ========================= ===================================================== + result-parameter information + ------------------------- ----------------------------------------------------- + template is set to ``videos.html`` + ========================= ===================================================== + url string, url of the result + title string, title of the result + content *(not implemented yet)* + publishedDate :py:class:`datetime.datetime`, time of publish + thumbnail string, url to a small-preview image + ========================= ===================================================== + +.. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme + +.. table:: Parameter of the **torrent** media type: + :width: 100% + + ========================= ===================================================== + result-parameter information + ------------------------- ----------------------------------------------------- + template is set to ``torrent.html`` + ========================= ===================================================== + url string, url of the result + title string, title of the result + content string, general result-text + publishedDate :py:class:`datetime.datetime`, + time of publish *(not implemented yet)* + seed int, number of seeder + leech int, number of leecher + filesize int, size of file in bytes + files int, number of files + magnetlink string, magnetlink_ of the result + torrentfile string, torrentfile of the result + ========================= ===================================================== + +.. table:: Parameter of the **map** media type: + :width: 100% + + ========================= ===================================================== + result-parameter information + ------------------------- ----------------------------------------------------- + template is set to ``map.html`` + ========================= ===================================================== + url string, url of the result + title string, title of the result + content string, general result-text + publishedDate :py:class:`datetime.datetime`, time of publish + latitude latitude of result (in decimal format) + longitude longitude of result (in decimal format) + boundingbox boundingbox of result (array of 4. values + ``[lat-min, lat-max, lon-min, lon-max]``) + geojson geojson of result (https://geojson.org/) + osm.type type of osm-object (if OSM-Result) + osm.id id of osm-object (if OSM-Result) + address.name name of object + address.road street name of object + address.house_number house number of object + address.locality city, place of object + address.postcode postcode of object + address.country country of object + ========================= ===================================================== + +.. _BibTeX format: https://www.bibtex.com/g/bibtex-format/ +.. _BibTeX field types: https://en.wikipedia.org/wiki/BibTeX#Field_types + +.. list-table:: Parameter of the **paper** media type / + see `BibTeX field types`_ and `BibTeX format`_ + :header-rows: 2 + :width: 100% + + * - result-parameter + - Python type + - information + + * - template + - :py:class:`str` + - is set to ``paper.html`` + + * - title + - :py:class:`str` + - title of the result + + * - content + - :py:class:`str` + - abstract + + * - comments + - :py:class:`str` + - free text display in italic below the content + + * - tags + - :py:class:`List `\ [\ :py:class:`str`\ ] + - free tag list + + * - publishedDate + - :py:class:`datetime ` + - last publication date + + * - type + - :py:class:`str` + - short description of medium type, e.g. *book*, *pdf* or *html* ... + + * - authors + - :py:class:`List `\ [\ :py:class:`str`\ ] + - list of authors of the work (authors with a "s") + + * - editor + - :py:class:`str` + - list of editors of a book + + * - publisher + - :py:class:`str` + - name of the publisher + + * - journal + - :py:class:`str` + - name of the journal or magazine the article was + published in + + * - volume + - :py:class:`str` + - volume number + + * - pages + - :py:class:`str` + - page range where the article is + + * - number + - :py:class:`str` + - number of the report or the issue number for a journal article + + * - doi + - :py:class:`str` + - DOI number (like ``10.1038/d41586-018-07848-2``) + + * - issn + - :py:class:`List `\ [\ :py:class:`str`\ ] + - ISSN number like ``1476-4687`` + + * - isbn + - :py:class:`List `\ [\ :py:class:`str`\ ] + - ISBN number like ``9780201896831`` + + * - pdf_url + - :py:class:`str` + - URL to the full article, the PDF version + + * - html_url + - :py:class:`str` + - URL to full article, HTML version diff --git a/_sources/dev/index.rst.txt b/_sources/dev/index.rst.txt new file mode 100644 index 00000000..39be0885 --- /dev/null +++ b/_sources/dev/index.rst.txt @@ -0,0 +1,19 @@ +======================= +Developer documentation +======================= + +.. toctree:: + :maxdepth: 2 + :caption: Contents + + quickstart + contribution_guide + engine_overview + offline_engines + search_api + plugins + translation + lxcdev + makefile + reST + searxng_extra/index diff --git a/_sources/dev/lxcdev.rst.txt b/_sources/dev/lxcdev.rst.txt new file mode 100644 index 00000000..6688c21f --- /dev/null +++ b/_sources/dev/lxcdev.rst.txt @@ -0,0 +1,403 @@ +.. _lxcdev: + +============================== +Developing in Linux Containers +============================== + +.. _LXC: https://linuxcontainers.org/lxc/introduction/ + +In this article we will show, how you can make use of Linux Containers (LXC_) in +*distributed and heterogeneous development cycles* (TL;DR; jump to the +:ref:`lxcdev summary`). + +.. sidebar:: Audience + + This blog post is written for experienced admins and developers. Readers + should have a serious meaning about the terms: *distributed*, *merge* and + *linux container*. + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + + +Motivation +========== + +Usually in our development cycle, we edit the sources and run some test and/or +builds by using ``make`` :ref:`[ref] ` before we commit. This cycle +is simple and perfect but might fail in some aspects we should not overlook. + + **The environment in which we run all our development processes matters!** + +The :ref:`makefile` and the :ref:`make install` encapsulate a lot for us, but +they do not have access to all prerequisites. For example, there may have +dependencies on packages that are installed on the developer's desktop, but +usually are not preinstalled on a server or client system. Another example is; +settings have been made to the software on developer's desktop that would never +be set on a *production* system. + + **Linux Containers are isolate environments and not to mix up all the + prerequisites from various projects on developer's desktop is always a good + choice.** + +The scripts from :ref:`searx_utils` can divide in those to install and maintain +software: + +- :ref:`searxng.sh` + +and the script :ref:`lxc.sh`, with we can scale our installation, maintenance or +even development tasks over a stack of isolated containers / what we call the: + + **SearXNG LXC suite** + +.. hint:: + + If you see any problems with the internet connectivity of your + containers read section :ref:`internet connectivity docker`. + + +Gentlemen, start your engines! +============================== + +.. _LXD: https://linuxcontainers.org/lxd/introduction/ +.. _archlinux: https://www.archlinux.org/ + +Before you can start with containers, you need to install and initiate LXD_ +once: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: bash + + $ snap install lxd + $ lxd init --auto + +And you need to clone from origin or if you have your own fork, clone from your +fork: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: bash + + $ cd ~/Downloads + $ git clone https://github.com/searxng/searxng.git searxng + $ cd searxng + +The :ref:`lxc-searxng.env` consists of several images, see ``export +LXC_SUITE=(...`` near by :origin:`utils/lxc-searxng.env#L19`. For this blog post +we exercise on a archlinux_ image. The container of this image is named +``searxng-archlinux``. Lets build the container, but be sure that this container +does not already exists, so first lets remove possible old one: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: bash + + $ sudo -H ./utils/lxc.sh remove searxng-archlinux + $ sudo -H ./utils/lxc.sh build searxng-archlinux + +.. sidebar:: The ``searxng-archlinux`` container + + is the base of all our exercises here. + +In this container we install all services :ref:`including searx, morty & filtron +` in once: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: bash + + $ sudo -H ./utils/lxc.sh install suite searxng-archlinux + +To proxy HTTP from filtron and morty in the container to the outside of the +container, install nginx into the container. Once for the bot blocker filtron: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ + ./utils/filtron.sh nginx install + ... + INFO: got 429 from http://10.174.184.156/searx + +and once for the content sanitizer (content proxy morty): + +.. tabs:: + + .. group-tab:: desktop + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ + ./utils/morty.sh nginx install + ... + INFO: got 200 from http://10.174.184.156/morty/ + +.. sidebar:: Fully functional SearXNG suite + + From here on you have a fully functional SearXNG suite running with bot + blocker (filtron) and WEB content sanitizer (content proxy morty), both are + needed for a *privacy protecting* search engine. + +On your system, the IP of your ``searxng-archlinux`` container differs from +http://10.174.184.156/searx, just open the URL reported in your installation +protocol in your WEB browser from the desktop to test the instance from outside +of the container. + +In such a earXNG suite admins can maintain and access the debug log of the +different services quite easy. + +.. _working in containers: + +In containers, work as usual +============================ + +Usually you open a root-bash using ``sudo -H bash``. In case of LXC containers +open the root-bash in the container using ``./utils/lxc.sh cmd +searxng-archlinux``: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash + INFO: [searxng-archlinux] bash + [root@searxng-archlinux searx]# pwd + /share/searxng + +The prompt ``[root@searxng-archlinux ...]`` signals, that you are the root user in +the searxng-container. To debug the running SearXNG instance use: + +.. tabs:: + + .. group-tab:: root@searxng-archlinux + + .. code:: bash + + $ ./utils/searx.sh inspect service + ... + use [CTRL-C] to stop monitoring the log + ... + +Back in the browser on your desktop open the service http://10.174.184.156/searx +and run your application tests while the debug log is shown in the terminal from +above. You can stop monitoring using ``CTRL-C``, this also disables the *"debug +option"* in SearXNG's settings file and restarts the SearXNG uwsgi application. +To debug services from filtron and morty analogous use: + +Another point we have to notice is that the service (:ref:`SearXNG ` +runs under dedicated system user account with the same name (compare +:ref:`create searxng user`). To get a shell from these accounts, simply call: + +.. tabs:: + + .. group-tab:: root@searxng-archlinux + + .. code:: bash + + $ ./utils/searxng.sh instance cmd bash + +To get in touch, open a shell from the service user (searxng@searxng-archlinux): + +.. tabs:: + + .. group-tab:: desktop + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance cmd bash + INFO: [searxng-archlinux] ./utils/searxng.sh instance cmd bash + [searxng@searxng-archlinux ~]$ + +The prompt ``[searxng@searxng-archlinux]`` signals that you are logged in as system +user ``searx`` in the ``searxng-archlinux`` container and the python *virtualenv* +``(searxng-pyenv)`` environment is activated. + +.. tabs:: + + .. group-tab:: searxng@searxng-archlinux + + .. code:: bash + + (searxng-pyenv) [searxng@searxng-archlinux ~]$ pwd + /usr/local/searxng + + +Wrap production into developer suite +==================================== + +In this section we will see how to change the *"Fully functional SearXNG suite"* +from a LXC container (which is quite ready for production) into a developer +suite. For this, we have to keep an eye on the :ref:`installation basic`: + +- SearXNG setup in: ``/etc/searxng/settings.yml`` +- SearXNG user's home: ``/usr/local/searxng`` +- virtualenv in: ``/usr/local/searxng/searxng-pyenv`` +- SearXNG software in: ``/usr/local/searxng/searxng-src`` + +With the use of the :ref:`searxng.sh` the SearXNG service was installed as +:ref:`uWSGI application `. To maintain this service, we can use +``systemctl`` (compare :ref:`uWSGI maintenance`). + +.. tabs:: + + .. group-tab:: desktop + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ + systemctl stop uwsgi@searxng + +With the command above, we stopped the SearXNG uWSGI-App in the archlinux +container. + +The uWSGI-App for the archlinux dsitros is configured in +:origin:`utils/templates/etc/uwsgi/apps-archlinux/searxng.ini`, from where at +least you should attend the settings of ``uid``, ``chdir``, ``env`` and +``http``:: + + env = SEARXNG_SETTINGS_PATH=/etc/searxng/settings.yml + http = 127.0.0.1:8888 + + chdir = /usr/local/searxng/searxng-src/searx + virtualenv = /usr/local/searxng/searxng-pyenv + pythonpath = /usr/local/searxng/searxng-src + +If you have read the :ref:`"Good to know section" ` you remember, that +each container shares the root folder of the repository and the command +``utils/lxc.sh cmd`` handles relative path names **transparent**. To wrap the +SearXNG installation into a developer one, we simple have to create a smylink to +the **transparent** reposetory from the desktop. Now lets replace the +repository at ``searxng-src`` in the container with the working tree from outside +of the container: + +.. tabs:: + + .. group-tab:: container becomes a developer suite + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ + mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ + ln -s /share/searx/ /usr/local/searxng/searxng-src + +Now we can develop as usual in the working tree of our desktop system. Every +time the software was changed, you have to restart the SearXNG service (in the +container): + +.. tabs:: + + .. group-tab:: desktop + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ + systemctl restart uwsgi@searx + + +Remember: :ref:`working in containers` .. here are just some examples from my +daily usage: + +.. tabs:: + + .. group-tab:: desktop + + To *inspect* the SearXNG instance (already described above): + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ + ./utils/searx.sh inspect service + + Run :ref:`makefile`, e.g. to test inside the container: + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ + make test + + To install all prerequisites needed for a :ref:`buildhosts`: + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ + ./utils/searxng.sh install buildhost + + To build the docs on a buildhost :ref:`buildhosts`: + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ + make docs.html + +.. _lxcdev summary: + +Summary +======= + +We build up a fully functional SearXNG suite in a archlinux container: + +.. code:: bash + + $ sudo -H ./utils/lxc.sh install suite searxng-archlinux + +To access HTTP from the desktop we installed nginx for the services inside the +container: + +.. tabs:: + + .. group-tab:: [root@searxng-archlinux] + + .. code:: bash + + $ ./utils/filtron.sh nginx install + $ ./utils/morty.sh nginx install + +To wrap the suite into a developer one, we created a symbolic link to the +repository which is shared **transparent** from the desktop's file system into +the container : + +.. tabs:: + + .. group-tab:: [root@searxng-archlinux] + + .. code:: bash + + $ mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old + $ ln -s /share/searx/ /usr/local/searxng/searxng-src + $ systemctl restart uwsgi@searx + +To get information about the searxNG suite in the archlinux container we can +use: + +.. tabs:: + + .. group-tab:: desktop + + .. code:: bash + + $ sudo -H ./utils/lxc.sh show suite searxng-archlinux + ... + [searxng-archlinux] INFO: (eth0) filtron: http://10.174.184.156:4004/ http://10.174.184.156/searx + [searxng-archlinux] INFO: (eth0) morty: http://10.174.184.156:3000/ + [searxng-archlinux] INFO: (eth0) docs.live: http://10.174.184.156:8080/ + [searxng-archlinux] INFO: (eth0) IPv6: http://[fd42:573b:e0b3:e97e:216:3eff:fea5:9b65] + ... + diff --git a/_sources/dev/makefile.rst.txt b/_sources/dev/makefile.rst.txt new file mode 100644 index 00000000..68c708a8 --- /dev/null +++ b/_sources/dev/makefile.rst.txt @@ -0,0 +1,290 @@ +.. _makefile: + +======== +Makefile +======== + +.. _gnu-make: https://www.gnu.org/software/make/manual/make.html#Introduction + +.. sidebar:: build environment + + Before looking deeper at the targets, first read about :ref:`make + install`. + + To install system requirements follow :ref:`buildhosts`. + +All relevant build tasks are implemented in :origin:`manage` and for CI or +IDE integration a small ``Makefile`` wrapper is available. If you are not +familiar with Makefiles, we recommend to read gnu-make_ introduction. + +The usage is simple, just type ``make {target-name}`` to *build* a target. +Calling the ``help`` target gives a first overview (``make help``): + +.. program-output:: bash -c "cd ..; make --no-print-directory help" + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +.. _make install: + +Python environment (``make install``) +===================================== + +.. sidebar:: activate environment + + ``source ./local/py3/bin/activate`` + +We do no longer need to build up the virtualenv manually. Jump into your git +working tree and release a ``make install`` to get a virtualenv with a +*developer install* of SearXNG (:origin:`setup.py`). :: + + $ cd ~/searxng-clone + $ make install + PYENV [virtualenv] installing ./requirements*.txt into local/py3 + ... + PYENV OK + PYENV [install] pip install -e 'searx[test]' + ... + Successfully installed argparse-1.4.0 searx + BUILDENV INFO:searx:load the default settings from ./searx/settings.yml + BUILDENV INFO:searx:Initialisation done + BUILDENV build utils/brand.env + +If you release ``make install`` multiple times the installation will only +rebuild if the sha256 sum of the *requirement files* fails. With other words: +the check fails if you edit the requirements listed in +:origin:`requirements-dev.txt` and :origin:`requirements.txt`). :: + + $ make install + PYENV OK + PYENV [virtualenv] requirements.sha256 failed + [virtualenv] - 6cea6eb6def9e14a18bf32f8a3e... ./requirements-dev.txt + [virtualenv] - 471efef6c73558e391c3adb35f4... ./requirements.txt + ... + PYENV [virtualenv] installing ./requirements*.txt into local/py3 + ... + PYENV OK + PYENV [install] pip install -e 'searx[test]' + ... + Successfully installed argparse-1.4.0 searx + BUILDENV INFO:searx:load the default settings from ./searx/settings.yml + BUILDENV INFO:searx:Initialisation done + BUILDENV build utils/brand.env + +.. sidebar:: drop environment + + To get rid of the existing environment before re-build use :ref:`clean target + ` first. + +If you think, something goes wrong with your ./local environment or you change +the :origin:`setup.py` file, you have to call :ref:`make clean`. + +.. _make buildenv: + +``make buildenv`` +================= + +Rebuild instance's environment with the modified settings from the +:ref:`settings brand` and :ref:`settings server` section of your +:ref:`settings.yml `. + +We have all SearXNG setups are centralized in the :ref:`settings.yml` file. +This setup is available as long we are in a *installed instance*. E.g. the +*installed instance* on the server or the *installed developer instance* at +``./local`` (the later one is created by a :ref:`make install ` or :ref:`make run `). + +Tasks running outside of an *installed instance*, especially those tasks and +scripts running at (pre-) installation time do not have access to the SearXNG +setup (from a *installed instance*). Those tasks need a *build environment*. + +The ``make buildenv`` target will update the *build environment* in: + +- :origin:`utils/brand.env` + +Tasks running outside of an *installed instance*, need the following settings +from the YAML configuration: + +- ``SEARXNG_URL`` from :ref:`server.base_url ` (aka + ``PUBLIC_URL``) +- ``SEARXNG_BIND_ADDRESS`` from :ref:`server.bind_address ` +- ``SEARXNG_PORT`` from :ref:`server.port ` + +.. _make node.env: + +Node.js environment (``make node.env``) +======================================= + +.. _Node.js: https://nodejs.org/ +.. _nvm: https://github.com/nvm-sh +.. _npm: https://www.npmjs.com/ + +.. jinja:: searx + + Node.js_ version {{version.node}} or higher is required to build the themes. + If the requirement is not met, the build chain uses nvm_ (Node Version + Manager) to install latest LTS of Node.js_ locally: there is no need to + install nvm_ or npm_ on your system. + +Use ``make nvm.status`` to get the current status of you Node.js_ and nvm_ setup. + +Here is the output you will typically get on a Ubuntu 20.04 system which serves +only a `no longer active `_ Release +`Node.js v10.19.0 `_. + +:: + + $ make nvm.status + INFO: Node.js is installed at /usr/bin/node + INFO: Node.js is version v10.19.0 + WARN: minimal Node.js version is 16.13.0 + INFO: npm is installed at /usr/bin/npm + INFO: npm is version 6.14.4 + WARN: NVM is not installed + INFO: to install NVM and Node.js (LTS) use: manage nvm install --lts + +To install you can also use :ref:`make nvm.nodejs` + +.. _make nvm.nodejs: + +``make nvm.nodejs`` +=================== + +Install latest Node.js_ LTS locally (uses nvm_):: + + $ make nvm.nodejs + INFO: install (update) NVM at /share/searxng/.nvm + INFO: clone: https://github.com/nvm-sh/nvm.git + ... + Downloading and installing node v16.13.0... + ... + INFO: Node.js is installed at searxng/.nvm/versions/node/v16.13.0/bin/node + INFO: Node.js is version v16.13.0 + INFO: npm is installed at searxng/.nvm/versions/node/v16.13.0/bin/npm + INFO: npm is version 8.1.0 + INFO: NVM is installed at searxng/.nvm + +.. _make run: + +``make run`` +============ + +To get up a running a developer instance simply call ``make run``. This enables +*debug* option in :origin:`searx/settings.yml`, starts a ``./searx/webapp.py`` +instance and opens the URL in your favorite WEB browser (:man:`xdg-open`):: + + $ make run + +Changes to theme's HTML templates (jinja2) are instant. Changes to the CSS & JS +sources of the theme need to be rebuild. You can do that by running:: + + $ make themes.all + +Alternatively to ``themes.all`` you can run *live builds* of the theme you are +modify:: + + $ LIVE_THEME=simple make run + +.. _make clean: + +``make clean`` +============== + +Drops all intermediate files, all builds, but keep sources untouched. Before +calling ``make clean`` stop all processes using the :ref:`make install` or +:ref:`make node.env`. :: + + $ make clean + CLEAN pyenv + PYENV [virtualenv] drop local/py3 + CLEAN docs -- build/docs dist/docs + CLEAN themes -- locally installed npm dependencies + ... + CLEAN test stuff + CLEAN common files + +.. _make docs: + +``make docs docs.autobuild docs.clean`` +======================================= + +We describe the usage of the ``doc.*`` targets in the :ref:`How to contribute / +Documentation ` section. If you want to edit the documentation +read our :ref:`make docs.live` section. If you are working in your own brand, +adjust your :ref:`settings global`. + +.. _make docs.gh-pages: + +``make docs.gh-pages`` +====================== + +To deploy on github.io first adjust your :ref:`settings global`. For any +further read :ref:`deploy on github.io`. + +.. _make test: + +``make test`` +============= + +Runs a series of tests: :ref:`make test.pylint`, ``test.pep8``, ``test.unit`` +and ``test.robot``. You can run tests selective, e.g.:: + + $ make test.pep8 test.unit test.sh + TEST test.pep8 OK + ... + TEST test.unit OK + ... + TEST test.sh OK + +.. _make test.shell: + +``make test.shell`` +=================== + +:ref:`sh lint` / if you have changed some bash scripting run this test before +commit. + +.. _make test.pylint: + +``make test.pylint`` +==================== + +.. _Pylint: https://www.pylint.org/ + +Pylint_ is known as one of the best source-code, bug and quality checker for the +Python programming language. The pylint profile used in the SearXNG project is +found in project's root folder :origin:`.pylintrc`. + +.. _make search.checker: + +``search.checker.{engine name}`` +================================ + +To check all engines:: + + make search.checker + +To check a engine with whitespace in the name like *google news* replace space +by underline:: + + make search.checker.google_news + +To see HTTP requests and more use SEARXNG_DEBUG:: + + make SEARXNG_DEBUG=1 search.checker.google_news + +.. _3xx: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_redirection + +To filter out HTTP redirects (3xx_):: + + make SEARXNG_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]" + ... + Engine google news Checking + https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0 + https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None + -- + https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0 + https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None + -- diff --git a/_sources/dev/offline_engines.rst.txt b/_sources/dev/offline_engines.rst.txt new file mode 100644 index 00000000..bfb2664f --- /dev/null +++ b/_sources/dev/offline_engines.rst.txt @@ -0,0 +1,78 @@ +.. _offline engines: + +=============== +Offline Engines +=============== + +.. sidebar:: offline engines + + - :ref:`demo offline engine` + - :ref:`sql engines` + - :ref:`engine command` + - :origin:`Redis ` + +To extend the functionality of SearXNG, offline engines are going to be +introduced. An offline engine is an engine which does not need Internet +connection to perform a search and does not use HTTP to communicate. + +Offline engines can be configured, by adding those to the `engines` list of +:origin:`settings.yml `. An example skeleton for offline +engines can be found in :ref:`demo offline engine` (:origin:`demo_offline.py +`). + + +Programming Interface +===================== + +:py:func:`init(engine_settings=None) ` + All offline engines can have their own init function to setup the engine before + accepting requests. The function gets the settings from settings.yml as a + parameter. This function can be omitted, if there is no need to setup anything + in advance. + +:py:func:`search(query, params) ` + + Each offline engine has a function named ``search``. This function is + responsible to perform a search and return the results in a presentable + format. (Where *presentable* means presentable by the selected result + template.) + + The return value is a list of results retrieved by the engine. + +Engine representation in ``/config`` + If an engine is offline, the attribute ``offline`` is set to ``True``. + +.. _offline requirements: + +Extra Dependencies +================== + +If an offline engine depends on an external tool, SearXNG does not install it by +default. When an administrator configures such engine and starts the instance, +the process returns an error with the list of missing dependencies. Also, +required dependencies will be added to the comment/description of the engine, so +admins can install packages in advance. + +If there is a need to install additional packages in *Python's Virtual +Environment* of your SearXNG instance you need to switch into the environment +(:ref:`searxng-src`) first, for this you can use :ref:`searxng.sh`:: + + $ sudo utils/searxng.sh instance cmd bash + (searxng-pyenv)$ pip install ... + + +Private engines (Security) +========================== + +To limit the access to offline engines, if an instance is available publicly, +administrators can set token(s) for each of the :ref:`private engines`. If a +query contains a valid token, then SearXNG performs the requested private +search. If not, requests from an offline engines return errors. + + +Acknowledgement +=============== + +This development was sponsored by `Search and Discovery Fund +`_ of `NLnet Foundation `_ . + diff --git a/_sources/dev/plugins.rst.txt b/_sources/dev/plugins.rst.txt new file mode 100644 index 00000000..36a44323 --- /dev/null +++ b/_sources/dev/plugins.rst.txt @@ -0,0 +1,110 @@ +.. _dev plugin: + +======= +Plugins +======= + +.. sidebar:: Further reading .. + + - :ref:`plugins generic` + +Plugins can extend or replace functionality of various components of searx. + +Example plugin +============== + +.. code:: python + + name = 'Example plugin' + description = 'This plugin extends the suggestions with the word "example"' + default_on = False # disabled by default + + js_dependencies = tuple() # optional, list of static js files + css_dependencies = tuple() # optional, list of static css files + + + # attach callback to the post search hook + # request: flask request object + # ctx: the whole local context of the post search hook + def post_search(request, search): + search.result_container.suggestions.add('example') + return True + +External plugins +================ + +SearXNG supports *external plugins* / there is no need to install one, SearXNG +runs out of the box. But to demonstrate; in the example below we install the +SearXNG plugins from *The Green Web Foundation* `[ref] +`__: + +.. code:: bash + + $ sudo utils/searxng.sh instance cmd bash + (searxng-pyenv)$ pip install git+https://github.com/return42/tgwf-searx-plugins + +In the :ref:`settings.yml` activate the ``plugins:`` section and add module +``only_show_green_results`` from ``tgwf-searx-plugins``. + +.. code:: yaml + + plugins: + ... + - only_show_green_results + ... + + +Plugin entry points +=================== + +Entry points (hooks) define when a plugin runs. Right now only three hooks are +implemented. So feel free to implement a hook if it fits the behaviour of your +plugin. A plugin doesn't need to implement all the hooks. + + +.. py:function:: pre_search(request, search) -> bool + + Runs BEFORE the search request. + + `search.result_container` can be changed. + + Return a boolean: + + * True to continue the search + * False to stop the search + + :param flask.request request: + :param searx.search.SearchWithPlugins search: + :return: False to stop the search + :rtype: bool + + +.. py:function:: post_search(request, search) -> None + + Runs AFTER the search request. + + :param flask.request request: Flask request. + :param searx.search.SearchWithPlugins search: Context. + + +.. py:function:: on_result(request, search, result) -> bool + + Runs for each result of each engine. + + `result` can be changed. + + If `result["url"]` is defined, then `result["parsed_url"] = urlparse(result['url'])` + + .. warning:: + `result["url"]` can be changed, but `result["parsed_url"]` must be updated too. + + Return a boolean: + + * True to keep the result + * False to remove the result + + :param flask.request request: + :param searx.search.SearchWithPlugins search: + :param typing.Dict result: Result, see - :ref:`engine results` + :return: True to keep the result + :rtype: bool diff --git a/_sources/dev/quickstart.rst.txt b/_sources/dev/quickstart.rst.txt new file mode 100644 index 00000000..921384aa --- /dev/null +++ b/_sources/dev/quickstart.rst.txt @@ -0,0 +1,72 @@ +.. _devquickstart: + +====================== +Development Quickstart +====================== + +.. _npm: https://www.npmjs.com/ +.. _Node.js: https://nodejs.org/ + +SearXNG loves developers, just clone and start hacking. All the rest is done for +you simply by using :ref:`make `. + +.. code:: bash + + git clone https://github.com/searxng/searxng.git searxng + +Here is how a minimal workflow looks like: + +1. *start* hacking +2. *run* your code: :ref:`make run` +3. *test* your code: :ref:`make test` + +If you think at some point something fails, go back to *start*. Otherwise, +choose a meaningful commit message and we are happy to receive your pull +request. To not end in *wild west* we have some directives, please pay attention +to our ":ref:`how to contribute`" guideline. + +If you implement themes, you will need to setup a :ref:`make node.env` once: + +.. code:: bash + + make node.env + +Before you call *make run* (2.), you need to compile the modified styles and +JavaScript: + +.. code:: bash + + make themes.all + +Alternatively you can also compile selective the theme you have modified, +e.g. the *simple* theme. + +.. code:: bash + + make themes.simple + +.. tip:: + + To get live builds while modifying CSS & JS use: ``LIVE_THEME=simple make run`` + +If you finished your *tests* you can start to commit your changes. To separate +the modified source code from the build products first run: + +.. code:: bash + + make static.build.restore + +This will restore the old build products and only your changes of the code +remain in the working tree which can now be added & commited. When all sources +are commited, you can commit the build products simply by: + +.. code:: bash + + make static.build.commit + +Commiting the build products should be the last step, just before you send us +your PR. There is also a make target to rewind this last build commit: + +.. code:: bash + + make static.build.drop diff --git a/_sources/dev/reST.rst.txt b/_sources/dev/reST.rst.txt new file mode 100644 index 00000000..e5d49f67 --- /dev/null +++ b/_sources/dev/reST.rst.txt @@ -0,0 +1,1436 @@ +.. _reST primer: + +=========== +reST primer +=========== + +.. sidebar:: KISS_ and readability_ + + Instead of defining more and more roles, we at SearXNG encourage our + contributors to follow principles like KISS_ and readability_. + +We at SearXNG are using reStructuredText (aka reST_) markup for all kind of +documentation, with the builders from the Sphinx_ project a HTML output is +generated and deployed at :docs:`github.io <.>`. For build prerequisites read +:ref:`docs build`. + +The source files of Searx's documentation are located at :origin:`docs`. Sphinx +assumes source files to be encoded in UTF-8 by default. Run :ref:`make docs.live +` to build HTML while editing. + +.. sidebar:: Further reading + + - Sphinx-Primer_ + - `Sphinx markup constructs`_ + - reST_, docutils_, `docutils FAQ`_ + - Sphinx_, `sphinx-doc FAQ`_ + - `sphinx config`_, doctree_ + - `sphinx cross references`_ + - linuxdoc_ + - intersphinx_ + - sphinx-jinja_ + - `Sphinx's autodoc`_ + - `Sphinx's Python domain`_, `Sphinx's C domain`_ + - SVG_, ImageMagick_ + - DOT_, `Graphviz's dot`_, Graphviz_ + + +.. contents:: Contents + :depth: 3 + :local: + :backlinks: entry + +Sphinx_ and reST_ have their place in the python ecosystem. Over that reST is +used in popular projects, e.g the Linux kernel documentation `[kernel doc]`_. + +.. _[kernel doc]: https://www.kernel.org/doc/html/latest/doc-guide/sphinx.html + +.. sidebar:: Content matters + + The readability_ of the reST sources has its value, therefore we recommend to + make sparse usage of reST markup / .. content matters! + +**reST** is a plaintext markup language, its markup is *mostly* intuitive and +you will not need to learn much to produce well formed articles with. I use the +word *mostly*: like everything in live, reST has its advantages and +disadvantages, some markups feel a bit grumpy (especially if you are used to +other plaintext markups). + +Soft skills +=========== + +Before going any deeper into the markup let's face on some **soft skills** a +trained author brings with, to reach a well feedback from readers: + +- Documentation is dedicated to an audience and answers questions from the + audience point of view. +- Don't detail things which are general knowledge from the audience point of + view. +- Limit the subject, use cross links for any further reading. + +To be more concrete what a *point of view* means. In the (:origin:`docs`) +folder we have three sections (and the *blog* folder), each dedicate to a +different group of audience. + +User's POV: :origin:`docs/user` + A typical user knows about search engines and might have heard about + meta crawlers and privacy. + +Admin's POV: :origin:`docs/admin` + A typical Admin knows about setting up services on a linux system, but he does + not know all the pros and cons of a SearXNG setup. + +Developer's POV: :origin:`docs/dev` + Depending on the readability_ of code, a typical developer is able to read and + understand source code. Describe what a item aims to do (e.g. a function). + If the chronological order matters, describe it. Name the *out-of-limits + conditions* and all the side effects a external developer will not know. + +.. _reST inline markup: + +Basic inline markup +=================== + +.. sidebar:: Inline markup + + - :ref:`reST roles` + - :ref:`reST smart ref` + +Basic inline markup is done with asterisks and backquotes. If asterisks or +backquotes appear in running text and could be confused with inline markup +delimiters, they have to be escaped with a backslash (``\*pointer``). + +.. table:: basic inline markup + :widths: 4 3 7 + + ================================================ ==================== ======================== + description rendered markup + ================================================ ==================== ======================== + one asterisk for emphasis *italics* ``*italics*`` + two asterisks for strong emphasis **boldface** ``**boldface**`` + backquotes for code samples and literals ``foo()`` ````foo()```` + quote asterisks or backquotes \*foo is a pointer ``\*foo is a pointer`` + ================================================ ==================== ======================== + +.. _reST basic structure: + +Basic article structure +======================= + +The basic structure of an article makes use of heading adornments to markup +chapter, sections and subsections. + +.. _reST template: + +reST template +------------- + +reST template for an simple article: + +.. code:: reST + + .. _doc refname: + + ============== + Document title + ============== + + Lorem ipsum dolor sit amet, consectetur adipisici elit .. Further read + :ref:`chapter refname`. + + .. _chapter refname: + + Chapter + ======= + + Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut + aliquid ex ea commodi consequat ... + + .. _section refname: + + Section + ------- + + lorem .. + + .. _subsection refname: + + Subsection + ~~~~~~~~~~ + + lorem .. + + +Headings +-------- + +#. title - with overline for document title: + + .. code:: reST + + ============== + Document title + ============== + + +#. chapter - with anchor named ``anchor name``: + + .. code:: reST + + .. _anchor name: + + Chapter + ======= + +#. section + + .. code:: reST + + Section + ------- + +#. subsection + + .. code:: reST + + Subsection + ~~~~~~~~~~ + + + +Anchors & Links +=============== + +.. _reST anchor: + +Anchors +------- + +.. _ref role: + https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-ref + +To refer a point in the documentation a anchor is needed. The :ref:`reST +template ` shows an example where a chapter titled *"Chapters"* +gets an anchor named ``chapter title``. Another example from *this* document, +where the anchor named ``reST anchor``: + +.. code:: reST + + .. _reST anchor: + + Anchors + ------- + + To refer a point in the documentation a anchor is needed ... + +To refer anchors use the `ref role`_ markup: + +.. code:: reST + + Visit chapter :ref:`reST anchor`. Or set hyperlink text manually :ref:`foo + bar `. + +.. admonition:: ``:ref:`` role + :class: rst-example + + Visist chapter :ref:`reST anchor`. Or set hyperlink text manually :ref:`foo + bar `. + +.. _reST ordinary ref: + +Link ordinary URL +----------------- + +If you need to reference external URLs use *named* hyperlinks to maintain +readability of reST sources. Here is a example taken from *this* article: + +.. code:: reST + + .. _Sphinx Field Lists: + https://www.sphinx-doc.org/en/master/usage/restructuredtext/field-lists.html + + With the *named* hyperlink `Sphinx Field Lists`_, the raw text is much more + readable. + + And this shows the alternative (less readable) hyperlink markup `Sphinx Field + Lists + `__. + +.. admonition:: Named hyperlink + :class: rst-example + + With the *named* hyperlink `Sphinx Field Lists`_, the raw text is much more + readable. + + And this shows the alternative (less readable) hyperlink markup `Sphinx Field + Lists + `__. + + +.. _reST smart ref: + +Smart refs +---------- + +With the power of sphinx.ext.extlinks_ and intersphinx_ referencing external +content becomes smart. + +.. table:: smart refs with sphinx.ext.extlinks_ and intersphinx_ + :widths: 4 3 7 + + ========================== ================================== ==================================== + refer ... rendered example markup + ========================== ================================== ==================================== + :rst:role:`rfc` :rfc:`822` ``:rfc:`822``` + :rst:role:`pep` :pep:`8` ``:pep:`8``` + sphinx.ext.extlinks_ + -------------------------------------------------------------------------------------------------- + project's wiki article :wiki:`Offline-engines` ``:wiki:`Offline-engines``` + to docs public URL :docs:`dev/reST.html` ``:docs:`dev/reST.html``` + files & folders origin :origin:`docs/dev/reST.rst` ``:origin:`docs/dev/reST.rst``` + pull request :pull:`4` ``:pull:`4``` + patch :patch:`af2cae6` ``:patch:`af2cae6``` + PyPi package :pypi:`searx` ``:pypi:`searx``` + manual page man :man:`bash` ``:man:`bash``` + intersphinx_ + -------------------------------------------------------------------------------------------------- + external anchor :ref:`python:and` ``:ref:`python:and``` + external doc anchor :doc:`jinja:templates` ``:doc:`jinja:templates``` + python code object :py:obj:`datetime.datetime` ``:py:obj:`datetime.datetime``` + flask code object :py:obj:`flask.Flask` ``:py:obj:`flask.Flask``` + ========================== ================================== ==================================== + + +Intersphinx is configured in :origin:`docs/conf.py`: + +.. code:: python + + intersphinx_mapping = { + "python": ("https://docs.python.org/3/", None), + "flask": ("https://flask.palletsprojects.com/", None), + "jinja": ("https://jinja.palletsprojects.com/", None), + "linuxdoc" : ("https://return42.github.io/linuxdoc/", None), + "sphinx" : ("https://www.sphinx-doc.org/en/master/", None), + } + + +To list all anchors of the inventory (e.g. ``python``) use: + +.. code:: sh + + $ python -m sphinx.ext.intersphinx https://docs.python.org/3/objects.inv + ... + $ python -m sphinx.ext.intersphinx https://docs.searxng.org/objects.inv + ... + +Literal blocks +============== + +The simplest form of :duref:`literal-blocks` is a indented block introduced by +two colons (``::``). For highlighting use :dudir:`highlight` or :ref:`reST +code` directive. To include literals from external files use +:rst:dir:`literalinclude` or :ref:`kernel-include ` +directive (latter one expands environment variables in the path name). + +.. _reST literal: + +``::`` +------ + +.. code:: reST + + :: + + Literal block + + Lorem ipsum dolor:: + + Literal block + + Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy + eirmod tempor invidunt ut labore :: + + Literal block + +.. admonition:: Literal block + :class: rst-example + + :: + + Literal block + + Lorem ipsum dolor:: + + Literal block + + Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy + eirmod tempor invidunt ut labore :: + + Literal block + + +.. _reST code: + +``code-block`` +-------------- + +.. _pygments: https://pygments.org/languages/ + +.. sidebar:: Syntax highlighting + + is handled by pygments_. + +The :rst:dir:`code-block` directive is a variant of the :dudir:`code` directive +with additional options. To learn more about code literals visit +:ref:`sphinx:code-examples`. + +.. code-block:: reST + + The URL ``/stats`` handle is shown in :ref:`stats-handle` + + .. code-block:: Python + :caption: python code block + :name: stats-handle + + @app.route('/stats', methods=['GET']) + def stats(): + """Render engine statistics page.""" + stats = get_engines_stats() + return render( + 'stats.html' + , stats = stats ) + +.. code-block:: reST + +.. admonition:: Code block + :class: rst-example + + The URL ``/stats`` handle is shown in :ref:`stats-handle` + + .. code-block:: Python + :caption: python code block + :name: stats-handle + + @app.route('/stats', methods=['GET']) + def stats(): + """Render engine statistics page.""" + stats = get_engines_stats() + return render( + 'stats.html' + , stats = stats ) + +Unicode substitution +==================== + +The :dudir:`unicode directive ` converts Unicode +character codes (numerical values) to characters. This directive can only be +used within a substitution definition. + +.. code-block:: reST + + .. |copy| unicode:: 0xA9 .. copyright sign + .. |(TM)| unicode:: U+2122 + + Trademark |(TM)| and copyright |copy| glyphs. + +.. admonition:: Unicode + :class: rst-example + + .. |copy| unicode:: 0xA9 .. copyright sign + .. |(TM)| unicode:: U+2122 + + Trademark |(TM)| and copyright |copy| glyphs. + + +.. _reST roles: + +Roles +===== + +.. sidebar:: Further reading + + - `Sphinx Roles`_ + - :doc:`sphinx:usage/restructuredtext/domains` + + +A *custom interpreted text role* (:duref:`ref `) is an inline piece of +explicit markup. It signifies that that the enclosed text should be interpreted +in a specific way. + +The general markup is one of: + +.. code:: reST + + :rolename:`ref-name` + :rolename:`ref text ` + +.. table:: smart refs with sphinx.ext.extlinks_ and intersphinx_ + :widths: 4 3 7 + + ========================== ================================== ==================================== + role rendered example markup + ========================== ================================== ==================================== + :rst:role:`guilabel` :guilabel:`&Cancel` ``:guilabel:`&Cancel``` + :rst:role:`kbd` :kbd:`C-x C-f` ``:kbd:`C-x C-f``` + :rst:role:`menuselection` :menuselection:`Open --> File` ``:menuselection:`Open --> File``` + :rst:role:`download` :download:`this file ` ``:download:`this file ``` + math_ :math:`a^2 + b^2 = c^2` ``:math:`a^2 + b^2 = c^2``` + :rst:role:`ref` :ref:`svg image example` ``:ref:`svg image example``` + :rst:role:`command` :command:`ls -la` ``:command:`ls -la``` + :durole:`emphasis` :emphasis:`italic` ``:emphasis:`italic``` + :durole:`strong` :strong:`bold` ``:strong:`bold``` + :durole:`literal` :literal:`foo()` ``:literal:`foo()``` + :durole:`subscript` H\ :sub:`2`\ O ``H\ :sub:`2`\ O`` + :durole:`superscript` E = mc\ :sup:`2` ``E = mc\ :sup:`2``` + :durole:`title-reference` :title:`Time` ``:title:`Time``` + ========================== ================================== ==================================== + +Figures & Images +================ + +.. sidebar:: Image processing + + With the directives from :ref:`linuxdoc ` the build process + is flexible. To get best results in the generated output format, install + ImageMagick_ and Graphviz_. + +Searx's sphinx setup includes: :ref:`linuxdoc:kfigure`. Scalable here means; +scalable in sense of the build process. Normally in absence of a converter +tool, the build process will break. From the authors POV it’s annoying to care +about the build process when handling with images, especially since he has no +access to the build process. With :ref:`linuxdoc:kfigure` the build process +continues and scales output quality in dependence of installed image processors. + +If you want to add an image, you should use the ``kernel-figure`` (inheritance +of :dudir:`figure`) and ``kernel-image`` (inheritance of :dudir:`image`) +directives. E.g. to insert a figure with a scalable image format use SVG +(:ref:`svg image example`): + +.. code:: reST + + .. _svg image example: + + .. kernel-figure:: svg_image.svg + :alt: SVG image example + + Simple SVG image + + To refer the figure, a caption block is needed: :ref:`svg image example`. + +.. _svg image example: + +.. kernel-figure:: svg_image.svg + :alt: SVG image example + + Simple SVG image. + +To refer the figure, a caption block is needed: :ref:`svg image example`. + +DOT files (aka Graphviz) +------------------------ + +With :ref:`linuxdoc:kernel-figure` reST support for **DOT** formatted files is +given. + +- `Graphviz's dot`_ +- DOT_ +- Graphviz_ + +A simple example is shown in :ref:`dot file example`: + +.. code:: reST + + .. _dot file example: + + .. kernel-figure:: hello.dot + :alt: hello world + + DOT's hello world example + +.. admonition:: hello.dot + :class: rst-example + + .. _dot file example: + + .. kernel-figure:: hello.dot + :alt: hello world + + DOT's hello world example + +``kernel-render`` DOT +--------------------- + +Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the +:ref:`linuxdoc:kernel-render` directive. A simple example of embedded DOT_ is +shown in figure :ref:`dot render example`: + +.. code:: reST + + .. _dot render example: + + .. kernel-render:: DOT + :alt: digraph + :caption: Embedded DOT (Graphviz) code + + digraph foo { + "bar" -> "baz"; + } + + Attribute ``caption`` is needed, if you want to refer the figure: :ref:`dot + render example`. + +Please note :ref:`build tools `. If Graphviz_ is +installed, you will see an vector image. If not, the raw markup is inserted as +*literal-block*. + +.. admonition:: kernel-render DOT + :class: rst-example + + .. _dot render example: + + .. kernel-render:: DOT + :alt: digraph + :caption: Embedded DOT (Graphviz) code + + digraph foo { + "bar" -> "baz"; + } + + Attribute ``caption`` is needed, if you want to refer the figure: :ref:`dot + render example`. + +``kernel-render`` SVG +--------------------- + +A simple example of embedded SVG_ is shown in figure :ref:`svg render example`: + +.. code:: reST + + .. _svg render example: + + .. kernel-render:: SVG + :caption: Embedded **SVG** markup + :alt: so-nw-arrow +.. + + .. code:: xml + + + + + + + +.. admonition:: kernel-render SVG + :class: rst-example + + .. _svg render example: + + .. kernel-render:: SVG + :caption: Embedded **SVG** markup + :alt: so-nw-arrow + + + + + + + + + + +.. _reST lists: + +List markups +============ + +Bullet list +----------- + +List markup (:duref:`ref `) is simple: + +.. code:: reST + + - This is a bulleted list. + + 1. Nested lists are possible, but be aware that they must be separated from + the parent list items by blank line + 2. Second item of nested list + + - It has two items, the second + item uses two lines. + + #. This is a numbered list. + #. It has two items too. + +.. admonition:: bullet list + :class: rst-example + + - This is a bulleted list. + + 1. Nested lists are possible, but be aware that they must be separated from + the parent list items by blank line + 2. Second item of nested list + + - It has two items, the second + item uses two lines. + + #. This is a numbered list. + #. It has two items too. + + +Horizontal list +--------------- + +The :rst:dir:`.. hlist:: ` transforms a bullet list into a more compact +list. + +.. code:: reST + + .. hlist:: + + - first list item + - second list item + - third list item + ... + +.. admonition:: hlist + :class: rst-example + + .. hlist:: + + - first list item + - second list item + - third list item + - next list item + - next list item xxxx + - next list item yyyy + - next list item zzzz + + +Definition list +--------------- + +.. sidebar:: Note .. + + - the term cannot have more than one line of text + + - there is **no blank line between term and definition block** // this + distinguishes definition lists (:duref:`ref `) from block + quotes (:duref:`ref `). + +Each definition list (:duref:`ref `) item contains a term, +optional classifiers and a definition. A term is a simple one-line word or +phrase. Optional classifiers may follow the term on the same line, each after +an inline ' : ' (**space, colon, space**). A definition is a block indented +relative to the term, and may contain multiple paragraphs and other body +elements. There may be no blank line between a term line and a definition block +(*this distinguishes definition lists from block quotes*). Blank lines are +required before the first and after the last definition list item, but are +optional in-between. + +Definition lists are created as follows: + +.. code:: reST + + term 1 (up to a line of text) + Definition 1. + + See the typo : this line is not a term! + + And this is not term's definition. **There is a blank line** in between + the line above and this paragraph. That's why this paragraph is taken as + **block quote** (:duref:`ref `) and not as term's definition! + + term 2 + Definition 2, paragraph 1. + + Definition 2, paragraph 2. + + term 3 : classifier + Definition 3. + + term 4 : classifier one : classifier two + Definition 4. + +.. admonition:: definition list + :class: rst-example + + term 1 (up to a line of text) + Definition 1. + + See the typo : this line is not a term! + + And this is not term's definition. **There is a blank line** in between + the line above and this paragraph. That's why this paragraph is taken as + **block quote** (:duref:`ref `) and not as term's definition! + + + term 2 + Definition 2, paragraph 1. + + Definition 2, paragraph 2. + + term 3 : classifier + Definition 3. + + term 4 : classifier one : classifier two + + +Quoted paragraphs +----------------- + +Quoted paragraphs (:duref:`ref `) are created by just indenting +them more than the surrounding paragraphs. Line blocks (:duref:`ref +`) are a way of preserving line breaks: + +.. code:: reST + + normal paragraph ... + lorem ipsum. + + Quoted paragraph ... + lorem ipsum. + + | These lines are + | broken exactly like in + | the source file. + + +.. admonition:: Quoted paragraph and line block + :class: rst-example + + normal paragraph ... + lorem ipsum. + + Quoted paragraph ... + lorem ipsum. + + | These lines are + | broken exactly like in + | the source file. + + +.. _reST field list: + +Field Lists +----------- + +.. _Sphinx Field Lists: + https://www.sphinx-doc.org/en/master/usage/restructuredtext/field-lists.html + +.. sidebar:: bibliographic fields + + First lines fields are bibliographic fields, see `Sphinx Field Lists`_. + +Field lists are used as part of an extension syntax, such as options for +directives, or database-like records meant for further processing. Field lists +are mappings from field names to field bodies. They marked up like this: + +.. code:: reST + + :fieldname: Field content + :foo: first paragraph in field foo + + second paragraph in field foo + + :bar: Field content + +.. admonition:: Field List + :class: rst-example + + :fieldname: Field content + :foo: first paragraph in field foo + + second paragraph in field foo + + :bar: Field content + + +They are commonly used in Python documentation: + +.. code:: python + + def my_function(my_arg, my_other_arg): + """A function just for me. + + :param my_arg: The first of my arguments. + :param my_other_arg: The second of my arguments. + + :returns: A message (just for me, of course). + """ + +Further list blocks +------------------- + +- field lists (:duref:`ref `, with caveats noted in + :ref:`reST field list`) +- option lists (:duref:`ref `) +- quoted literal blocks (:duref:`ref `) +- doctest blocks (:duref:`ref `) + + +Admonitions +=========== + +Sidebar +------- + +Sidebar is an eye catcher, often used for admonitions pointing further stuff or +site effects. Here is the source of the sidebar :ref:`on top of this page `. + +.. code:: reST + + .. sidebar:: KISS_ and readability_ + + Instead of defining more and more roles, we at SearXNG encourage our + contributors to follow principles like KISS_ and readability_. + +Generic admonition +------------------ + +The generic :dudir:`admonition ` needs a title: + +.. code:: reST + + .. admonition:: generic admonition title + + lorem ipsum .. + + +.. admonition:: generic admonition title + + lorem ipsum .. + + +Specific admonitions +-------------------- + +Specific admonitions: :dudir:`hint`, :dudir:`note`, :dudir:`tip` :dudir:`attention`, +:dudir:`caution`, :dudir:`danger`, :dudir:`error`, , :dudir:`important`, and +:dudir:`warning` . + +.. code:: reST + + .. hint:: + + lorem ipsum .. + + .. note:: + + lorem ipsum .. + + .. warning:: + + lorem ipsum .. + + +.. hint:: + + lorem ipsum .. + +.. note:: + + lorem ipsum .. + +.. tip:: + + lorem ipsum .. + +.. attention:: + + lorem ipsum .. + +.. caution:: + + lorem ipsum .. + +.. danger:: + + lorem ipsum .. + +.. important:: + + lorem ipsum .. + +.. error:: + + lorem ipsum .. + +.. warning:: + + lorem ipsum .. + + +Tables +====== + +.. sidebar:: Nested tables + + Nested tables are ugly! Not all builder support nested tables, don't use + them! + +ASCII-art tables like :ref:`reST simple table` and :ref:`reST grid table` might +be comfortable for readers of the text-files, but they have huge disadvantages +in the creation and modifying. First, they are hard to edit. Think about +adding a row or a column to a ASCII-art table or adding a paragraph in a cell, +it is a nightmare on big tables. + + +.. sidebar:: List tables + + For meaningful patch and diff use :ref:`reST flat table`. + +Second the diff of modifying ASCII-art tables is not meaningful, e.g. widening a +cell generates a diff in which also changes are included, which are only +ascribable to the ASCII-art. Anyway, if you prefer ASCII-art for any reason, +here are some helpers: + +* `Emacs Table Mode`_ +* `Online Tables Generator`_ + +.. _reST simple table: + +Simple tables +------------- + +:duref:`Simple tables ` allow *colspan* but not *rowspan*. If +your table need some metadata (e.g. a title) you need to add the ``.. table:: +directive`` :dudir:`(ref)
` in front and place the table in its body: + +.. code:: reST + + .. table:: foo gate truth table + :widths: grid + :align: left + + ====== ====== ====== + Inputs Output + ------------- ------ + A B A or B + ====== ====== ====== + False + -------------------- + True + -------------------- + True False True + (foo) + ------ ------ ------ + False True + (foo) + ====== ============= + +.. admonition:: Simple ASCII table + :class: rst-example + + .. table:: foo gate truth table + :widths: grid + :align: left + + ====== ====== ====== + Inputs Output + ------------- ------ + A B A or B + ====== ====== ====== + False + -------------------- + True + -------------------- + True False True + (foo) + ------ ------ ------ + False True + (foo) + ====== ============= + + + +.. _reST grid table: + +Grid tables +----------- + +:duref:`Grid tables ` allow colspan *colspan* and *rowspan*: + +.. code:: reST + + .. table:: grid table example + :widths: 1 1 5 + + +------------+------------+-----------+ + | Header 1 | Header 2 | Header 3 | + +============+============+===========+ + | body row 1 | column 2 | column 3 | + +------------+------------+-----------+ + | body row 2 | Cells may span columns.| + +------------+------------+-----------+ + | body row 3 | Cells may | - Cells | + +------------+ span rows. | - contain | + | body row 4 | | - blocks. | + +------------+------------+-----------+ + +.. admonition:: ASCII grid table + :class: rst-example + + .. table:: grid table example + :widths: 1 1 5 + + +------------+------------+-----------+ + | Header 1 | Header 2 | Header 3 | + +============+============+===========+ + | body row 1 | column 2 | column 3 | + +------------+------------+-----------+ + | body row 2 | Cells may span columns.| + +------------+------------+-----------+ + | body row 3 | Cells may | - Cells | + +------------+ span rows. | - contain | + | body row 4 | | - blocks. | + +------------+------------+-----------+ + + +.. _reST flat table: + +flat-table +---------- + +The ``flat-table`` is a further developed variant of the :ref:`list tables +`. It is a double-stage list similar to the +:dudir:`list-table` with some additional features: + +column-span: ``cspan`` + with the role ``cspan`` a cell can be extended through additional columns + +row-span: ``rspan`` + with the role ``rspan`` a cell can be extended through additional rows + +auto-span: + spans rightmost cell of a table row over the missing cells on the right side + of that table-row. With Option ``:fill-cells:`` this behavior can changed + from *auto span* to *auto fill*, which automatically inserts (empty) cells + instead of spanning the last cell. + +options: + :header-rows: [int] count of header rows + :stub-columns: [int] count of stub columns + :widths: [[int] [int] ... ] widths of columns + :fill-cells: instead of auto-span missing cells, insert missing cells + +roles: + :cspan: [int] additional columns (*morecols*) + :rspan: [int] additional rows (*morerows*) + +The example below shows how to use this markup. The first level of the staged +list is the *table-row*. In the *table-row* there is only one markup allowed, +the list of the cells in this *table-row*. Exception are *comments* ( ``..`` ) +and *targets* (e.g. a ref to :ref:`row 2 of table's body `). + +.. code:: reST + + .. flat-table:: ``flat-table`` example + :header-rows: 2 + :stub-columns: 1 + :widths: 1 1 1 1 2 + + * - :rspan:`1` head / stub + - :cspan:`3` head 1.1-4 + + * - head 2.1 + - head 2.2 + - head 2.3 + - head 2.4 + + * .. row body 1 / this is a comment + + - row 1 + - :rspan:`2` cell 1-3.1 + - cell 1.2 + - cell 1.3 + - cell 1.4 + + * .. Comments and targets are allowed on *table-row* stage. + .. _`row body 2`: + + - row 2 + - cell 2.2 + - :rspan:`1` :cspan:`1` + cell 2.3 with a span over + + * col 3-4 & + * row 2-3 + + * - row 3 + - cell 3.2 + + * - row 4 + - cell 4.1 + - cell 4.2 + - cell 4.3 + - cell 4.4 + + * - row 5 + - cell 5.1 with automatic span to right end + + * - row 6 + - cell 6.1 + - .. + + +.. admonition:: List table + :class: rst-example + + .. flat-table:: ``flat-table`` example + :header-rows: 2 + :stub-columns: 1 + :widths: 1 1 1 1 2 + + * - :rspan:`1` head / stub + - :cspan:`3` head 1.1-4 + + * - head 2.1 + - head 2.2 + - head 2.3 + - head 2.4 + + * .. row body 1 / this is a comment + + - row 1 + - :rspan:`2` cell 1-3.1 + - cell 1.2 + - cell 1.3 + - cell 1.4 + + * .. Comments and targets are allowed on *table-row* stage. + .. _`row body 2`: + + - row 2 + - cell 2.2 + - :rspan:`1` :cspan:`1` + cell 2.3 with a span over + + * col 3-4 & + * row 2-3 + + * - row 3 + - cell 3.2 + + * - row 4 + - cell 4.1 + - cell 4.2 + - cell 4.3 + - cell 4.4 + + * - row 5 + - cell 5.1 with automatic span to right end + + * - row 6 + - cell 6.1 + - .. + + +CSV table +--------- + +CSV table might be the choice if you want to include CSV-data from a outstanding +(build) process into your documentation. + +.. code:: reST + + .. csv-table:: CSV table example + :header: .. , Column 1, Column 2 + :widths: 2 5 5 + :stub-columns: 1 + :file: csv_table.txt + +Content of file ``csv_table.txt``: + +.. literalinclude:: csv_table.txt + +.. admonition:: CSV table + :class: rst-example + + .. csv-table:: CSV table example + :header: .. , Column 1, Column 2 + :widths: 3 5 5 + :stub-columns: 1 + :file: csv_table.txt + +Templating +========== + +.. sidebar:: Build environment + + All *generic-doc* tasks are running in the :ref:`make install`. + +Templating is suitable for documentation which is created generic at the build +time. The sphinx-jinja_ extension evaluates jinja_ templates in the :ref:`make +install` (with SearXNG modules installed). We use this e.g. to build chapter: +:ref:`configured engines`. Below the jinja directive from the +:origin:`docs/admin/engines.rst` is shown: + +.. literalinclude:: ../admin/engines/configured_engines.rst + :language: reST + :start-after: .. _configured engines: + +The context for the template is selected in the line ``.. jinja:: searx``. In +sphinx's build configuration (:origin:`docs/conf.py`) the ``searx`` context +contains the ``engines`` and ``plugins``. + +.. code:: py + + import searx.search + import searx.engines + import searx.plugins + searx.search.initialize() + jinja_contexts = { + 'searx': { + 'engines': searx.engines.engines, + 'plugins': searx.plugins.plugins + }, + } + + +Tabbed views +============ + +.. _sphinx-tabs: https://github.com/djungelorm/sphinx-tabs +.. _basic-tabs: https://github.com/djungelorm/sphinx-tabs#basic-tabs +.. _group-tabs: https://github.com/djungelorm/sphinx-tabs#group-tabs +.. _code-tabs: https://github.com/djungelorm/sphinx-tabs#code-tabs + +With `sphinx-tabs`_ extension we have *tabbed views*. To provide installation +instructions with one tab per distribution we use the `group-tabs`_ directive, +others are basic-tabs_ and code-tabs_. Below a *group-tab* example from +:ref:`docs build` is shown: + +.. literalinclude:: ../admin/buildhosts.rst + :language: reST + :start-after: .. SNIP sh lint requirements + :end-before: .. SNAP sh lint requirements + +.. _math: + +Math equations +============== + +.. _Mathematics: https://en.wikibooks.org/wiki/LaTeX/Mathematics +.. _amsmath user guide: + http://vesta.informatik.rwth-aachen.de/ftp/pub/mirror/ctan/macros/latex/required/amsmath/amsldoc.pdf + +.. sidebar:: About LaTeX + + - `amsmath user guide`_ + - Mathematics_ + - :ref:`docs build` + +The input language for mathematics is LaTeX markup using the :ctan:`amsmath` +package. + +To embed LaTeX markup in reST documents, use role :rst:role:`:math: ` for +inline and directive :rst:dir:`.. math:: ` for block markup. + +.. code:: reST + + In :math:numref:`schroedinger general` the time-dependent Schrödinger equation + is shown. + + .. math:: + :label: schroedinger general + + \mathrm{i}\hbar\dfrac{\partial}{\partial t} |\,\psi (t) \rangle = + \hat{H} |\,\psi (t) \rangle. + +.. admonition:: LaTeX math equation + :class: rst-example + + In :math:numref:`schroedinger general` the time-dependent Schrödinger equation + is shown. + + .. math:: + :label: schroedinger general + + \mathrm{i}\hbar\dfrac{\partial}{\partial t} |\,\psi (t) \rangle = + \hat{H} |\,\psi (t) \rangle. + + +The next example shows the difference of ``\tfrac`` (*textstyle*) and ``\dfrac`` +(*displaystyle*) used in a inline markup or another fraction. + +.. code:: reST + + ``\tfrac`` **inline example** :math:`\tfrac{\tfrac{1}{x}+\tfrac{1}{y}}{y-z}` + ``\dfrac`` **inline example** :math:`\dfrac{\dfrac{1}{x}+\dfrac{1}{y}}{y-z}` + +.. admonition:: Line spacing + :class: rst-example + + Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy + eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam + voluptua. ... + ``\tfrac`` **inline example** :math:`\tfrac{\tfrac{1}{x}+\tfrac{1}{y}}{y-z}` + At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd + gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. + + Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy + eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam + voluptua. ... + ``\tfrac`` **inline example** :math:`\dfrac{\dfrac{1}{x}+\dfrac{1}{y}}{y-z}` + At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd + gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. + + +.. _KISS: https://en.wikipedia.org/wiki/KISS_principle + +.. _readability: https://docs.python-guide.org/writing/style/ +.. _Sphinx-Primer: + https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html +.. _reST: https://docutils.sourceforge.io/rst.html +.. _Sphinx Roles: + https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html +.. _Sphinx: https://www.sphinx-doc.org +.. _`sphinx-doc FAQ`: https://www.sphinx-doc.org/en/stable/faq.html +.. _Sphinx markup constructs: + https://www.sphinx-doc.org/en/stable/markup/index.html +.. _`sphinx cross references`: + https://www.sphinx-doc.org/en/stable/markup/inline.html#cross-referencing-arbitrary-locations +.. _sphinx.ext.extlinks: + https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html +.. _intersphinx: https://www.sphinx-doc.org/en/stable/ext/intersphinx.html +.. _sphinx config: https://www.sphinx-doc.org/en/stable/config.html +.. _Sphinx's autodoc: https://www.sphinx-doc.org/en/stable/ext/autodoc.html +.. _Sphinx's Python domain: + https://www.sphinx-doc.org/en/stable/domains.html#the-python-domain +.. _Sphinx's C domain: + https://www.sphinx-doc.org/en/stable/domains.html#cross-referencing-c-constructs +.. _doctree: + https://www.sphinx-doc.org/en/master/extdev/tutorial.html?highlight=doctree#build-phases +.. _docutils: http://docutils.sourceforge.net/docs/index.html +.. _docutils FAQ: http://docutils.sourceforge.net/FAQ.html +.. _linuxdoc: https://return42.github.io/linuxdoc +.. _jinja: https://jinja.palletsprojects.com/ +.. _sphinx-jinja: https://github.com/tardyp/sphinx-jinja +.. _SVG: https://www.w3.org/TR/SVG11/expanded-toc.html +.. _DOT: https://graphviz.gitlab.io/_pages/doc/info/lang.html +.. _`Graphviz's dot`: https://graphviz.gitlab.io/_pages/pdf/dotguide.pdf +.. _Graphviz: https://graphviz.gitlab.io +.. _ImageMagick: https://www.imagemagick.org + +.. _`Emacs Table Mode`: https://www.emacswiki.org/emacs/TableMode +.. _`Online Tables Generator`: https://www.tablesgenerator.com/text_tables +.. _`OASIS XML Exchange Table Model`: https://www.oasis-open.org/specs/tm9901.html diff --git a/_sources/dev/search_api.rst.txt b/_sources/dev/search_api.rst.txt new file mode 100644 index 00000000..2333474a --- /dev/null +++ b/_sources/dev/search_api.rst.txt @@ -0,0 +1,126 @@ +.. _search API: + +========== +Search API +========== + +The search supports both ``GET`` and ``POST``. + +Furthermore, two endpoints ``/`` and ``/search`` are available for querying. + + +``GET /`` + +``GET /search`` + +Parameters +========== + +.. sidebar:: Further reading .. + + - :ref:`engines-dev` + - :ref:`settings.yml` + - :ref:`configured engines` + +``q`` : required + The search query. This string is passed to external search services. Thus, + SearXNG supports syntax of each search service. For example, ``site:github.com + SearXNG`` is a valid query for Google. However, if simply the query above is + passed to any search engine which does not filter its results based on this + syntax, you might not get the results you wanted. + + See more at :ref:`search-syntax` + +``categories`` : optional + Comma separated list, specifies the active search categories (see + :ref:`configured engines`) + +``engines`` : optional + Comma separated list, specifies the active search engines (see + :ref:`configured engines`). + +``language`` : default from :ref:`settings search` + Code of the language. + +``pageno`` : default ``1`` + Search page number. + +``time_range`` : optional + [ ``day``, ``month``, ``year`` ] + + Time range of search for engines which support it. See if an engine supports + time range search in the preferences page of an instance. + +``format`` : optional + [ ``json``, ``csv``, ``rss`` ] + + Output format of results. Format needs to be activated in :ref:`settings + search`. + +``results_on_new_tab`` : default ``0`` + [ ``0``, ``1`` ] + + Open search results on new tab. + +``image_proxy`` : default from :ref:`settings server` + [ ``True``, ``False`` ] + + Proxy image results through SearXNG. + +``autocomplete`` : default from :ref:`settings search` + [ ``google``, ``dbpedia``, ``duckduckgo``, ``startpage``, ``wikipedia``, + ``swisscows``, ``qwant`` ] + + Service which completes words as you type. + +``safesearch`` : default from :ref:`settings search` + [ ``0``, ``1``, ``2`` ] + + Filter search results of engines which support safe search. See if an engine + supports safe search in the preferences page of an instance. + +``theme`` : default ``simple`` + [ ``simple`` ] + + Theme of instance. + + Please note, available themes depend on an instance. It is possible that an + instance administrator deleted, created or renamed themes on their instance. + See the available options in the preferences page of the instance. + +``enabled_plugins`` : optional + List of enabled plugins. + + :default: + ``Hash_plugin``, ``Search_on_category_select``, + ``Self_Information``, ``Tracker_URL_remover``, + ``Ahmia_blacklist`` + + :values: + .. enabled by default + + ``Hash_plugin``, ``Search_on_category_select``, + ``Self_Information``, ``Tracker_URL_remover``, + ``Ahmia_blacklist``, + + .. disabled by default + + ``Hostname_replace``, ``Open_Access_DOI_rewrite``, + ``Vim-like_hotkeys``, ``Tor_check_plugin`` + +``disabled_plugins``: optional + List of disabled plugins. + + :default: + ``Hostname_replace``, ``Open_Access_DOI_rewrite``, + ``Vim-like_hotkeys``, ``Tor_check_plugin`` + + :values: + see values from ``enabled_plugins`` + +``enabled_engines`` : optional : *all* :origin:`engines ` + List of enabled engines. + +``disabled_engines`` : optional : *all* :origin:`engines ` + List of disabled engines. + diff --git a/_sources/dev/searxng_extra/index.rst.txt b/_sources/dev/searxng_extra/index.rst.txt new file mode 100644 index 00000000..c2b5c312 --- /dev/null +++ b/_sources/dev/searxng_extra/index.rst.txt @@ -0,0 +1,15 @@ +.. _searxng_extra: + +============================= +Tooling box ``searxng_extra`` +============================= + +In the folder :origin:`searxng_extra/` we maintain some tools useful for CI and +developers. + +.. toctree:: + :maxdepth: 2 + :caption: Contents + + update + standalone_searx.py diff --git a/_sources/dev/searxng_extra/standalone_searx.py.rst.txt b/_sources/dev/searxng_extra/standalone_searx.py.rst.txt new file mode 100644 index 00000000..7cbbccfd --- /dev/null +++ b/_sources/dev/searxng_extra/standalone_searx.py.rst.txt @@ -0,0 +1,9 @@ + +.. _standalone_searx.py: + +===================================== +``searxng_extra/standalone_searx.py`` +===================================== + +.. automodule:: searxng_extra.standalone_searx + :members: diff --git a/_sources/dev/searxng_extra/update.rst.txt b/_sources/dev/searxng_extra/update.rst.txt new file mode 100644 index 00000000..d05c8140 --- /dev/null +++ b/_sources/dev/searxng_extra/update.rst.txt @@ -0,0 +1,88 @@ +========================= +``searxng_extra/update/`` +========================= + +:origin:`[source] ` + +Scripts to update static data in :origin:`searx/data/` + +.. _update_ahmia_blacklist.py: + +``update_ahmia_blacklist.py`` +============================= + +:origin:`[source] ` + +.. automodule:: searxng_extra.update.update_ahmia_blacklist + :members: + + +``update_currencies.py`` +======================== + +:origin:`[source] ` + +.. automodule:: searxng_extra.update.update_currencies + :members: + +``update_engine_descriptions.py`` +================================= + +:origin:`[source] ` + +.. automodule:: searxng_extra.update.update_engine_descriptions + :members: + + +``update_external_bangs.py`` +============================ + +:origin:`[source] ` + +.. automodule:: searxng_extra.update.update_external_bangs + :members: + + +``update_firefox_version.py`` +============================= + +:origin:`[source] ` + +.. automodule:: searxng_extra.update.update_firefox_version + :members: + + +``update_languages.py`` +======================= + +:origin:`[source] ` + +.. automodule:: searxng_extra.update.update_languages + :members: + + +``update_osm_keys_tags.py`` +=========================== + +:origin:`[source] ` + +.. automodule:: searxng_extra.update.update_osm_keys_tags + :members: + + +``update_pygments.py`` +====================== + +:origin:`[source] ` + +.. automodule:: searxng_extra.update.update_pygments + :members: + + +``update_wikidata_units.py`` +============================ + +:origin:`[source] ` + +.. automodule:: searxng_extra.update.update_wikidata_units + :members: diff --git a/_sources/dev/translation.rst.txt b/_sources/dev/translation.rst.txt new file mode 100644 index 00000000..693f6d2d --- /dev/null +++ b/_sources/dev/translation.rst.txt @@ -0,0 +1,81 @@ +.. _translation: + +=========== +Translation +=========== + +.. _translate.codeberg.org: https://translate.codeberg.org/projects/searxng/ +.. _Weblate: https://docs.weblate.org +.. _translations branch: https://github.com/searxng/searxng/tree/translations +.. _orphan branch: https://git-scm.com/docs/git-checkout#Documentation/git-checkout.txt---orphanltnewbranchgt +.. _Weblate repository: https://translate.codeberg.org/projects/searxng/searxng/#repository +.. _wlc: https://docs.weblate.org/en/latest/wlc.html + +.. |translated| image:: https://translate.codeberg.org/widgets/searxng/-/searxng/svg-badge.svg + :target: https://translate.codeberg.org/projects/searxng/ + +.. sidebar:: |translated| + + - :ref:`searx.babel_extract` + - Weblate_ + - SearXNG `translations branch`_ + - SearXNG `Weblate repository`_ + - Weblate Client: wlc_ + - Babel Command-Line: `pybabel `_ + - `weblate workflow `_ + +Translation takes place on translate.codeberg.org_. + +Translations which has been added by translators on the translate.codeberg.org_ UI are +committed to Weblate's counterpart of the SearXNG *origin* repository which is +located at ``https://translate.codeberg.org/git/searxng/searxng``. + +There is no need to clone this repository, :ref:`SearXNG Weblate workflow` take +care of the synchronization with the *origin*. To avoid merging commits from +the counterpart directly on the ``master`` branch of *SearXNG origin*, a *pull +request* (PR) is created by this workflow. + +Weblate monitors the `translations branch`_, not the ``master`` branch. This +branch is an `orphan branch`_, decoupled from the master branch (we already know +orphan branches from the ``gh-pages``). The `translations branch`_ contains +only the + +- ``translation/messages.pot`` and the +- ``translation/*/messages.po`` files, nothing else. + + +.. _SearXNG Weblate workflow: + +.. figure:: translation.svg + + SearXNG's PR workflow to be in sync with Weblate + +Sync from *origin* to *weblate*: using ``make weblate.push.translations`` + For each commit on the ``master`` branch of SearXNG *origin* the GitHub job + :origin:`babel / Update translations branch + <.github/workflows/integration.yml>` checks for updated translations. + +Sync from *weblate* to *origin*: using ``make weblate.translations.commit`` + Every Friday, the GitHub workflow :origin:`babel / create PR for additons from + weblate <.github/workflows/translations-update.yml>` creates a PR with the + updated translation files: + + - ``translation/messages.pot``, + - ``translation/*/messages.po`` and + - ``translation/*/messages.mo`` + +wlc +=== + +.. _wlc configuration: https://docs.weblate.org/en/latest/wlc.html#wlc-config +.. _API key: https://translate.codeberg.org/accounts/profile/#api + +All weblate integration is done by GitHub workflows, but if you want to use wlc_, +copy this content into `wlc configuration`_ in your HOME ``~/.config/weblate`` + +.. code-block:: ini + + [keys] + https://translate.codeberg.org/api/ = APIKEY + +Replace ``APIKEY`` by your `API key`_. diff --git a/_sources/donate.rst.txt b/_sources/donate.rst.txt new file mode 100644 index 00000000..b68ed807 --- /dev/null +++ b/_sources/donate.rst.txt @@ -0,0 +1,40 @@ +Donate to searxng.org +===================== + +Why donating? +------------- + +If you want to support the SearXNG team you can make a donation. + +This will help us to pay the costs for: + +- the two VPS servers for searx.space +- the domain names (searxng.org and searx.space) +- the protonmail account + +If there is enough fund we can ask for a security audit or pay an User Experience (UX) designer. + +Payment methods +--------------- + +- Credit/debit card and bank transfer + + - `Liberapay`_ (recurrent donation) + - `Buy Me a Coffee`_ (one time donation) + +- Cryptocurrency + + - Bitcoin: `bc1qn3rw8t86h05cs3grx2kmwmptw9k4kt4hyzktqj`_ (Segwit + compatible) + - Bitcoin cash: `qpead2yu482e3h9amy5zk45l8qrfhk59jcpw3cth9e`_ + - Ethereum: `0xCf82c7eb915Ee70b5B69C1bBB5525e157F35FA43`_ + - Dogecoin: `DBCYS9issTt84pddXSsTHpQxyQDTFp1TE4`_ + - Litecoin: `ltc1q5j6x6f4f2htldhq570e353clc8fmw44ra5er5q`_ + +.. _Liberapay: https://liberapay.com/SearXNG/ +.. _Buy Me a Coffee: https://buymeacoffee.com/searxng +.. _bc1qn3rw8t86h05cs3grx2kmwmptw9k4kt4hyzktqj: bitcoin:bc1qn3rw8t86h05cs3grx2kmwmptw9k4kt4hyzktqj +.. _qpead2yu482e3h9amy5zk45l8qrfhk59jcpw3cth9e: bitcoincash:qpead2yu482e3h9amy5zk45l8qrfhk59jcpw3cth9e +.. _0xCf82c7eb915Ee70b5B69C1bBB5525e157F35FA43: ethereum:0xCf82c7eb915Ee70b5B69C1bBB5525e157F35FA43 +.. _DBCYS9issTt84pddXSsTHpQxyQDTFp1TE4: dogecoin:DBCYS9issTt84pddXSsTHpQxyQDTFp1TE4 +.. _ltc1q5j6x6f4f2htldhq570e353clc8fmw44ra5er5q: litecoin:ltc1q5j6x6f4f2htldhq570e353clc8fmw44ra5er5q diff --git a/_sources/index.rst.txt b/_sources/index.rst.txt new file mode 100644 index 00000000..f331e321 --- /dev/null +++ b/_sources/index.rst.txt @@ -0,0 +1,41 @@ +================== +Welcome to SearXNG +================== + + *Search without being tracked.* + +SearXNG is a free internet metasearch engine which aggregates results from more +than 70 search services. Users are neither tracked nor profiled. Additionally, +SearXNG can be used over Tor for online anonymity. + +Get started with SearXNG by using one of the instances listed at searx.space_. +If you don't trust anyone, you can set up your own, see :ref:`installation`. + +.. sidebar:: Features + + - Self hosted + - No user tracking + - No user profiling + - About 70 supported search engines + - Easy integration with any search engine + - Cookies are not used by default + - Secure, encrypted connections (HTTPS/SSL) + +.. sidebar:: info + + SearXNG development has been started in the middle of 2021 as a fork of the + searx project. + +.. toctree:: + :maxdepth: 2 + :caption: Contents + + user/index + own-instance + admin/index + dev/index + utils/index + src/index + donate + +.. _searx.space: https://searx.space diff --git a/_sources/own-instance.rst.txt b/_sources/own-instance.rst.txt new file mode 100644 index 00000000..c53f2830 --- /dev/null +++ b/_sources/own-instance.rst.txt @@ -0,0 +1,79 @@ +=========================== +Why use a private instance? +=========================== + + *"Is it worth to run my own instance?"* + +\.\. is a common question among SearXNG users. Before answering this question, +see what options a SearXNG user has. + +Public instances are open to everyone who has access to its URL. Usually, these +are operated by unknown parties (from the users' point of view). Private +instances can be used by a select group of people. It is for example a SearXNG of +group of friends or a company which can be accessed through VPN. Also it can be +single user one which runs on the user's laptop. + +To gain more insight on how these instances work let's dive into how SearXNG +protects its users. + +How does SearXNG protect privacy? +================================= + +SearXNG protects the privacy of its users in multiple ways regardless of the type +of the instance (private, public). Removal of private data from search requests +comes in three forms: + + 1. removal of private data from requests going to search services + 2. not forwarding anything from a third party services through search services + (e.g. advertisement) + 3. removal of private data from requests going to the result pages + +Removing private data means not sending cookies to external search engines and +generating a random browser profile for every request. Thus, it does not matter +if a public or private instance handles the request, because it is anonymized in +both cases. IP addresses will be the IP of the instance. But SearXNG can be +configured to use proxy or Tor. `Result proxy +`__ is supported, too. + +SearXNG does not serve ads or tracking content unlike most search services. So +private data is not forwarded to third parties who might monetize it. Besides +protecting users from search services, both referring page and search query are +hidden from visited result pages. + + +What are the consequences of using public instances? +---------------------------------------------------- + +If someone uses a public instance, they have to trust the administrator of that +instance. This means that the user of the public instance does not know whether +their requests are logged, aggregated and sent or sold to a third party. + +Also, public instances without proper protection are more vulnerable to abusing +the search service, In this case the external service in exchange returns +CAPTCHAs or bans the IP of the instance. Thus, search requests return less +results. + +I see. What about private instances? +------------------------------------ + +If users run their :ref:`own instances `, everything is in their +control: the source code, logging settings and private data. Unknown instance +administrators do not have to be trusted. + +Furthermore, as the default settings of their instance is editable, there is no +need to use cookies to tailor SearXNG to their needs. So preferences will not be +reset to defaults when clearing browser cookies. As settings are stored on +their computer, it will not be accessible to others as long as their computer is +not compromised. + +Conclusion +========== + +Always use an instance which is operated by people you trust. The privacy +features of SearXNG are available to users no matter what kind of instance they +use. + +If someone is on the go or just wants to try SearXNG for the first time public +instances are the best choices. Additionally, public instance are making a +world a better place, because those who cannot or do not want to run an +instance, have access to a privacy respecting search service. diff --git a/_sources/src/index.rst.txt b/_sources/src/index.rst.txt new file mode 100644 index 00000000..efa7a509 --- /dev/null +++ b/_sources/src/index.rst.txt @@ -0,0 +1,14 @@ +=========== +Source-Code +=========== + +This is a partial documentation of our source code. We are not aiming to document +every item from the source code, but we will add documentation when requested. + + +.. toctree:: + :maxdepth: 2 + :caption: Contents + :glob: + + searx.* diff --git a/_sources/src/searx.babel_extract.rst.txt b/_sources/src/searx.babel_extract.rst.txt new file mode 100644 index 00000000..741d67fc --- /dev/null +++ b/_sources/src/searx.babel_extract.rst.txt @@ -0,0 +1,8 @@ +.. _searx.babel_extract: + +=============================== +Custom message extractor (i18n) +=============================== + +.. automodule:: searx.babel_extract + :members: diff --git a/_sources/src/searx.engines.demo_offline.rst.txt b/_sources/src/searx.engines.demo_offline.rst.txt new file mode 100644 index 00000000..9424244f --- /dev/null +++ b/_sources/src/searx.engines.demo_offline.rst.txt @@ -0,0 +1,9 @@ +.. _demo offline engine: + +=================== +Demo Offline Engine +=================== + +.. automodule:: searx.engines.demo_offline + :members: + diff --git a/_sources/src/searx.engines.demo_online.rst.txt b/_sources/src/searx.engines.demo_online.rst.txt new file mode 100644 index 00000000..0a8c8e98 --- /dev/null +++ b/_sources/src/searx.engines.demo_online.rst.txt @@ -0,0 +1,9 @@ +.. _demo online engine: + +================== +Demo Online Engine +================== + +.. automodule:: searx.engines.demo_online + :members: + diff --git a/_sources/src/searx.engines.google.rst.txt b/_sources/src/searx.engines.google.rst.txt new file mode 100644 index 00000000..2d10b5ee --- /dev/null +++ b/_sources/src/searx.engines.google.rst.txt @@ -0,0 +1,55 @@ +.. _google engines: + +============== +Google Engines +============== + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + + +.. _google API: + +google API +========== + +.. _Query Parameter Definitions: + https://developers.google.com/custom-search/docs/xml_results#WebSearch_Query_Parameter_Definitions + +For detailed description of the *REST-full* API see: `Query Parameter +Definitions`_. Not all parameters can be appied and some engines are *special* +(e.g. :ref:`google news engine`). + +.. _google web engine: + +Google WEB +========== + +.. automodule:: searx.engines.google + :members: + +.. _google images engine: + +Google Images +============= + +.. automodule:: searx.engines.google_images + :members: + +.. _google videos engine: + +Google Videos +============= + +.. automodule:: searx.engines.google_videos + :members: + +.. _google news engine: + +Google News +=========== + +.. automodule:: searx.engines.google_news + :members: diff --git a/_sources/src/searx.engines.rst.txt b/_sources/src/searx.engines.rst.txt new file mode 100644 index 00000000..687fdb0b --- /dev/null +++ b/_sources/src/searx.engines.rst.txt @@ -0,0 +1,8 @@ +.. _load_engines: + +============ +Load Engines +============ + +.. automodule:: searx.engines + :members: diff --git a/_sources/src/searx.engines.tineye.rst.txt b/_sources/src/searx.engines.tineye.rst.txt new file mode 100644 index 00000000..79e24cfb --- /dev/null +++ b/_sources/src/searx.engines.tineye.rst.txt @@ -0,0 +1,9 @@ +.. _tineye engine: + +====== +Tineye +====== + +.. automodule:: searx.engines.tineye + :members: + diff --git a/_sources/src/searx.engines.yahoo.rst.txt b/_sources/src/searx.engines.yahoo.rst.txt new file mode 100644 index 00000000..df08550c --- /dev/null +++ b/_sources/src/searx.engines.yahoo.rst.txt @@ -0,0 +1,8 @@ +.. _yahoo engine: + +============ +Yahoo Engine +============ + +.. automodule:: searx.engines.yahoo + :members: diff --git a/_sources/src/searx.infopage.rst.txt b/_sources/src/searx.infopage.rst.txt new file mode 100644 index 00000000..243e5b0b --- /dev/null +++ b/_sources/src/searx.infopage.rst.txt @@ -0,0 +1,8 @@ +.. _searx.infopage: + +================ +Online ``/info`` +================ + +.. automodule:: searx.infopage + :members: diff --git a/_sources/src/searx.locales.rst.txt b/_sources/src/searx.locales.rst.txt new file mode 100644 index 00000000..579247af --- /dev/null +++ b/_sources/src/searx.locales.rst.txt @@ -0,0 +1,8 @@ +.. _searx.locales: + +======= +Locales +======= + +.. automodule:: searx.locales + :members: diff --git a/_sources/src/searx.plugins.autodetect_search_language.rst.txt b/_sources/src/searx.plugins.autodetect_search_language.rst.txt new file mode 100644 index 00000000..7b66a6bf --- /dev/null +++ b/_sources/src/searx.plugins.autodetect_search_language.rst.txt @@ -0,0 +1,8 @@ +.. _autodetect search language: + +====================== +Search language plugin +====================== + +.. automodule:: searx.plugins.autodetect_search_language + :members: diff --git a/_sources/src/searx.plugins.limiter.rst.txt b/_sources/src/searx.plugins.limiter.rst.txt new file mode 100644 index 00000000..75d06f5c --- /dev/null +++ b/_sources/src/searx.plugins.limiter.rst.txt @@ -0,0 +1,13 @@ +.. _limiter plugin: + +============== +Limiter Plugin +============== + +.. sidebar:: info + + The :ref:`limiter plugin` requires a :ref:`Redis ` database. + +.. automodule:: searx.plugins.limiter + :members: + diff --git a/_sources/src/searx.plugins.tor_check.rst.txt b/_sources/src/searx.plugins.tor_check.rst.txt new file mode 100644 index 00000000..905328eb --- /dev/null +++ b/_sources/src/searx.plugins.tor_check.rst.txt @@ -0,0 +1,9 @@ +.. _tor check plugin: + +================ +Tor check plugin +================ + +.. automodule:: searx.plugins.tor_check + :members: + diff --git a/_sources/src/searx.redisdb.rst.txt b/_sources/src/searx.redisdb.rst.txt new file mode 100644 index 00000000..625378c9 --- /dev/null +++ b/_sources/src/searx.redisdb.rst.txt @@ -0,0 +1,8 @@ +.. _redis db: + +======== +Redis DB +======== + +.. automodule:: searx.redisdb + :members: diff --git a/_sources/src/searx.redislib.rst.txt b/_sources/src/searx.redislib.rst.txt new file mode 100644 index 00000000..b4604574 --- /dev/null +++ b/_sources/src/searx.redislib.rst.txt @@ -0,0 +1,8 @@ +.. _searx.redis: + +============= +Redis Library +============= + +.. automodule:: searx.redislib + :members: diff --git a/_sources/src/searx.search.rst.txt b/_sources/src/searx.search.rst.txt new file mode 100644 index 00000000..ad76d418 --- /dev/null +++ b/_sources/src/searx.search.rst.txt @@ -0,0 +1,38 @@ +.. _searx.search: + +====== +Search +====== + +.. autoclass:: searx.search.EngineRef + :members: + +.. autoclass:: searx.search.SearchQuery + :members: + +.. autoclass:: searx.search.Search + + .. attribute:: search_query + :type: searx.search.SearchQuery + + .. attribute:: result_container + :type: searx.results.ResultContainer + + .. automethod:: search() -> searx.results.ResultContainer + +.. autoclass:: searx.search.SearchWithPlugins + :members: + + .. attribute:: search_query + :type: searx.search.SearchQuery + + .. attribute:: result_container + :type: searx.results.ResultContainer + + .. attribute:: ordered_plugin_list + :type: typing.List + + .. attribute:: request + :type: flask.request + + .. automethod:: search() -> searx.results.ResultContainer diff --git a/_sources/src/searx.utils.rst.txt b/_sources/src/searx.utils.rst.txt new file mode 100644 index 00000000..6496700c --- /dev/null +++ b/_sources/src/searx.utils.rst.txt @@ -0,0 +1,8 @@ +.. _searx.utils: + +================================= +Utility functions for the engines +================================= + +.. automodule:: searx.utils + :members: diff --git a/_sources/user/index.rst.txt b/_sources/user/index.rst.txt new file mode 100644 index 00000000..af2051f0 --- /dev/null +++ b/_sources/user/index.rst.txt @@ -0,0 +1,15 @@ +================ +User information +================ + +.. contents:: Contents + :depth: 3 + :local: + :backlinks: entry + + +.. _search-syntax: + +.. include:: search-syntax.md + :parser: myst_parser.sphinx_ + diff --git a/_sources/utils/index.rst.txt b/_sources/utils/index.rst.txt new file mode 100644 index 00000000..2da26ed1 --- /dev/null +++ b/_sources/utils/index.rst.txt @@ -0,0 +1,30 @@ +.. _searx_utils: +.. _toolboxing: + +================== +DevOps tooling box +================== + +In the folder :origin:`utils/` we maintain some tools useful for administrators +and developers. + +.. toctree:: + :maxdepth: 2 + :caption: Contents + + searxng.sh + lxc.sh + +Common command environments +=========================== + +The scripts in our tooling box often dispose of common environments: + +``FORCE_TIMEOUT`` : environment + Sets timeout for interactive prompts. If you want to run a script in batch + job, with defaults choices, set ``FORCE_TIMEOUT=0``. By example; to install a + SearXNG server and nginx proxy on all containers of the :ref:`SearXNG suite + ` use:: + + sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install all + sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx diff --git a/_sources/utils/lxc.sh.rst.txt b/_sources/utils/lxc.sh.rst.txt new file mode 100644 index 00000000..4308a12c --- /dev/null +++ b/_sources/utils/lxc.sh.rst.txt @@ -0,0 +1,196 @@ + +.. _snap: https://snapcraft.io +.. _snapcraft LXD: https://snapcraft.io/lxd +.. _LXC/LXD Image Server: https://uk.images.linuxcontainers.org/ +.. _LXC: https://linuxcontainers.org/lxc/introduction/ +.. _LXD: https://linuxcontainers.org/lxd/introduction/ +.. _`LXD@github`: https://github.com/lxc/lxd + +.. _archlinux: https://www.archlinux.org/ + +.. _lxc.sh: + +================ +``utils/lxc.sh`` +================ + +.. sidebar:: further reading + + - snap_, `snapcraft LXD`_ + - LXC_, LXD_ + - `LXC/LXD Image Server`_ + - `LXD@github`_ + +With the use of *Linux Containers* (LXC_) we can scale our tasks over a stack of +containers, what we call the: *lxc suite*. The *SearXNG suite* +(:origin:`lxc-searxng.env `) is loaded by default, every time +you start the ``lxc.sh`` script (*you do not need to care about*). + +Before you can start with containers, you need to install and initiate LXD_ +once:: + + $ snap install lxd + $ lxd init --auto + +To make use of the containers from the *SearXNG suite*, you have to build the +:ref:`LXC suite containers ` initial. But be warned, **this might +take some time**:: + + $ sudo -H ./utils/lxc.sh build + +A cup of coffee later, your LXC suite is build up and you can run whatever task +you want / in a selected or even in all :ref:`LXC suite containers `. + +.. hint:: + + If you see any problems with the internet connectivity of your + containers read section :ref:`internet connectivity docker`. + +If you do not want to build all containers, **you can build just one**:: + + $ sudo -H ./utils/lxc.sh build searxng-archlinux + +*Good to know ...* + +Each container shares the root folder of the repository and the command +``utils/lxc.sh cmd`` **handles relative path names transparent**, compare output +of:: + + $ sudo -H ./utils/lxc.sh cmd -- ls -la Makefile + ... + +In the containers, you can run what ever you want, e.g. to start a bash use:: + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash + INFO: [searxng-archlinux] bash + [root@searxng-archlinux SearXNG]# + +If there comes the time you want to **get rid off all** the containers and +**clean up local images** just type:: + + $ sudo -H ./utils/lxc.sh remove + $ sudo -H ./utils/lxc.sh remove images + +.. _internet connectivity docker: + +Internet Connectivity & Docker +============================== + +.. sidebar:: further read + + - `Docker blocking network of existing LXC containers `__ + - `Docker and IPtables (fralef.me) `__ + - `Docker and iptables (docker.com) `__ + +There is a conflict in the ``iptables`` setup of Docker & LXC. If you have +docker installed, you may find that the internet connectivity of your LXD +containers no longer work. + +Whenever docker is started (reboot) it sets the iptables policy for the +``FORWARD`` chain to ``DROP`` `[ref] +`__:: + + $ sudo -H iptables-save | grep FORWARD + :FORWARD ACCEPT [7048:7851230] + :FORWARD DROP [7048:7851230] + +A handy solution of this problem might be to reset the policy for the +``FORWARD`` chain after the network has been initialized. For this create a +file in the ``if-up`` section of the network (``/etc/network/if-up.d/iptable``) +and insert the following lines:: + + #!/bin/sh + iptables -F FORWARD + iptables -P FORWARD ACCEPT + +Don't forget to set the execution bit:: + + sudo chmod ugo+x /etc/network/if-up.d/iptable + +Reboot your system and check the iptables rules:: + + $ sudo -H iptables-save | grep FORWARD + :FORWARD ACCEPT [7048:7851230] + :FORWARD ACCEPT [7048:7851230] + + +.. _lxc.sh install suite: + +Install suite +============= + +To install the complete :ref:`SearXNG suite (includes searx, morty & filtron) +` into all LXC_ use:: + + $ sudo -H ./utils/lxc.sh install suite + +The command above installs a SearXNG suite (see :ref:`installation scripts`). +To :ref:`install a nginx ` reverse proxy (or alternatively +use :ref:`apache `):: + + sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx + +To get the IP (URL) of the SearXNG service in the containers use ``show suite`` +command. To test instances from containers just open the URLs in your +WEB-Browser:: + + $ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL + + [searxng-ubu2110] SEARXNG_URL : http://n.n.n.147/searxng + [searxng-ubu2004] SEARXNG_URL : http://n.n.n.246/searxng + [searxnggfedora35] SEARXNG_URL : http://n.n.n.140/searxng + [searxng-archlinux] SEARXNG_URL : http://n.n.n.165/searxng + + +Running commands +================ + +**Inside containers, you can use make or run scripts** from the +:ref:`toolboxing`. By example: to setup a :ref:`buildhosts` and run the +Makefile target ``test`` in the archlinux_ container:: + + sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh install buildhost + sudo -H ./utils/lxc.sh cmd searxng-archlinux make test + + +Setup SearXNG buildhost +======================= + +You can **install the SearXNG buildhost environment** into one or all containers. +The installation procedure to set up a :ref:`build host` takes its +time. Installation in all containers will take more time (time for another cup +of coffee).:: + + sudo -H ./utils/lxc.sh cmd -- ./utils/searxng.sh install buildhost + +To build (live) documentation inside a archlinux_ container:: + + sudo -H ./utils/lxc.sh cmd searxng-archlinux make docs.clean docs.live + ... + [I 200331 15:00:42 server:296] Serving on http://0.0.0.0:8080 + +To get IP of the container and the port number *live docs* is listening:: + + $ sudo ./utils/lxc.sh show suite | grep docs.live + ... + [searxng-archlinux] INFO: (eth0) docs.live: http://n.n.n.12:8080/ + + +.. _lxc.sh help: + +Overview +======== + +The ``--help`` output of the script is largely self-explanatory: + +.. program-output:: ../utils/lxc.sh --help + + +.. _lxc-searxng.env: + +SearXNG suite +============= + +.. literalinclude:: ../../utils/lxc-searxng.env + :language: bash diff --git a/_sources/utils/searxng.sh.rst.txt b/_sources/utils/searxng.sh.rst.txt new file mode 100644 index 00000000..f6578f7b --- /dev/null +++ b/_sources/utils/searxng.sh.rst.txt @@ -0,0 +1,36 @@ + +.. _searxng.sh: + +==================== +``utils/searxng.sh`` +==================== + +.. sidebar:: further reading + + - :ref:`architecture` + - :ref:`installation` + - :ref:`installation nginx` + - :ref:`installation apache` + +To simplify the installation and maintenance of a SearXNG instance you can use the +script :origin:`utils/searxng.sh`. + +Install +======= + +In most cases you will install SearXNG simply by running the command: + +.. code:: bash + + sudo -H ./utils/searx.sh install all + +The installation is described in chapter :ref:`installation basic`. + +.. _searxng.sh overview: + +Overview +======== + +The ``--help`` output of the script is largely self-explanatory: + +.. program-output:: ../utils/searxng.sh --help diff --git a/_static/_sphinx_javascript_frameworks_compat.js b/_static/_sphinx_javascript_frameworks_compat.js new file mode 100644 index 00000000..8549469d --- /dev/null +++ b/_static/_sphinx_javascript_frameworks_compat.js @@ -0,0 +1,134 @@ +/* + * _sphinx_javascript_frameworks_compat.js + * ~~~~~~~~~~ + * + * Compatability shim for jQuery and underscores.js. + * + * WILL BE REMOVED IN Sphinx 6.0 + * xref RemovedInSphinx60Warning + * + */ + +/** + * select a different prefix for underscore + */ +$u = _.noConflict(); + + +/** + * small helper function to urldecode strings + * + * See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/decodeURIComponent#Decoding_query_parameters_from_a_URL + */ +jQuery.urldecode = function(x) { + if (!x) { + return x + } + return decodeURIComponent(x.replace(/\+/g, ' ')); +}; + +/** + * small helper function to urlencode strings + */ +jQuery.urlencode = encodeURIComponent; + +/** + * This function returns the parsed url parameters of the + * current request. Multiple values per key are supported, + * it will always return arrays of strings for the value parts. + */ +jQuery.getQueryParameters = function(s) { + if (typeof s === 'undefined') + s = document.location.search; + var parts = s.substr(s.indexOf('?') + 1).split('&'); + var result = {}; + for (var i = 0; i < parts.length; i++) { + var tmp = parts[i].split('=', 2); + var key = jQuery.urldecode(tmp[0]); + var value = jQuery.urldecode(tmp[1]); + if (key in result) + result[key].push(value); + else + result[key] = [value]; + } + return result; +}; + +/** + * highlight a given string on a jquery object by wrapping it in + * span elements with the given class name. + */ +jQuery.fn.highlightText = function(text, className) { + function highlight(node, addItems) { + if (node.nodeType === 3) { + var val = node.nodeValue; + var pos = val.toLowerCase().indexOf(text); + if (pos >= 0 && + !jQuery(node.parentNode).hasClass(className) && + !jQuery(node.parentNode).hasClass("nohighlight")) { + var span; + var isInSVG = jQuery(node).closest("body, svg, foreignObject").is("svg"); + if (isInSVG) { + span = document.createElementNS("http://www.w3.org/2000/svg", "tspan"); + } else { + span = document.createElement("span"); + span.className = className; + } + span.appendChild(document.createTextNode(val.substr(pos, text.length))); + node.parentNode.insertBefore(span, node.parentNode.insertBefore( + document.createTextNode(val.substr(pos + text.length)), + node.nextSibling)); + node.nodeValue = val.substr(0, pos); + if (isInSVG) { + var rect = document.createElementNS("http://www.w3.org/2000/svg", "rect"); + var bbox = node.parentElement.getBBox(); + rect.x.baseVal.value = bbox.x; + rect.y.baseVal.value = bbox.y; + rect.width.baseVal.value = bbox.width; + rect.height.baseVal.value = bbox.height; + rect.setAttribute('class', className); + addItems.push({ + "parent": node.parentNode, + "target": rect}); + } + } + } + else if (!jQuery(node).is("button, select, textarea")) { + jQuery.each(node.childNodes, function() { + highlight(this, addItems); + }); + } + } + var addItems = []; + var result = this.each(function() { + highlight(this, addItems); + }); + for (var i = 0; i < addItems.length; ++i) { + jQuery(addItems[i].parent).before(addItems[i].target); + } + return result; +}; + +/* + * backward compatibility for jQuery.browser + * This will be supported until firefox bug is fixed. + */ +if (!jQuery.browser) { + jQuery.uaMatch = function(ua) { + ua = ua.toLowerCase(); + + var match = /(chrome)[ \/]([\w.]+)/.exec(ua) || + /(webkit)[ \/]([\w.]+)/.exec(ua) || + /(opera)(?:.*version|)[ \/]([\w.]+)/.exec(ua) || + /(msie) ([\w.]+)/.exec(ua) || + ua.indexOf("compatible") < 0 && /(mozilla)(?:.*? rv:([\w.]+)|)/.exec(ua) || + []; + + return { + browser: match[ 1 ] || "", + version: match[ 2 ] || "0" + }; + }; + jQuery.browser = {}; + jQuery.browser[jQuery.uaMatch(navigator.userAgent).browser] = true; +} diff --git a/_static/basic.css b/_static/basic.css new file mode 100644 index 00000000..4e9a9f1f --- /dev/null +++ b/_static/basic.css @@ -0,0 +1,900 @@ +/* + * basic.css + * ~~~~~~~~~ + * + * Sphinx stylesheet -- basic theme. + * + * :copyright: Copyright 2007-2022 by the Sphinx team, see AUTHORS. + * :license: BSD, see LICENSE for details. + * + */ + +/* -- main layout ----------------------------------------------------------- */ + +div.clearer { + clear: both; +} + +div.section::after { + display: block; + content: ''; + clear: left; +} + +/* -- relbar ---------------------------------------------------------------- */ + +div.related { + width: 100%; + font-size: 90%; +} + +div.related h3 { + display: none; +} + +div.related ul { + margin: 0; + padding: 0 0 0 10px; + list-style: none; +} + +div.related li { + display: inline; +} + +div.related li.right { + float: right; + margin-right: 5px; +} + +/* -- sidebar --------------------------------------------------------------- */ + +div.sphinxsidebarwrapper { + padding: 10px 5px 0 10px; +} + +div.sphinxsidebar { + float: left; + width: 230px; + margin-left: -100%; + font-size: 90%; + word-wrap: break-word; + overflow-wrap : break-word; +} + +div.sphinxsidebar ul { + list-style: none; +} + +div.sphinxsidebar ul ul, +div.sphinxsidebar ul.want-points { + margin-left: 20px; + list-style: square; +} + +div.sphinxsidebar ul ul { + margin-top: 0; + margin-bottom: 0; +} + +div.sphinxsidebar form { + margin-top: 10px; +} + +div.sphinxsidebar input { + border: 1px solid #98dbcc; + font-family: sans-serif; + font-size: 1em; +} + +div.sphinxsidebar #searchbox form.search { + overflow: hidden; +} + +div.sphinxsidebar #searchbox input[type="text"] { + float: left; + width: 80%; + padding: 0.25em; + box-sizing: border-box; +} + +div.sphinxsidebar #searchbox input[type="submit"] { + float: left; + width: 20%; + border-left: none; + padding: 0.25em; + box-sizing: border-box; +} + + +img { + border: 0; + max-width: 100%; +} + +/* -- search page ----------------------------------------------------------- */ + +ul.search { + margin: 10px 0 0 20px; + padding: 0; +} + +ul.search li { + padding: 5px 0 5px 20px; + background-image: url(file.png); + background-repeat: no-repeat; + background-position: 0 7px; +} + +ul.search li a { + font-weight: bold; +} + +ul.search li p.context { + color: #888; + margin: 2px 0 0 30px; + text-align: left; +} + +ul.keywordmatches li.goodmatch a { + font-weight: bold; +} + +/* -- index page ------------------------------------------------------------ */ + +table.contentstable { + width: 90%; + margin-left: auto; + margin-right: auto; +} + +table.contentstable p.biglink { + line-height: 150%; +} + +a.biglink { + font-size: 1.3em; +} + +span.linkdescr { + font-style: italic; + padding-top: 5px; + font-size: 90%; +} + +/* -- general index --------------------------------------------------------- */ + +table.indextable { + width: 100%; +} + +table.indextable td { + text-align: left; + vertical-align: top; +} + +table.indextable ul { + margin-top: 0; + margin-bottom: 0; + list-style-type: none; +} + +table.indextable > tbody > tr > td > ul { + padding-left: 0em; +} + +table.indextable tr.pcap { + height: 10px; +} + +table.indextable tr.cap { + margin-top: 10px; + background-color: #f2f2f2; +} + +img.toggler { + margin-right: 3px; + margin-top: 3px; + cursor: pointer; +} + +div.modindex-jumpbox { + border-top: 1px solid #ddd; + border-bottom: 1px solid #ddd; + margin: 1em 0 1em 0; + padding: 0.4em; +} + +div.genindex-jumpbox { + border-top: 1px solid #ddd; + border-bottom: 1px solid #ddd; + margin: 1em 0 1em 0; + padding: 0.4em; +} + +/* -- domain module index --------------------------------------------------- */ + +table.modindextable td { + padding: 2px; + border-collapse: collapse; +} + +/* -- general body styles --------------------------------------------------- */ + +div.body { + min-width: 360px; + max-width: 800px; +} + +div.body p, div.body dd, div.body li, div.body blockquote { + -moz-hyphens: auto; + -ms-hyphens: auto; + -webkit-hyphens: auto; + hyphens: auto; +} + +a.headerlink { + visibility: hidden; +} + +h1:hover > a.headerlink, +h2:hover > a.headerlink, +h3:hover > a.headerlink, +h4:hover > a.headerlink, +h5:hover > a.headerlink, +h6:hover > a.headerlink, +dt:hover > a.headerlink, +caption:hover > a.headerlink, +p.caption:hover > a.headerlink, +div.code-block-caption:hover > a.headerlink { + visibility: visible; +} + +div.body p.caption { + text-align: inherit; +} + +div.body td { + text-align: left; +} + +.first { + margin-top: 0 !important; +} + +p.rubric { + margin-top: 30px; + font-weight: bold; +} + +img.align-left, figure.align-left, .figure.align-left, object.align-left { + clear: left; + float: left; + margin-right: 1em; +} + +img.align-right, figure.align-right, .figure.align-right, object.align-right { + clear: right; + float: right; + margin-left: 1em; +} + +img.align-center, figure.align-center, .figure.align-center, object.align-center { + display: block; + margin-left: auto; + margin-right: auto; +} + +img.align-default, figure.align-default, .figure.align-default { + display: block; + margin-left: auto; + margin-right: auto; +} + +.align-left { + text-align: left; +} + +.align-center { + text-align: center; +} + +.align-default { + text-align: center; +} + +.align-right { + text-align: right; +} + +/* -- sidebars -------------------------------------------------------------- */ + +div.sidebar, +aside.sidebar { + margin: 0 0 0.5em 1em; + border: 1px solid #ddb; + padding: 7px; + background-color: #ffe; + width: 40%; + float: right; + clear: right; + overflow-x: auto; +} + +p.sidebar-title { + font-weight: bold; +} +nav.contents, +aside.topic, +div.admonition, div.topic, blockquote { + clear: left; +} + +/* -- topics ---------------------------------------------------------------- */ +nav.contents, +aside.topic, +div.topic { + border: 1px solid #ccc; + padding: 7px; + margin: 10px 0 10px 0; +} + +p.topic-title { + font-size: 1.1em; + font-weight: bold; + margin-top: 10px; +} + +/* -- admonitions ----------------------------------------------------------- */ + +div.admonition { + margin-top: 10px; + margin-bottom: 10px; + padding: 7px; +} + +div.admonition dt { + font-weight: bold; +} + +p.admonition-title { + margin: 0px 10px 5px 0px; + font-weight: bold; +} + +div.body p.centered { + text-align: center; + margin-top: 25px; +} + +/* -- content of sidebars/topics/admonitions -------------------------------- */ + +div.sidebar > :last-child, +aside.sidebar > :last-child, +nav.contents > :last-child, +aside.topic > :last-child, +div.topic > :last-child, +div.admonition > :last-child { + margin-bottom: 0; +} + +div.sidebar::after, +aside.sidebar::after, +nav.contents::after, +aside.topic::after, +div.topic::after, +div.admonition::after, +blockquote::after { + display: block; + content: ''; + clear: both; +} + +/* -- tables ---------------------------------------------------------------- */ + +table.docutils { + margin-top: 10px; + margin-bottom: 10px; + border: 0; + border-collapse: collapse; +} + +table.align-center { + margin-left: auto; + margin-right: auto; +} + +table.align-default { + margin-left: auto; + margin-right: auto; +} + +table caption span.caption-number { + font-style: italic; +} + +table caption span.caption-text { +} + +table.docutils td, table.docutils th { + padding: 1px 8px 1px 5px; + border-top: 0; + border-left: 0; + border-right: 0; + border-bottom: 1px solid #aaa; +} + +th { + text-align: left; + padding-right: 5px; +} + +table.citation { + border-left: solid 1px gray; + margin-left: 1px; +} + +table.citation td { + border-bottom: none; +} + +th > :first-child, +td > :first-child { + margin-top: 0px; +} + +th > :last-child, +td > :last-child { + margin-bottom: 0px; +} + +/* -- figures --------------------------------------------------------------- */ + +div.figure, figure { + margin: 0.5em; + padding: 0.5em; +} + +div.figure p.caption, figcaption { + padding: 0.3em; +} + +div.figure p.caption span.caption-number, +figcaption span.caption-number { + font-style: italic; +} + +div.figure p.caption span.caption-text, +figcaption span.caption-text { +} + +/* -- field list styles ----------------------------------------------------- */ + +table.field-list td, table.field-list th { + border: 0 !important; +} + +.field-list ul { + margin: 0; + padding-left: 1em; +} + +.field-list p { + margin: 0; +} + +.field-name { + -moz-hyphens: manual; + -ms-hyphens: manual; + -webkit-hyphens: manual; + hyphens: manual; +} + +/* -- hlist styles ---------------------------------------------------------- */ + +table.hlist { + margin: 1em 0; +} + +table.hlist td { + vertical-align: top; +} + +/* -- object description styles --------------------------------------------- */ + +.sig { + font-family: 'Consolas', 'Menlo', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace; +} + +.sig-name, code.descname { + background-color: transparent; + font-weight: bold; +} + +.sig-name { + font-size: 1.1em; +} + +code.descname { + font-size: 1.2em; +} + +.sig-prename, code.descclassname { + background-color: transparent; +} + +.optional { + font-size: 1.3em; +} + +.sig-paren { + font-size: larger; +} + +.sig-param.n { + font-style: italic; +} + +/* C++ specific styling */ + +.sig-inline.c-texpr, +.sig-inline.cpp-texpr { + font-family: unset; +} + +.sig.c .k, .sig.c .kt, +.sig.cpp .k, .sig.cpp .kt { + color: #0033B3; +} + +.sig.c .m, +.sig.cpp .m { + color: #1750EB; +} + +.sig.c .s, .sig.c .sc, +.sig.cpp .s, .sig.cpp .sc { + color: #067D17; +} + + +/* -- other body styles ----------------------------------------------------- */ + +ol.arabic { + list-style: decimal; +} + +ol.loweralpha { + list-style: lower-alpha; +} + +ol.upperalpha { + list-style: upper-alpha; +} + +ol.lowerroman { + list-style: lower-roman; +} + +ol.upperroman { + list-style: upper-roman; +} + +:not(li) > ol > li:first-child > :first-child, +:not(li) > ul > li:first-child > :first-child { + margin-top: 0px; +} + +:not(li) > ol > li:last-child > :last-child, +:not(li) > ul > li:last-child > :last-child { + margin-bottom: 0px; +} + +ol.simple ol p, +ol.simple ul p, +ul.simple ol p, +ul.simple ul p { + margin-top: 0; +} + +ol.simple > li:not(:first-child) > p, +ul.simple > li:not(:first-child) > p { + margin-top: 0; +} + +ol.simple p, +ul.simple p { + margin-bottom: 0; +} +aside.footnote > span, +div.citation > span { + float: left; +} +aside.footnote > span:last-of-type, +div.citation > span:last-of-type { + padding-right: 0.5em; +} +aside.footnote > p { + margin-left: 2em; +} +div.citation > p { + margin-left: 4em; +} +aside.footnote > p:last-of-type, +div.citation > p:last-of-type { + margin-bottom: 0em; +} +aside.footnote > p:last-of-type:after, +div.citation > p:last-of-type:after { + content: ""; + clear: both; +} + +dl.field-list { + display: grid; + grid-template-columns: fit-content(30%) auto; +} + +dl.field-list > dt { + font-weight: bold; + word-break: break-word; + padding-left: 0.5em; + padding-right: 5px; +} + +dl.field-list > dd { + padding-left: 0.5em; + margin-top: 0em; + margin-left: 0em; + margin-bottom: 0em; +} + +dl { + margin-bottom: 15px; +} + +dd > :first-child { + margin-top: 0px; +} + +dd ul, dd table { + margin-bottom: 10px; +} + +dd { + margin-top: 3px; + margin-bottom: 10px; + margin-left: 30px; +} + +dl > dd:last-child, +dl > dd:last-child > :last-child { + margin-bottom: 0; +} + +dt:target, span.highlighted { + background-color: #fbe54e; +} + +rect.highlighted { + fill: #fbe54e; +} + +dl.glossary dt { + font-weight: bold; + font-size: 1.1em; +} + +.versionmodified { + font-style: italic; +} + +.system-message { + background-color: #fda; + padding: 5px; + border: 3px solid red; +} + +.footnote:target { + background-color: #ffa; +} + +.line-block { + display: block; + margin-top: 1em; + margin-bottom: 1em; +} + +.line-block .line-block { + margin-top: 0; + margin-bottom: 0; + margin-left: 1.5em; +} + +.guilabel, .menuselection { + font-family: sans-serif; +} + +.accelerator { + text-decoration: underline; +} + +.classifier { + font-style: oblique; +} + +.classifier:before { + font-style: normal; + margin: 0 0.5em; + content: ":"; + display: inline-block; +} + +abbr, acronym { + border-bottom: dotted 1px; + cursor: help; +} + +/* -- code displays --------------------------------------------------------- */ + +pre { + overflow: auto; + overflow-y: hidden; /* fixes display issues on Chrome browsers */ +} + +pre, div[class*="highlight-"] { + clear: both; +} + +span.pre { + -moz-hyphens: none; + -ms-hyphens: none; + -webkit-hyphens: none; + hyphens: none; + white-space: nowrap; +} + +div[class*="highlight-"] { + margin: 1em 0; +} + +td.linenos pre { + border: 0; + background-color: transparent; + color: #aaa; +} + +table.highlighttable { + display: block; +} + +table.highlighttable tbody { + display: block; +} + +table.highlighttable tr { + display: flex; +} + +table.highlighttable td { + margin: 0; + padding: 0; +} + +table.highlighttable td.linenos { + padding-right: 0.5em; +} + +table.highlighttable td.code { + flex: 1; + overflow: hidden; +} + +.highlight .hll { + display: block; +} + +div.highlight pre, +table.highlighttable pre { + margin: 0; +} + +div.code-block-caption + div { + margin-top: 0; +} + +div.code-block-caption { + margin-top: 1em; + padding: 2px 5px; + font-size: small; +} + +div.code-block-caption code { + background-color: transparent; +} + +table.highlighttable td.linenos, +span.linenos, +div.highlight span.gp { /* gp: Generic.Prompt */ + user-select: none; + -webkit-user-select: text; /* Safari fallback only */ + -webkit-user-select: none; /* Chrome/Safari */ + -moz-user-select: none; /* Firefox */ + -ms-user-select: none; /* IE10+ */ +} + +div.code-block-caption span.caption-number { + padding: 0.1em 0.3em; + font-style: italic; +} + +div.code-block-caption span.caption-text { +} + +div.literal-block-wrapper { + margin: 1em 0; +} + +code.xref, a code { + background-color: transparent; + font-weight: bold; +} + +h1 code, h2 code, h3 code, h4 code, h5 code, h6 code { + background-color: transparent; +} + +.viewcode-link { + float: right; +} + +.viewcode-back { + float: right; + font-family: sans-serif; +} + +div.viewcode-block:target { + margin: -1px -10px; + padding: 0 10px; +} + +/* -- math display ---------------------------------------------------------- */ + +img.math { + vertical-align: middle; +} + +div.body div.math p { + text-align: center; +} + +span.eqno { + float: right; +} + +span.eqno a.headerlink { + position: absolute; + z-index: 1; +} + +div.math:hover a.headerlink { + visibility: visible; +} + +/* -- printout stylesheet --------------------------------------------------- */ + +@media print { + div.document, + div.documentwrapper, + div.bodywrapper { + margin: 0 !important; + width: 100%; + } + + div.sphinxsidebar, + div.related, + div.footer, + #top-link { + display: none; + } +} \ No newline at end of file diff --git a/_static/doctools.js b/_static/doctools.js new file mode 100644 index 00000000..527b876c --- /dev/null +++ b/_static/doctools.js @@ -0,0 +1,156 @@ +/* + * doctools.js + * ~~~~~~~~~~~ + * + * Base JavaScript utilities for all Sphinx HTML documentation. + * + * :copyright: Copyright 2007-2022 by the Sphinx team, see AUTHORS. + * :license: BSD, see LICENSE for details. + * + */ +"use strict"; + +const BLACKLISTED_KEY_CONTROL_ELEMENTS = new Set([ + "TEXTAREA", + "INPUT", + "SELECT", + "BUTTON", +]); + +const _ready = (callback) => { + if (document.readyState !== "loading") { + callback(); + } else { + document.addEventListener("DOMContentLoaded", callback); + } +}; + +/** + * Small JavaScript module for the documentation. + */ +const Documentation = { + init: () => { + Documentation.initDomainIndexTable(); + Documentation.initOnKeyListeners(); + }, + + /** + * i18n support + */ + TRANSLATIONS: {}, + PLURAL_EXPR: (n) => (n === 1 ? 0 : 1), + LOCALE: "unknown", + + // gettext and ngettext don't access this so that the functions + // can safely bound to a different name (_ = Documentation.gettext) + gettext: (string) => { + const translated = Documentation.TRANSLATIONS[string]; + switch (typeof translated) { + case "undefined": + return string; // no translation + case "string": + return translated; // translation exists + default: + return translated[0]; // (singular, plural) translation tuple exists + } + }, + + ngettext: (singular, plural, n) => { + const translated = Documentation.TRANSLATIONS[singular]; + if (typeof translated !== "undefined") + return translated[Documentation.PLURAL_EXPR(n)]; + return n === 1 ? singular : plural; + }, + + addTranslations: (catalog) => { + Object.assign(Documentation.TRANSLATIONS, catalog.messages); + Documentation.PLURAL_EXPR = new Function( + "n", + `return (${catalog.plural_expr})` + ); + Documentation.LOCALE = catalog.locale; + }, + + /** + * helper function to focus on search bar + */ + focusSearchBar: () => { + document.querySelectorAll("input[name=q]")[0]?.focus(); + }, + + /** + * Initialise the domain index toggle buttons + */ + initDomainIndexTable: () => { + const toggler = (el) => { + const idNumber = el.id.substr(7); + const toggledRows = document.querySelectorAll(`tr.cg-${idNumber}`); + if (el.src.substr(-9) === "minus.png") { + el.src = `${el.src.substr(0, el.src.length - 9)}plus.png`; + toggledRows.forEach((el) => (el.style.display = "none")); + } else { + el.src = `${el.src.substr(0, el.src.length - 8)}minus.png`; + toggledRows.forEach((el) => (el.style.display = "")); + } + }; + + const togglerElements = document.querySelectorAll("img.toggler"); + togglerElements.forEach((el) => + el.addEventListener("click", (event) => toggler(event.currentTarget)) + ); + togglerElements.forEach((el) => (el.style.display = "")); + if (DOCUMENTATION_OPTIONS.COLLAPSE_INDEX) togglerElements.forEach(toggler); + }, + + initOnKeyListeners: () => { + // only install a listener if it is really needed + if ( + !DOCUMENTATION_OPTIONS.NAVIGATION_WITH_KEYS && + !DOCUMENTATION_OPTIONS.ENABLE_SEARCH_SHORTCUTS + ) + return; + + document.addEventListener("keydown", (event) => { + // bail for input elements + if (BLACKLISTED_KEY_CONTROL_ELEMENTS.has(document.activeElement.tagName)) return; + // bail with special keys + if (event.altKey || event.ctrlKey || event.metaKey) return; + + if (!event.shiftKey) { + switch (event.key) { + case "ArrowLeft": + if (!DOCUMENTATION_OPTIONS.NAVIGATION_WITH_KEYS) break; + + const prevLink = document.querySelector('link[rel="prev"]'); + if (prevLink && prevLink.href) { + window.location.href = prevLink.href; + event.preventDefault(); + } + break; + case "ArrowRight": + if (!DOCUMENTATION_OPTIONS.NAVIGATION_WITH_KEYS) break; + + const nextLink = document.querySelector('link[rel="next"]'); + if (nextLink && nextLink.href) { + window.location.href = nextLink.href; + event.preventDefault(); + } + break; + } + } + + // some keyboard layouts may need Shift to get / + switch (event.key) { + case "/": + if (!DOCUMENTATION_OPTIONS.ENABLE_SEARCH_SHORTCUTS) break; + Documentation.focusSearchBar(); + event.preventDefault(); + } + }); + }, +}; + +// quick alias for translations +const _ = Documentation.gettext; + +_ready(Documentation.init); diff --git a/_static/documentation_options.js b/_static/documentation_options.js new file mode 100644 index 00000000..bffa9248 --- /dev/null +++ b/_static/documentation_options.js @@ -0,0 +1,14 @@ +var DOCUMENTATION_OPTIONS = { + URL_ROOT: document.getElementById("documentation_options").getAttribute('data-url_root'), + VERSION: '2023.1.23+522ba9a1', + LANGUAGE: 'en', + COLLAPSE_INDEX: false, + BUILDER: 'html', + FILE_SUFFIX: '.html', + LINK_SUFFIX: '.html', + HAS_SOURCE: true, + SOURCELINK_SUFFIX: '.txt', + NAVIGATION_WITH_KEYS: false, + SHOW_SEARCH_SUMMARY: true, + ENABLE_SEARCH_SHORTCUTS: true, +}; \ No newline at end of file diff --git a/_static/file.png b/_static/file.png new file mode 100644 index 00000000..a858a410 Binary files /dev/null and b/_static/file.png differ diff --git a/_static/jquery-3.6.0.js b/_static/jquery-3.6.0.js new file mode 100644 index 00000000..fc6c299b --- /dev/null +++ b/_static/jquery-3.6.0.js @@ -0,0 +1,10881 @@ +/*! + * jQuery JavaScript Library v3.6.0 + * https://jquery.com/ + * + * Includes Sizzle.js + * https://sizzlejs.com/ + * + * Copyright OpenJS Foundation and other contributors + * Released under the MIT license + * https://jquery.org/license + * + * Date: 2021-03-02T17:08Z + */ +( function( global, factory ) { + + "use strict"; + + if ( typeof module === "object" && typeof module.exports === "object" ) { + + // For CommonJS and CommonJS-like environments where a proper `window` + // is present, execute the factory and get jQuery. + // For environments that do not have a `window` with a `document` + // (such as Node.js), expose a factory as module.exports. + // This accentuates the need for the creation of a real `window`. + // e.g. var jQuery = require("jquery")(window); + // See ticket #14549 for more info. + module.exports = global.document ? + factory( global, true ) : + function( w ) { + if ( !w.document ) { + throw new Error( "jQuery requires a window with a document" ); + } + return factory( w ); + }; + } else { + factory( global ); + } + +// Pass this if window is not defined yet +} )( typeof window !== "undefined" ? window : this, function( window, noGlobal ) { + +// Edge <= 12 - 13+, Firefox <=18 - 45+, IE 10 - 11, Safari 5.1 - 9+, iOS 6 - 9.1 +// throw exceptions when non-strict code (e.g., ASP.NET 4.5) accesses strict mode +// arguments.callee.caller (trac-13335). But as of jQuery 3.0 (2016), strict mode should be common +// enough that all such attempts are guarded in a try block. +"use strict"; + +var arr = []; + +var getProto = Object.getPrototypeOf; + +var slice = arr.slice; + +var flat = arr.flat ? function( array ) { + return arr.flat.call( array ); +} : function( array ) { + return arr.concat.apply( [], array ); +}; + + +var push = arr.push; + +var indexOf = arr.indexOf; + +var class2type = {}; + +var toString = class2type.toString; + +var hasOwn = class2type.hasOwnProperty; + +var fnToString = hasOwn.toString; + +var ObjectFunctionString = fnToString.call( Object ); + +var support = {}; + +var isFunction = function isFunction( obj ) { + + // Support: Chrome <=57, Firefox <=52 + // In some browsers, typeof returns "function" for HTML elements + // (i.e., `typeof document.createElement( "object" ) === "function"`). + // We don't want to classify *any* DOM node as a function. + // Support: QtWeb <=3.8.5, WebKit <=534.34, wkhtmltopdf tool <=0.12.5 + // Plus for old WebKit, typeof returns "function" for HTML collections + // (e.g., `typeof document.getElementsByTagName("div") === "function"`). (gh-4756) + return typeof obj === "function" && typeof obj.nodeType !== "number" && + typeof obj.item !== "function"; + }; + + +var isWindow = function isWindow( obj ) { + return obj != null && obj === obj.window; + }; + + +var document = window.document; + + + + var preservedScriptAttributes = { + type: true, + src: true, + nonce: true, + noModule: true + }; + + function DOMEval( code, node, doc ) { + doc = doc || document; + + var i, val, + script = doc.createElement( "script" ); + + script.text = code; + if ( node ) { + for ( i in preservedScriptAttributes ) { + + // Support: Firefox 64+, Edge 18+ + // Some browsers don't support the "nonce" property on scripts. + // On the other hand, just using `getAttribute` is not enough as + // the `nonce` attribute is reset to an empty string whenever it + // becomes browsing-context connected. + // See https://github.com/whatwg/html/issues/2369 + // See https://html.spec.whatwg.org/#nonce-attributes + // The `node.getAttribute` check was added for the sake of + // `jQuery.globalEval` so that it can fake a nonce-containing node + // via an object. + val = node[ i ] || node.getAttribute && node.getAttribute( i ); + if ( val ) { + script.setAttribute( i, val ); + } + } + } + doc.head.appendChild( script ).parentNode.removeChild( script ); + } + + +function toType( obj ) { + if ( obj == null ) { + return obj + ""; + } + + // Support: Android <=2.3 only (functionish RegExp) + return typeof obj === "object" || typeof obj === "function" ? + class2type[ toString.call( obj ) ] || "object" : + typeof obj; +} +/* global Symbol */ +// Defining this global in .eslintrc.json would create a danger of using the global +// unguarded in another place, it seems safer to define global only for this module + + + +var + version = "3.6.0", + + // Define a local copy of jQuery + jQuery = function( selector, context ) { + + // The jQuery object is actually just the init constructor 'enhanced' + // Need init if jQuery is called (just allow error to be thrown if not included) + return new jQuery.fn.init( selector, context ); + }; + +jQuery.fn = jQuery.prototype = { + + // The current version of jQuery being used + jquery: version, + + constructor: jQuery, + + // The default length of a jQuery object is 0 + length: 0, + + toArray: function() { + return slice.call( this ); + }, + + // Get the Nth element in the matched element set OR + // Get the whole matched element set as a clean array + get: function( num ) { + + // Return all the elements in a clean array + if ( num == null ) { + return slice.call( this ); + } + + // Return just the one element from the set + return num < 0 ? this[ num + this.length ] : this[ num ]; + }, + + // Take an array of elements and push it onto the stack + // (returning the new matched element set) + pushStack: function( elems ) { + + // Build a new jQuery matched element set + var ret = jQuery.merge( this.constructor(), elems ); + + // Add the old object onto the stack (as a reference) + ret.prevObject = this; + + // Return the newly-formed element set + return ret; + }, + + // Execute a callback for every element in the matched set. + each: function( callback ) { + return jQuery.each( this, callback ); + }, + + map: function( callback ) { + return this.pushStack( jQuery.map( this, function( elem, i ) { + return callback.call( elem, i, elem ); + } ) ); + }, + + slice: function() { + return this.pushStack( slice.apply( this, arguments ) ); + }, + + first: function() { + return this.eq( 0 ); + }, + + last: function() { + return this.eq( -1 ); + }, + + even: function() { + return this.pushStack( jQuery.grep( this, function( _elem, i ) { + return ( i + 1 ) % 2; + } ) ); + }, + + odd: function() { + return this.pushStack( jQuery.grep( this, function( _elem, i ) { + return i % 2; + } ) ); + }, + + eq: function( i ) { + var len = this.length, + j = +i + ( i < 0 ? len : 0 ); + return this.pushStack( j >= 0 && j < len ? [ this[ j ] ] : [] ); + }, + + end: function() { + return this.prevObject || this.constructor(); + }, + + // For internal use only. + // Behaves like an Array's method, not like a jQuery method. + push: push, + sort: arr.sort, + splice: arr.splice +}; + +jQuery.extend = jQuery.fn.extend = function() { + var options, name, src, copy, copyIsArray, clone, + target = arguments[ 0 ] || {}, + i = 1, + length = arguments.length, + deep = false; + + // Handle a deep copy situation + if ( typeof target === "boolean" ) { + deep = target; + + // Skip the boolean and the target + target = arguments[ i ] || {}; + i++; + } + + // Handle case when target is a string or something (possible in deep copy) + if ( typeof target !== "object" && !isFunction( target ) ) { + target = {}; + } + + // Extend jQuery itself if only one argument is passed + if ( i === length ) { + target = this; + i--; + } + + for ( ; i < length; i++ ) { + + // Only deal with non-null/undefined values + if ( ( options = arguments[ i ] ) != null ) { + + // Extend the base object + for ( name in options ) { + copy = options[ name ]; + + // Prevent Object.prototype pollution + // Prevent never-ending loop + if ( name === "__proto__" || target === copy ) { + continue; + } + + // Recurse if we're merging plain objects or arrays + if ( deep && copy && ( jQuery.isPlainObject( copy ) || + ( copyIsArray = Array.isArray( copy ) ) ) ) { + src = target[ name ]; + + // Ensure proper type for the source value + if ( copyIsArray && !Array.isArray( src ) ) { + clone = []; + } else if ( !copyIsArray && !jQuery.isPlainObject( src ) ) { + clone = {}; + } else { + clone = src; + } + copyIsArray = false; + + // Never move original objects, clone them + target[ name ] = jQuery.extend( deep, clone, copy ); + + // Don't bring in undefined values + } else if ( copy !== undefined ) { + target[ name ] = copy; + } + } + } + } + + // Return the modified object + return target; +}; + +jQuery.extend( { + + // Unique for each copy of jQuery on the page + expando: "jQuery" + ( version + Math.random() ).replace( /\D/g, "" ), + + // Assume jQuery is ready without the ready module + isReady: true, + + error: function( msg ) { + throw new Error( msg ); + }, + + noop: function() {}, + + isPlainObject: function( obj ) { + var proto, Ctor; + + // Detect obvious negatives + // Use toString instead of jQuery.type to catch host objects + if ( !obj || toString.call( obj ) !== "[object Object]" ) { + return false; + } + + proto = getProto( obj ); + + // Objects with no prototype (e.g., `Object.create( null )`) are plain + if ( !proto ) { + return true; + } + + // Objects with prototype are plain iff they were constructed by a global Object function + Ctor = hasOwn.call( proto, "constructor" ) && proto.constructor; + return typeof Ctor === "function" && fnToString.call( Ctor ) === ObjectFunctionString; + }, + + isEmptyObject: function( obj ) { + var name; + + for ( name in obj ) { + return false; + } + return true; + }, + + // Evaluates a script in a provided context; falls back to the global one + // if not specified. + globalEval: function( code, options, doc ) { + DOMEval( code, { nonce: options && options.nonce }, doc ); + }, + + each: function( obj, callback ) { + var length, i = 0; + + if ( isArrayLike( obj ) ) { + length = obj.length; + for ( ; i < length; i++ ) { + if ( callback.call( obj[ i ], i, obj[ i ] ) === false ) { + break; + } + } + } else { + for ( i in obj ) { + if ( callback.call( obj[ i ], i, obj[ i ] ) === false ) { + break; + } + } + } + + return obj; + }, + + // results is for internal usage only + makeArray: function( arr, results ) { + var ret = results || []; + + if ( arr != null ) { + if ( isArrayLike( Object( arr ) ) ) { + jQuery.merge( ret, + typeof arr === "string" ? + [ arr ] : arr + ); + } else { + push.call( ret, arr ); + } + } + + return ret; + }, + + inArray: function( elem, arr, i ) { + return arr == null ? -1 : indexOf.call( arr, elem, i ); + }, + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + merge: function( first, second ) { + var len = +second.length, + j = 0, + i = first.length; + + for ( ; j < len; j++ ) { + first[ i++ ] = second[ j ]; + } + + first.length = i; + + return first; + }, + + grep: function( elems, callback, invert ) { + var callbackInverse, + matches = [], + i = 0, + length = elems.length, + callbackExpect = !invert; + + // Go through the array, only saving the items + // that pass the validator function + for ( ; i < length; i++ ) { + callbackInverse = !callback( elems[ i ], i ); + if ( callbackInverse !== callbackExpect ) { + matches.push( elems[ i ] ); + } + } + + return matches; + }, + + // arg is for internal usage only + map: function( elems, callback, arg ) { + var length, value, + i = 0, + ret = []; + + // Go through the array, translating each of the items to their new values + if ( isArrayLike( elems ) ) { + length = elems.length; + for ( ; i < length; i++ ) { + value = callback( elems[ i ], i, arg ); + + if ( value != null ) { + ret.push( value ); + } + } + + // Go through every key on the object, + } else { + for ( i in elems ) { + value = callback( elems[ i ], i, arg ); + + if ( value != null ) { + ret.push( value ); + } + } + } + + // Flatten any nested arrays + return flat( ret ); + }, + + // A global GUID counter for objects + guid: 1, + + // jQuery.support is not used in Core but other projects attach their + // properties to it so it needs to exist. + support: support +} ); + +if ( typeof Symbol === "function" ) { + jQuery.fn[ Symbol.iterator ] = arr[ Symbol.iterator ]; +} + +// Populate the class2type map +jQuery.each( "Boolean Number String Function Array Date RegExp Object Error Symbol".split( " " ), + function( _i, name ) { + class2type[ "[object " + name + "]" ] = name.toLowerCase(); + } ); + +function isArrayLike( obj ) { + + // Support: real iOS 8.2 only (not reproducible in simulator) + // `in` check used to prevent JIT error (gh-2145) + // hasOwn isn't used here due to false negatives + // regarding Nodelist length in IE + var length = !!obj && "length" in obj && obj.length, + type = toType( obj ); + + if ( isFunction( obj ) || isWindow( obj ) ) { + return false; + } + + return type === "array" || length === 0 || + typeof length === "number" && length > 0 && ( length - 1 ) in obj; +} +var Sizzle = +/*! + * Sizzle CSS Selector Engine v2.3.6 + * https://sizzlejs.com/ + * + * Copyright JS Foundation and other contributors + * Released under the MIT license + * https://js.foundation/ + * + * Date: 2021-02-16 + */ +( function( window ) { +var i, + support, + Expr, + getText, + isXML, + tokenize, + compile, + select, + outermostContext, + sortInput, + hasDuplicate, + + // Local document vars + setDocument, + document, + docElem, + documentIsHTML, + rbuggyQSA, + rbuggyMatches, + matches, + contains, + + // Instance-specific data + expando = "sizzle" + 1 * new Date(), + preferredDoc = window.document, + dirruns = 0, + done = 0, + classCache = createCache(), + tokenCache = createCache(), + compilerCache = createCache(), + nonnativeSelectorCache = createCache(), + sortOrder = function( a, b ) { + if ( a === b ) { + hasDuplicate = true; + } + return 0; + }, + + // Instance methods + hasOwn = ( {} ).hasOwnProperty, + arr = [], + pop = arr.pop, + pushNative = arr.push, + push = arr.push, + slice = arr.slice, + + // Use a stripped-down indexOf as it's faster than native + // https://jsperf.com/thor-indexof-vs-for/5 + indexOf = function( list, elem ) { + var i = 0, + len = list.length; + for ( ; i < len; i++ ) { + if ( list[ i ] === elem ) { + return i; + } + } + return -1; + }, + + booleans = "checked|selected|async|autofocus|autoplay|controls|defer|disabled|hidden|" + + "ismap|loop|multiple|open|readonly|required|scoped", + + // Regular expressions + + // http://www.w3.org/TR/css3-selectors/#whitespace + whitespace = "[\\x20\\t\\r\\n\\f]", + + // https://www.w3.org/TR/css-syntax-3/#ident-token-diagram + identifier = "(?:\\\\[\\da-fA-F]{1,6}" + whitespace + + "?|\\\\[^\\r\\n\\f]|[\\w-]|[^\0-\\x7f])+", + + // Attribute selectors: http://www.w3.org/TR/selectors/#attribute-selectors + attributes = "\\[" + whitespace + "*(" + identifier + ")(?:" + whitespace + + + // Operator (capture 2) + "*([*^$|!~]?=)" + whitespace + + + // "Attribute values must be CSS identifiers [capture 5] + // or strings [capture 3 or capture 4]" + "*(?:'((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\"|(" + identifier + "))|)" + + whitespace + "*\\]", + + pseudos = ":(" + identifier + ")(?:\\((" + + + // To reduce the number of selectors needing tokenize in the preFilter, prefer arguments: + // 1. quoted (capture 3; capture 4 or capture 5) + "('((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\")|" + + + // 2. simple (capture 6) + "((?:\\\\.|[^\\\\()[\\]]|" + attributes + ")*)|" + + + // 3. anything else (capture 2) + ".*" + + ")\\)|)", + + // Leading and non-escaped trailing whitespace, capturing some non-whitespace characters preceding the latter + rwhitespace = new RegExp( whitespace + "+", "g" ), + rtrim = new RegExp( "^" + whitespace + "+|((?:^|[^\\\\])(?:\\\\.)*)" + + whitespace + "+$", "g" ), + + rcomma = new RegExp( "^" + whitespace + "*," + whitespace + "*" ), + rcombinators = new RegExp( "^" + whitespace + "*([>+~]|" + whitespace + ")" + whitespace + + "*" ), + rdescend = new RegExp( whitespace + "|>" ), + + rpseudo = new RegExp( pseudos ), + ridentifier = new RegExp( "^" + identifier + "$" ), + + matchExpr = { + "ID": new RegExp( "^#(" + identifier + ")" ), + "CLASS": new RegExp( "^\\.(" + identifier + ")" ), + "TAG": new RegExp( "^(" + identifier + "|[*])" ), + "ATTR": new RegExp( "^" + attributes ), + "PSEUDO": new RegExp( "^" + pseudos ), + "CHILD": new RegExp( "^:(only|first|last|nth|nth-last)-(child|of-type)(?:\\(" + + whitespace + "*(even|odd|(([+-]|)(\\d*)n|)" + whitespace + "*(?:([+-]|)" + + whitespace + "*(\\d+)|))" + whitespace + "*\\)|)", "i" ), + "bool": new RegExp( "^(?:" + booleans + ")$", "i" ), + + // For use in libraries implementing .is() + // We use this for POS matching in `select` + "needsContext": new RegExp( "^" + whitespace + + "*[>+~]|:(even|odd|eq|gt|lt|nth|first|last)(?:\\(" + whitespace + + "*((?:-\\d)?\\d*)" + whitespace + "*\\)|)(?=[^-]|$)", "i" ) + }, + + rhtml = /HTML$/i, + rinputs = /^(?:input|select|textarea|button)$/i, + rheader = /^h\d$/i, + + rnative = /^[^{]+\{\s*\[native \w/, + + // Easily-parseable/retrievable ID or TAG or CLASS selectors + rquickExpr = /^(?:#([\w-]+)|(\w+)|\.([\w-]+))$/, + + rsibling = /[+~]/, + + // CSS escapes + // http://www.w3.org/TR/CSS21/syndata.html#escaped-characters + runescape = new RegExp( "\\\\[\\da-fA-F]{1,6}" + whitespace + "?|\\\\([^\\r\\n\\f])", "g" ), + funescape = function( escape, nonHex ) { + var high = "0x" + escape.slice( 1 ) - 0x10000; + + return nonHex ? + + // Strip the backslash prefix from a non-hex escape sequence + nonHex : + + // Replace a hexadecimal escape sequence with the encoded Unicode code point + // Support: IE <=11+ + // For values outside the Basic Multilingual Plane (BMP), manually construct a + // surrogate pair + high < 0 ? + String.fromCharCode( high + 0x10000 ) : + String.fromCharCode( high >> 10 | 0xD800, high & 0x3FF | 0xDC00 ); + }, + + // CSS string/identifier serialization + // https://drafts.csswg.org/cssom/#common-serializing-idioms + rcssescape = /([\0-\x1f\x7f]|^-?\d)|^-$|[^\0-\x1f\x7f-\uFFFF\w-]/g, + fcssescape = function( ch, asCodePoint ) { + if ( asCodePoint ) { + + // U+0000 NULL becomes U+FFFD REPLACEMENT CHARACTER + if ( ch === "\0" ) { + return "\uFFFD"; + } + + // Control characters and (dependent upon position) numbers get escaped as code points + return ch.slice( 0, -1 ) + "\\" + + ch.charCodeAt( ch.length - 1 ).toString( 16 ) + " "; + } + + // Other potentially-special ASCII characters get backslash-escaped + return "\\" + ch; + }, + + // Used for iframes + // See setDocument() + // Removing the function wrapper causes a "Permission Denied" + // error in IE + unloadHandler = function() { + setDocument(); + }, + + inDisabledFieldset = addCombinator( + function( elem ) { + return elem.disabled === true && elem.nodeName.toLowerCase() === "fieldset"; + }, + { dir: "parentNode", next: "legend" } + ); + +// Optimize for push.apply( _, NodeList ) +try { + push.apply( + ( arr = slice.call( preferredDoc.childNodes ) ), + preferredDoc.childNodes + ); + + // Support: Android<4.0 + // Detect silently failing push.apply + // eslint-disable-next-line no-unused-expressions + arr[ preferredDoc.childNodes.length ].nodeType; +} catch ( e ) { + push = { apply: arr.length ? + + // Leverage slice if possible + function( target, els ) { + pushNative.apply( target, slice.call( els ) ); + } : + + // Support: IE<9 + // Otherwise append directly + function( target, els ) { + var j = target.length, + i = 0; + + // Can't trust NodeList.length + while ( ( target[ j++ ] = els[ i++ ] ) ) {} + target.length = j - 1; + } + }; +} + +function Sizzle( selector, context, results, seed ) { + var m, i, elem, nid, match, groups, newSelector, + newContext = context && context.ownerDocument, + + // nodeType defaults to 9, since context defaults to document + nodeType = context ? context.nodeType : 9; + + results = results || []; + + // Return early from calls with invalid selector or context + if ( typeof selector !== "string" || !selector || + nodeType !== 1 && nodeType !== 9 && nodeType !== 11 ) { + + return results; + } + + // Try to shortcut find operations (as opposed to filters) in HTML documents + if ( !seed ) { + setDocument( context ); + context = context || document; + + if ( documentIsHTML ) { + + // If the selector is sufficiently simple, try using a "get*By*" DOM method + // (excepting DocumentFragment context, where the methods don't exist) + if ( nodeType !== 11 && ( match = rquickExpr.exec( selector ) ) ) { + + // ID selector + if ( ( m = match[ 1 ] ) ) { + + // Document context + if ( nodeType === 9 ) { + if ( ( elem = context.getElementById( m ) ) ) { + + // Support: IE, Opera, Webkit + // TODO: identify versions + // getElementById can match elements by name instead of ID + if ( elem.id === m ) { + results.push( elem ); + return results; + } + } else { + return results; + } + + // Element context + } else { + + // Support: IE, Opera, Webkit + // TODO: identify versions + // getElementById can match elements by name instead of ID + if ( newContext && ( elem = newContext.getElementById( m ) ) && + contains( context, elem ) && + elem.id === m ) { + + results.push( elem ); + return results; + } + } + + // Type selector + } else if ( match[ 2 ] ) { + push.apply( results, context.getElementsByTagName( selector ) ); + return results; + + // Class selector + } else if ( ( m = match[ 3 ] ) && support.getElementsByClassName && + context.getElementsByClassName ) { + + push.apply( results, context.getElementsByClassName( m ) ); + return results; + } + } + + // Take advantage of querySelectorAll + if ( support.qsa && + !nonnativeSelectorCache[ selector + " " ] && + ( !rbuggyQSA || !rbuggyQSA.test( selector ) ) && + + // Support: IE 8 only + // Exclude object elements + ( nodeType !== 1 || context.nodeName.toLowerCase() !== "object" ) ) { + + newSelector = selector; + newContext = context; + + // qSA considers elements outside a scoping root when evaluating child or + // descendant combinators, which is not what we want. + // In such cases, we work around the behavior by prefixing every selector in the + // list with an ID selector referencing the scope context. + // The technique has to be used as well when a leading combinator is used + // as such selectors are not recognized by querySelectorAll. + // Thanks to Andrew Dupont for this technique. + if ( nodeType === 1 && + ( rdescend.test( selector ) || rcombinators.test( selector ) ) ) { + + // Expand context for sibling selectors + newContext = rsibling.test( selector ) && testContext( context.parentNode ) || + context; + + // We can use :scope instead of the ID hack if the browser + // supports it & if we're not changing the context. + if ( newContext !== context || !support.scope ) { + + // Capture the context ID, setting it first if necessary + if ( ( nid = context.getAttribute( "id" ) ) ) { + nid = nid.replace( rcssescape, fcssescape ); + } else { + context.setAttribute( "id", ( nid = expando ) ); + } + } + + // Prefix every selector in the list + groups = tokenize( selector ); + i = groups.length; + while ( i-- ) { + groups[ i ] = ( nid ? "#" + nid : ":scope" ) + " " + + toSelector( groups[ i ] ); + } + newSelector = groups.join( "," ); + } + + try { + push.apply( results, + newContext.querySelectorAll( newSelector ) + ); + return results; + } catch ( qsaError ) { + nonnativeSelectorCache( selector, true ); + } finally { + if ( nid === expando ) { + context.removeAttribute( "id" ); + } + } + } + } + } + + // All others + return select( selector.replace( rtrim, "$1" ), context, results, seed ); +} + +/** + * Create key-value caches of limited size + * @returns {function(string, object)} Returns the Object data after storing it on itself with + * property name the (space-suffixed) string and (if the cache is larger than Expr.cacheLength) + * deleting the oldest entry + */ +function createCache() { + var keys = []; + + function cache( key, value ) { + + // Use (key + " ") to avoid collision with native prototype properties (see Issue #157) + if ( keys.push( key + " " ) > Expr.cacheLength ) { + + // Only keep the most recent entries + delete cache[ keys.shift() ]; + } + return ( cache[ key + " " ] = value ); + } + return cache; +} + +/** + * Mark a function for special use by Sizzle + * @param {Function} fn The function to mark + */ +function markFunction( fn ) { + fn[ expando ] = true; + return fn; +} + +/** + * Support testing using an element + * @param {Function} fn Passed the created element and returns a boolean result + */ +function assert( fn ) { + var el = document.createElement( "fieldset" ); + + try { + return !!fn( el ); + } catch ( e ) { + return false; + } finally { + + // Remove from its parent by default + if ( el.parentNode ) { + el.parentNode.removeChild( el ); + } + + // release memory in IE + el = null; + } +} + +/** + * Adds the same handler for all of the specified attrs + * @param {String} attrs Pipe-separated list of attributes + * @param {Function} handler The method that will be applied + */ +function addHandle( attrs, handler ) { + var arr = attrs.split( "|" ), + i = arr.length; + + while ( i-- ) { + Expr.attrHandle[ arr[ i ] ] = handler; + } +} + +/** + * Checks document order of two siblings + * @param {Element} a + * @param {Element} b + * @returns {Number} Returns less than 0 if a precedes b, greater than 0 if a follows b + */ +function siblingCheck( a, b ) { + var cur = b && a, + diff = cur && a.nodeType === 1 && b.nodeType === 1 && + a.sourceIndex - b.sourceIndex; + + // Use IE sourceIndex if available on both nodes + if ( diff ) { + return diff; + } + + // Check if b follows a + if ( cur ) { + while ( ( cur = cur.nextSibling ) ) { + if ( cur === b ) { + return -1; + } + } + } + + return a ? 1 : -1; +} + +/** + * Returns a function to use in pseudos for input types + * @param {String} type + */ +function createInputPseudo( type ) { + return function( elem ) { + var name = elem.nodeName.toLowerCase(); + return name === "input" && elem.type === type; + }; +} + +/** + * Returns a function to use in pseudos for buttons + * @param {String} type + */ +function createButtonPseudo( type ) { + return function( elem ) { + var name = elem.nodeName.toLowerCase(); + return ( name === "input" || name === "button" ) && elem.type === type; + }; +} + +/** + * Returns a function to use in pseudos for :enabled/:disabled + * @param {Boolean} disabled true for :disabled; false for :enabled + */ +function createDisabledPseudo( disabled ) { + + // Known :disabled false positives: fieldset[disabled] > legend:nth-of-type(n+2) :can-disable + return function( elem ) { + + // Only certain elements can match :enabled or :disabled + // https://html.spec.whatwg.org/multipage/scripting.html#selector-enabled + // https://html.spec.whatwg.org/multipage/scripting.html#selector-disabled + if ( "form" in elem ) { + + // Check for inherited disabledness on relevant non-disabled elements: + // * listed form-associated elements in a disabled fieldset + // https://html.spec.whatwg.org/multipage/forms.html#category-listed + // https://html.spec.whatwg.org/multipage/forms.html#concept-fe-disabled + // * option elements in a disabled optgroup + // https://html.spec.whatwg.org/multipage/forms.html#concept-option-disabled + // All such elements have a "form" property. + if ( elem.parentNode && elem.disabled === false ) { + + // Option elements defer to a parent optgroup if present + if ( "label" in elem ) { + if ( "label" in elem.parentNode ) { + return elem.parentNode.disabled === disabled; + } else { + return elem.disabled === disabled; + } + } + + // Support: IE 6 - 11 + // Use the isDisabled shortcut property to check for disabled fieldset ancestors + return elem.isDisabled === disabled || + + // Where there is no isDisabled, check manually + /* jshint -W018 */ + elem.isDisabled !== !disabled && + inDisabledFieldset( elem ) === disabled; + } + + return elem.disabled === disabled; + + // Try to winnow out elements that can't be disabled before trusting the disabled property. + // Some victims get caught in our net (label, legend, menu, track), but it shouldn't + // even exist on them, let alone have a boolean value. + } else if ( "label" in elem ) { + return elem.disabled === disabled; + } + + // Remaining elements are neither :enabled nor :disabled + return false; + }; +} + +/** + * Returns a function to use in pseudos for positionals + * @param {Function} fn + */ +function createPositionalPseudo( fn ) { + return markFunction( function( argument ) { + argument = +argument; + return markFunction( function( seed, matches ) { + var j, + matchIndexes = fn( [], seed.length, argument ), + i = matchIndexes.length; + + // Match elements found at the specified indexes + while ( i-- ) { + if ( seed[ ( j = matchIndexes[ i ] ) ] ) { + seed[ j ] = !( matches[ j ] = seed[ j ] ); + } + } + } ); + } ); +} + +/** + * Checks a node for validity as a Sizzle context + * @param {Element|Object=} context + * @returns {Element|Object|Boolean} The input node if acceptable, otherwise a falsy value + */ +function testContext( context ) { + return context && typeof context.getElementsByTagName !== "undefined" && context; +} + +// Expose support vars for convenience +support = Sizzle.support = {}; + +/** + * Detects XML nodes + * @param {Element|Object} elem An element or a document + * @returns {Boolean} True iff elem is a non-HTML XML node + */ +isXML = Sizzle.isXML = function( elem ) { + var namespace = elem && elem.namespaceURI, + docElem = elem && ( elem.ownerDocument || elem ).documentElement; + + // Support: IE <=8 + // Assume HTML when documentElement doesn't yet exist, such as inside loading iframes + // https://bugs.jquery.com/ticket/4833 + return !rhtml.test( namespace || docElem && docElem.nodeName || "HTML" ); +}; + +/** + * Sets document-related variables once based on the current document + * @param {Element|Object} [doc] An element or document object to use to set the document + * @returns {Object} Returns the current document + */ +setDocument = Sizzle.setDocument = function( node ) { + var hasCompare, subWindow, + doc = node ? node.ownerDocument || node : preferredDoc; + + // Return early if doc is invalid or already selected + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( doc == document || doc.nodeType !== 9 || !doc.documentElement ) { + return document; + } + + // Update global variables + document = doc; + docElem = document.documentElement; + documentIsHTML = !isXML( document ); + + // Support: IE 9 - 11+, Edge 12 - 18+ + // Accessing iframe documents after unload throws "permission denied" errors (jQuery #13936) + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( preferredDoc != document && + ( subWindow = document.defaultView ) && subWindow.top !== subWindow ) { + + // Support: IE 11, Edge + if ( subWindow.addEventListener ) { + subWindow.addEventListener( "unload", unloadHandler, false ); + + // Support: IE 9 - 10 only + } else if ( subWindow.attachEvent ) { + subWindow.attachEvent( "onunload", unloadHandler ); + } + } + + // Support: IE 8 - 11+, Edge 12 - 18+, Chrome <=16 - 25 only, Firefox <=3.6 - 31 only, + // Safari 4 - 5 only, Opera <=11.6 - 12.x only + // IE/Edge & older browsers don't support the :scope pseudo-class. + // Support: Safari 6.0 only + // Safari 6.0 supports :scope but it's an alias of :root there. + support.scope = assert( function( el ) { + docElem.appendChild( el ).appendChild( document.createElement( "div" ) ); + return typeof el.querySelectorAll !== "undefined" && + !el.querySelectorAll( ":scope fieldset div" ).length; + } ); + + /* Attributes + ---------------------------------------------------------------------- */ + + // Support: IE<8 + // Verify that getAttribute really returns attributes and not properties + // (excepting IE8 booleans) + support.attributes = assert( function( el ) { + el.className = "i"; + return !el.getAttribute( "className" ); + } ); + + /* getElement(s)By* + ---------------------------------------------------------------------- */ + + // Check if getElementsByTagName("*") returns only elements + support.getElementsByTagName = assert( function( el ) { + el.appendChild( document.createComment( "" ) ); + return !el.getElementsByTagName( "*" ).length; + } ); + + // Support: IE<9 + support.getElementsByClassName = rnative.test( document.getElementsByClassName ); + + // Support: IE<10 + // Check if getElementById returns elements by name + // The broken getElementById methods don't pick up programmatically-set names, + // so use a roundabout getElementsByName test + support.getById = assert( function( el ) { + docElem.appendChild( el ).id = expando; + return !document.getElementsByName || !document.getElementsByName( expando ).length; + } ); + + // ID filter and find + if ( support.getById ) { + Expr.filter[ "ID" ] = function( id ) { + var attrId = id.replace( runescape, funescape ); + return function( elem ) { + return elem.getAttribute( "id" ) === attrId; + }; + }; + Expr.find[ "ID" ] = function( id, context ) { + if ( typeof context.getElementById !== "undefined" && documentIsHTML ) { + var elem = context.getElementById( id ); + return elem ? [ elem ] : []; + } + }; + } else { + Expr.filter[ "ID" ] = function( id ) { + var attrId = id.replace( runescape, funescape ); + return function( elem ) { + var node = typeof elem.getAttributeNode !== "undefined" && + elem.getAttributeNode( "id" ); + return node && node.value === attrId; + }; + }; + + // Support: IE 6 - 7 only + // getElementById is not reliable as a find shortcut + Expr.find[ "ID" ] = function( id, context ) { + if ( typeof context.getElementById !== "undefined" && documentIsHTML ) { + var node, i, elems, + elem = context.getElementById( id ); + + if ( elem ) { + + // Verify the id attribute + node = elem.getAttributeNode( "id" ); + if ( node && node.value === id ) { + return [ elem ]; + } + + // Fall back on getElementsByName + elems = context.getElementsByName( id ); + i = 0; + while ( ( elem = elems[ i++ ] ) ) { + node = elem.getAttributeNode( "id" ); + if ( node && node.value === id ) { + return [ elem ]; + } + } + } + + return []; + } + }; + } + + // Tag + Expr.find[ "TAG" ] = support.getElementsByTagName ? + function( tag, context ) { + if ( typeof context.getElementsByTagName !== "undefined" ) { + return context.getElementsByTagName( tag ); + + // DocumentFragment nodes don't have gEBTN + } else if ( support.qsa ) { + return context.querySelectorAll( tag ); + } + } : + + function( tag, context ) { + var elem, + tmp = [], + i = 0, + + // By happy coincidence, a (broken) gEBTN appears on DocumentFragment nodes too + results = context.getElementsByTagName( tag ); + + // Filter out possible comments + if ( tag === "*" ) { + while ( ( elem = results[ i++ ] ) ) { + if ( elem.nodeType === 1 ) { + tmp.push( elem ); + } + } + + return tmp; + } + return results; + }; + + // Class + Expr.find[ "CLASS" ] = support.getElementsByClassName && function( className, context ) { + if ( typeof context.getElementsByClassName !== "undefined" && documentIsHTML ) { + return context.getElementsByClassName( className ); + } + }; + + /* QSA/matchesSelector + ---------------------------------------------------------------------- */ + + // QSA and matchesSelector support + + // matchesSelector(:active) reports false when true (IE9/Opera 11.5) + rbuggyMatches = []; + + // qSa(:focus) reports false when true (Chrome 21) + // We allow this because of a bug in IE8/9 that throws an error + // whenever `document.activeElement` is accessed on an iframe + // So, we allow :focus to pass through QSA all the time to avoid the IE error + // See https://bugs.jquery.com/ticket/13378 + rbuggyQSA = []; + + if ( ( support.qsa = rnative.test( document.querySelectorAll ) ) ) { + + // Build QSA regex + // Regex strategy adopted from Diego Perini + assert( function( el ) { + + var input; + + // Select is set to empty string on purpose + // This is to test IE's treatment of not explicitly + // setting a boolean content attribute, + // since its presence should be enough + // https://bugs.jquery.com/ticket/12359 + docElem.appendChild( el ).innerHTML = "" + + ""; + + // Support: IE8, Opera 11-12.16 + // Nothing should be selected when empty strings follow ^= or $= or *= + // The test attribute must be unknown in Opera but "safe" for WinRT + // https://msdn.microsoft.com/en-us/library/ie/hh465388.aspx#attribute_section + if ( el.querySelectorAll( "[msallowcapture^='']" ).length ) { + rbuggyQSA.push( "[*^$]=" + whitespace + "*(?:''|\"\")" ); + } + + // Support: IE8 + // Boolean attributes and "value" are not treated correctly + if ( !el.querySelectorAll( "[selected]" ).length ) { + rbuggyQSA.push( "\\[" + whitespace + "*(?:value|" + booleans + ")" ); + } + + // Support: Chrome<29, Android<4.4, Safari<7.0+, iOS<7.0+, PhantomJS<1.9.8+ + if ( !el.querySelectorAll( "[id~=" + expando + "-]" ).length ) { + rbuggyQSA.push( "~=" ); + } + + // Support: IE 11+, Edge 15 - 18+ + // IE 11/Edge don't find elements on a `[name='']` query in some cases. + // Adding a temporary attribute to the document before the selection works + // around the issue. + // Interestingly, IE 10 & older don't seem to have the issue. + input = document.createElement( "input" ); + input.setAttribute( "name", "" ); + el.appendChild( input ); + if ( !el.querySelectorAll( "[name='']" ).length ) { + rbuggyQSA.push( "\\[" + whitespace + "*name" + whitespace + "*=" + + whitespace + "*(?:''|\"\")" ); + } + + // Webkit/Opera - :checked should return selected option elements + // http://www.w3.org/TR/2011/REC-css3-selectors-20110929/#checked + // IE8 throws error here and will not see later tests + if ( !el.querySelectorAll( ":checked" ).length ) { + rbuggyQSA.push( ":checked" ); + } + + // Support: Safari 8+, iOS 8+ + // https://bugs.webkit.org/show_bug.cgi?id=136851 + // In-page `selector#id sibling-combinator selector` fails + if ( !el.querySelectorAll( "a#" + expando + "+*" ).length ) { + rbuggyQSA.push( ".#.+[+~]" ); + } + + // Support: Firefox <=3.6 - 5 only + // Old Firefox doesn't throw on a badly-escaped identifier. + el.querySelectorAll( "\\\f" ); + rbuggyQSA.push( "[\\r\\n\\f]" ); + } ); + + assert( function( el ) { + el.innerHTML = "" + + ""; + + // Support: Windows 8 Native Apps + // The type and name attributes are restricted during .innerHTML assignment + var input = document.createElement( "input" ); + input.setAttribute( "type", "hidden" ); + el.appendChild( input ).setAttribute( "name", "D" ); + + // Support: IE8 + // Enforce case-sensitivity of name attribute + if ( el.querySelectorAll( "[name=d]" ).length ) { + rbuggyQSA.push( "name" + whitespace + "*[*^$|!~]?=" ); + } + + // FF 3.5 - :enabled/:disabled and hidden elements (hidden elements are still enabled) + // IE8 throws error here and will not see later tests + if ( el.querySelectorAll( ":enabled" ).length !== 2 ) { + rbuggyQSA.push( ":enabled", ":disabled" ); + } + + // Support: IE9-11+ + // IE's :disabled selector does not pick up the children of disabled fieldsets + docElem.appendChild( el ).disabled = true; + if ( el.querySelectorAll( ":disabled" ).length !== 2 ) { + rbuggyQSA.push( ":enabled", ":disabled" ); + } + + // Support: Opera 10 - 11 only + // Opera 10-11 does not throw on post-comma invalid pseudos + el.querySelectorAll( "*,:x" ); + rbuggyQSA.push( ",.*:" ); + } ); + } + + if ( ( support.matchesSelector = rnative.test( ( matches = docElem.matches || + docElem.webkitMatchesSelector || + docElem.mozMatchesSelector || + docElem.oMatchesSelector || + docElem.msMatchesSelector ) ) ) ) { + + assert( function( el ) { + + // Check to see if it's possible to do matchesSelector + // on a disconnected node (IE 9) + support.disconnectedMatch = matches.call( el, "*" ); + + // This should fail with an exception + // Gecko does not error, returns false instead + matches.call( el, "[s!='']:x" ); + rbuggyMatches.push( "!=", pseudos ); + } ); + } + + rbuggyQSA = rbuggyQSA.length && new RegExp( rbuggyQSA.join( "|" ) ); + rbuggyMatches = rbuggyMatches.length && new RegExp( rbuggyMatches.join( "|" ) ); + + /* Contains + ---------------------------------------------------------------------- */ + hasCompare = rnative.test( docElem.compareDocumentPosition ); + + // Element contains another + // Purposefully self-exclusive + // As in, an element does not contain itself + contains = hasCompare || rnative.test( docElem.contains ) ? + function( a, b ) { + var adown = a.nodeType === 9 ? a.documentElement : a, + bup = b && b.parentNode; + return a === bup || !!( bup && bup.nodeType === 1 && ( + adown.contains ? + adown.contains( bup ) : + a.compareDocumentPosition && a.compareDocumentPosition( bup ) & 16 + ) ); + } : + function( a, b ) { + if ( b ) { + while ( ( b = b.parentNode ) ) { + if ( b === a ) { + return true; + } + } + } + return false; + }; + + /* Sorting + ---------------------------------------------------------------------- */ + + // Document order sorting + sortOrder = hasCompare ? + function( a, b ) { + + // Flag for duplicate removal + if ( a === b ) { + hasDuplicate = true; + return 0; + } + + // Sort on method existence if only one input has compareDocumentPosition + var compare = !a.compareDocumentPosition - !b.compareDocumentPosition; + if ( compare ) { + return compare; + } + + // Calculate position if both inputs belong to the same document + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + compare = ( a.ownerDocument || a ) == ( b.ownerDocument || b ) ? + a.compareDocumentPosition( b ) : + + // Otherwise we know they are disconnected + 1; + + // Disconnected nodes + if ( compare & 1 || + ( !support.sortDetached && b.compareDocumentPosition( a ) === compare ) ) { + + // Choose the first element that is related to our preferred document + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( a == document || a.ownerDocument == preferredDoc && + contains( preferredDoc, a ) ) { + return -1; + } + + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( b == document || b.ownerDocument == preferredDoc && + contains( preferredDoc, b ) ) { + return 1; + } + + // Maintain original order + return sortInput ? + ( indexOf( sortInput, a ) - indexOf( sortInput, b ) ) : + 0; + } + + return compare & 4 ? -1 : 1; + } : + function( a, b ) { + + // Exit early if the nodes are identical + if ( a === b ) { + hasDuplicate = true; + return 0; + } + + var cur, + i = 0, + aup = a.parentNode, + bup = b.parentNode, + ap = [ a ], + bp = [ b ]; + + // Parentless nodes are either documents or disconnected + if ( !aup || !bup ) { + + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + /* eslint-disable eqeqeq */ + return a == document ? -1 : + b == document ? 1 : + /* eslint-enable eqeqeq */ + aup ? -1 : + bup ? 1 : + sortInput ? + ( indexOf( sortInput, a ) - indexOf( sortInput, b ) ) : + 0; + + // If the nodes are siblings, we can do a quick check + } else if ( aup === bup ) { + return siblingCheck( a, b ); + } + + // Otherwise we need full lists of their ancestors for comparison + cur = a; + while ( ( cur = cur.parentNode ) ) { + ap.unshift( cur ); + } + cur = b; + while ( ( cur = cur.parentNode ) ) { + bp.unshift( cur ); + } + + // Walk down the tree looking for a discrepancy + while ( ap[ i ] === bp[ i ] ) { + i++; + } + + return i ? + + // Do a sibling check if the nodes have a common ancestor + siblingCheck( ap[ i ], bp[ i ] ) : + + // Otherwise nodes in our document sort first + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + /* eslint-disable eqeqeq */ + ap[ i ] == preferredDoc ? -1 : + bp[ i ] == preferredDoc ? 1 : + /* eslint-enable eqeqeq */ + 0; + }; + + return document; +}; + +Sizzle.matches = function( expr, elements ) { + return Sizzle( expr, null, null, elements ); +}; + +Sizzle.matchesSelector = function( elem, expr ) { + setDocument( elem ); + + if ( support.matchesSelector && documentIsHTML && + !nonnativeSelectorCache[ expr + " " ] && + ( !rbuggyMatches || !rbuggyMatches.test( expr ) ) && + ( !rbuggyQSA || !rbuggyQSA.test( expr ) ) ) { + + try { + var ret = matches.call( elem, expr ); + + // IE 9's matchesSelector returns false on disconnected nodes + if ( ret || support.disconnectedMatch || + + // As well, disconnected nodes are said to be in a document + // fragment in IE 9 + elem.document && elem.document.nodeType !== 11 ) { + return ret; + } + } catch ( e ) { + nonnativeSelectorCache( expr, true ); + } + } + + return Sizzle( expr, document, null, [ elem ] ).length > 0; +}; + +Sizzle.contains = function( context, elem ) { + + // Set document vars if needed + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( ( context.ownerDocument || context ) != document ) { + setDocument( context ); + } + return contains( context, elem ); +}; + +Sizzle.attr = function( elem, name ) { + + // Set document vars if needed + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( ( elem.ownerDocument || elem ) != document ) { + setDocument( elem ); + } + + var fn = Expr.attrHandle[ name.toLowerCase() ], + + // Don't get fooled by Object.prototype properties (jQuery #13807) + val = fn && hasOwn.call( Expr.attrHandle, name.toLowerCase() ) ? + fn( elem, name, !documentIsHTML ) : + undefined; + + return val !== undefined ? + val : + support.attributes || !documentIsHTML ? + elem.getAttribute( name ) : + ( val = elem.getAttributeNode( name ) ) && val.specified ? + val.value : + null; +}; + +Sizzle.escape = function( sel ) { + return ( sel + "" ).replace( rcssescape, fcssescape ); +}; + +Sizzle.error = function( msg ) { + throw new Error( "Syntax error, unrecognized expression: " + msg ); +}; + +/** + * Document sorting and removing duplicates + * @param {ArrayLike} results + */ +Sizzle.uniqueSort = function( results ) { + var elem, + duplicates = [], + j = 0, + i = 0; + + // Unless we *know* we can detect duplicates, assume their presence + hasDuplicate = !support.detectDuplicates; + sortInput = !support.sortStable && results.slice( 0 ); + results.sort( sortOrder ); + + if ( hasDuplicate ) { + while ( ( elem = results[ i++ ] ) ) { + if ( elem === results[ i ] ) { + j = duplicates.push( i ); + } + } + while ( j-- ) { + results.splice( duplicates[ j ], 1 ); + } + } + + // Clear input after sorting to release objects + // See https://github.com/jquery/sizzle/pull/225 + sortInput = null; + + return results; +}; + +/** + * Utility function for retrieving the text value of an array of DOM nodes + * @param {Array|Element} elem + */ +getText = Sizzle.getText = function( elem ) { + var node, + ret = "", + i = 0, + nodeType = elem.nodeType; + + if ( !nodeType ) { + + // If no nodeType, this is expected to be an array + while ( ( node = elem[ i++ ] ) ) { + + // Do not traverse comment nodes + ret += getText( node ); + } + } else if ( nodeType === 1 || nodeType === 9 || nodeType === 11 ) { + + // Use textContent for elements + // innerText usage removed for consistency of new lines (jQuery #11153) + if ( typeof elem.textContent === "string" ) { + return elem.textContent; + } else { + + // Traverse its children + for ( elem = elem.firstChild; elem; elem = elem.nextSibling ) { + ret += getText( elem ); + } + } + } else if ( nodeType === 3 || nodeType === 4 ) { + return elem.nodeValue; + } + + // Do not include comment or processing instruction nodes + + return ret; +}; + +Expr = Sizzle.selectors = { + + // Can be adjusted by the user + cacheLength: 50, + + createPseudo: markFunction, + + match: matchExpr, + + attrHandle: {}, + + find: {}, + + relative: { + ">": { dir: "parentNode", first: true }, + " ": { dir: "parentNode" }, + "+": { dir: "previousSibling", first: true }, + "~": { dir: "previousSibling" } + }, + + preFilter: { + "ATTR": function( match ) { + match[ 1 ] = match[ 1 ].replace( runescape, funescape ); + + // Move the given value to match[3] whether quoted or unquoted + match[ 3 ] = ( match[ 3 ] || match[ 4 ] || + match[ 5 ] || "" ).replace( runescape, funescape ); + + if ( match[ 2 ] === "~=" ) { + match[ 3 ] = " " + match[ 3 ] + " "; + } + + return match.slice( 0, 4 ); + }, + + "CHILD": function( match ) { + + /* matches from matchExpr["CHILD"] + 1 type (only|nth|...) + 2 what (child|of-type) + 3 argument (even|odd|\d*|\d*n([+-]\d+)?|...) + 4 xn-component of xn+y argument ([+-]?\d*n|) + 5 sign of xn-component + 6 x of xn-component + 7 sign of y-component + 8 y of y-component + */ + match[ 1 ] = match[ 1 ].toLowerCase(); + + if ( match[ 1 ].slice( 0, 3 ) === "nth" ) { + + // nth-* requires argument + if ( !match[ 3 ] ) { + Sizzle.error( match[ 0 ] ); + } + + // numeric x and y parameters for Expr.filter.CHILD + // remember that false/true cast respectively to 0/1 + match[ 4 ] = +( match[ 4 ] ? + match[ 5 ] + ( match[ 6 ] || 1 ) : + 2 * ( match[ 3 ] === "even" || match[ 3 ] === "odd" ) ); + match[ 5 ] = +( ( match[ 7 ] + match[ 8 ] ) || match[ 3 ] === "odd" ); + + // other types prohibit arguments + } else if ( match[ 3 ] ) { + Sizzle.error( match[ 0 ] ); + } + + return match; + }, + + "PSEUDO": function( match ) { + var excess, + unquoted = !match[ 6 ] && match[ 2 ]; + + if ( matchExpr[ "CHILD" ].test( match[ 0 ] ) ) { + return null; + } + + // Accept quoted arguments as-is + if ( match[ 3 ] ) { + match[ 2 ] = match[ 4 ] || match[ 5 ] || ""; + + // Strip excess characters from unquoted arguments + } else if ( unquoted && rpseudo.test( unquoted ) && + + // Get excess from tokenize (recursively) + ( excess = tokenize( unquoted, true ) ) && + + // advance to the next closing parenthesis + ( excess = unquoted.indexOf( ")", unquoted.length - excess ) - unquoted.length ) ) { + + // excess is a negative index + match[ 0 ] = match[ 0 ].slice( 0, excess ); + match[ 2 ] = unquoted.slice( 0, excess ); + } + + // Return only captures needed by the pseudo filter method (type and argument) + return match.slice( 0, 3 ); + } + }, + + filter: { + + "TAG": function( nodeNameSelector ) { + var nodeName = nodeNameSelector.replace( runescape, funescape ).toLowerCase(); + return nodeNameSelector === "*" ? + function() { + return true; + } : + function( elem ) { + return elem.nodeName && elem.nodeName.toLowerCase() === nodeName; + }; + }, + + "CLASS": function( className ) { + var pattern = classCache[ className + " " ]; + + return pattern || + ( pattern = new RegExp( "(^|" + whitespace + + ")" + className + "(" + whitespace + "|$)" ) ) && classCache( + className, function( elem ) { + return pattern.test( + typeof elem.className === "string" && elem.className || + typeof elem.getAttribute !== "undefined" && + elem.getAttribute( "class" ) || + "" + ); + } ); + }, + + "ATTR": function( name, operator, check ) { + return function( elem ) { + var result = Sizzle.attr( elem, name ); + + if ( result == null ) { + return operator === "!="; + } + if ( !operator ) { + return true; + } + + result += ""; + + /* eslint-disable max-len */ + + return operator === "=" ? result === check : + operator === "!=" ? result !== check : + operator === "^=" ? check && result.indexOf( check ) === 0 : + operator === "*=" ? check && result.indexOf( check ) > -1 : + operator === "$=" ? check && result.slice( -check.length ) === check : + operator === "~=" ? ( " " + result.replace( rwhitespace, " " ) + " " ).indexOf( check ) > -1 : + operator === "|=" ? result === check || result.slice( 0, check.length + 1 ) === check + "-" : + false; + /* eslint-enable max-len */ + + }; + }, + + "CHILD": function( type, what, _argument, first, last ) { + var simple = type.slice( 0, 3 ) !== "nth", + forward = type.slice( -4 ) !== "last", + ofType = what === "of-type"; + + return first === 1 && last === 0 ? + + // Shortcut for :nth-*(n) + function( elem ) { + return !!elem.parentNode; + } : + + function( elem, _context, xml ) { + var cache, uniqueCache, outerCache, node, nodeIndex, start, + dir = simple !== forward ? "nextSibling" : "previousSibling", + parent = elem.parentNode, + name = ofType && elem.nodeName.toLowerCase(), + useCache = !xml && !ofType, + diff = false; + + if ( parent ) { + + // :(first|last|only)-(child|of-type) + if ( simple ) { + while ( dir ) { + node = elem; + while ( ( node = node[ dir ] ) ) { + if ( ofType ? + node.nodeName.toLowerCase() === name : + node.nodeType === 1 ) { + + return false; + } + } + + // Reverse direction for :only-* (if we haven't yet done so) + start = dir = type === "only" && !start && "nextSibling"; + } + return true; + } + + start = [ forward ? parent.firstChild : parent.lastChild ]; + + // non-xml :nth-child(...) stores cache data on `parent` + if ( forward && useCache ) { + + // Seek `elem` from a previously-cached index + + // ...in a gzip-friendly way + node = parent; + outerCache = node[ expando ] || ( node[ expando ] = {} ); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ node.uniqueID ] || + ( outerCache[ node.uniqueID ] = {} ); + + cache = uniqueCache[ type ] || []; + nodeIndex = cache[ 0 ] === dirruns && cache[ 1 ]; + diff = nodeIndex && cache[ 2 ]; + node = nodeIndex && parent.childNodes[ nodeIndex ]; + + while ( ( node = ++nodeIndex && node && node[ dir ] || + + // Fallback to seeking `elem` from the start + ( diff = nodeIndex = 0 ) || start.pop() ) ) { + + // When found, cache indexes on `parent` and break + if ( node.nodeType === 1 && ++diff && node === elem ) { + uniqueCache[ type ] = [ dirruns, nodeIndex, diff ]; + break; + } + } + + } else { + + // Use previously-cached element index if available + if ( useCache ) { + + // ...in a gzip-friendly way + node = elem; + outerCache = node[ expando ] || ( node[ expando ] = {} ); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ node.uniqueID ] || + ( outerCache[ node.uniqueID ] = {} ); + + cache = uniqueCache[ type ] || []; + nodeIndex = cache[ 0 ] === dirruns && cache[ 1 ]; + diff = nodeIndex; + } + + // xml :nth-child(...) + // or :nth-last-child(...) or :nth(-last)?-of-type(...) + if ( diff === false ) { + + // Use the same loop as above to seek `elem` from the start + while ( ( node = ++nodeIndex && node && node[ dir ] || + ( diff = nodeIndex = 0 ) || start.pop() ) ) { + + if ( ( ofType ? + node.nodeName.toLowerCase() === name : + node.nodeType === 1 ) && + ++diff ) { + + // Cache the index of each encountered element + if ( useCache ) { + outerCache = node[ expando ] || + ( node[ expando ] = {} ); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ node.uniqueID ] || + ( outerCache[ node.uniqueID ] = {} ); + + uniqueCache[ type ] = [ dirruns, diff ]; + } + + if ( node === elem ) { + break; + } + } + } + } + } + + // Incorporate the offset, then check against cycle size + diff -= last; + return diff === first || ( diff % first === 0 && diff / first >= 0 ); + } + }; + }, + + "PSEUDO": function( pseudo, argument ) { + + // pseudo-class names are case-insensitive + // http://www.w3.org/TR/selectors/#pseudo-classes + // Prioritize by case sensitivity in case custom pseudos are added with uppercase letters + // Remember that setFilters inherits from pseudos + var args, + fn = Expr.pseudos[ pseudo ] || Expr.setFilters[ pseudo.toLowerCase() ] || + Sizzle.error( "unsupported pseudo: " + pseudo ); + + // The user may use createPseudo to indicate that + // arguments are needed to create the filter function + // just as Sizzle does + if ( fn[ expando ] ) { + return fn( argument ); + } + + // But maintain support for old signatures + if ( fn.length > 1 ) { + args = [ pseudo, pseudo, "", argument ]; + return Expr.setFilters.hasOwnProperty( pseudo.toLowerCase() ) ? + markFunction( function( seed, matches ) { + var idx, + matched = fn( seed, argument ), + i = matched.length; + while ( i-- ) { + idx = indexOf( seed, matched[ i ] ); + seed[ idx ] = !( matches[ idx ] = matched[ i ] ); + } + } ) : + function( elem ) { + return fn( elem, 0, args ); + }; + } + + return fn; + } + }, + + pseudos: { + + // Potentially complex pseudos + "not": markFunction( function( selector ) { + + // Trim the selector passed to compile + // to avoid treating leading and trailing + // spaces as combinators + var input = [], + results = [], + matcher = compile( selector.replace( rtrim, "$1" ) ); + + return matcher[ expando ] ? + markFunction( function( seed, matches, _context, xml ) { + var elem, + unmatched = matcher( seed, null, xml, [] ), + i = seed.length; + + // Match elements unmatched by `matcher` + while ( i-- ) { + if ( ( elem = unmatched[ i ] ) ) { + seed[ i ] = !( matches[ i ] = elem ); + } + } + } ) : + function( elem, _context, xml ) { + input[ 0 ] = elem; + matcher( input, null, xml, results ); + + // Don't keep the element (issue #299) + input[ 0 ] = null; + return !results.pop(); + }; + } ), + + "has": markFunction( function( selector ) { + return function( elem ) { + return Sizzle( selector, elem ).length > 0; + }; + } ), + + "contains": markFunction( function( text ) { + text = text.replace( runescape, funescape ); + return function( elem ) { + return ( elem.textContent || getText( elem ) ).indexOf( text ) > -1; + }; + } ), + + // "Whether an element is represented by a :lang() selector + // is based solely on the element's language value + // being equal to the identifier C, + // or beginning with the identifier C immediately followed by "-". + // The matching of C against the element's language value is performed case-insensitively. + // The identifier C does not have to be a valid language name." + // http://www.w3.org/TR/selectors/#lang-pseudo + "lang": markFunction( function( lang ) { + + // lang value must be a valid identifier + if ( !ridentifier.test( lang || "" ) ) { + Sizzle.error( "unsupported lang: " + lang ); + } + lang = lang.replace( runescape, funescape ).toLowerCase(); + return function( elem ) { + var elemLang; + do { + if ( ( elemLang = documentIsHTML ? + elem.lang : + elem.getAttribute( "xml:lang" ) || elem.getAttribute( "lang" ) ) ) { + + elemLang = elemLang.toLowerCase(); + return elemLang === lang || elemLang.indexOf( lang + "-" ) === 0; + } + } while ( ( elem = elem.parentNode ) && elem.nodeType === 1 ); + return false; + }; + } ), + + // Miscellaneous + "target": function( elem ) { + var hash = window.location && window.location.hash; + return hash && hash.slice( 1 ) === elem.id; + }, + + "root": function( elem ) { + return elem === docElem; + }, + + "focus": function( elem ) { + return elem === document.activeElement && + ( !document.hasFocus || document.hasFocus() ) && + !!( elem.type || elem.href || ~elem.tabIndex ); + }, + + // Boolean properties + "enabled": createDisabledPseudo( false ), + "disabled": createDisabledPseudo( true ), + + "checked": function( elem ) { + + // In CSS3, :checked should return both checked and selected elements + // http://www.w3.org/TR/2011/REC-css3-selectors-20110929/#checked + var nodeName = elem.nodeName.toLowerCase(); + return ( nodeName === "input" && !!elem.checked ) || + ( nodeName === "option" && !!elem.selected ); + }, + + "selected": function( elem ) { + + // Accessing this property makes selected-by-default + // options in Safari work properly + if ( elem.parentNode ) { + // eslint-disable-next-line no-unused-expressions + elem.parentNode.selectedIndex; + } + + return elem.selected === true; + }, + + // Contents + "empty": function( elem ) { + + // http://www.w3.org/TR/selectors/#empty-pseudo + // :empty is negated by element (1) or content nodes (text: 3; cdata: 4; entity ref: 5), + // but not by others (comment: 8; processing instruction: 7; etc.) + // nodeType < 6 works because attributes (2) do not appear as children + for ( elem = elem.firstChild; elem; elem = elem.nextSibling ) { + if ( elem.nodeType < 6 ) { + return false; + } + } + return true; + }, + + "parent": function( elem ) { + return !Expr.pseudos[ "empty" ]( elem ); + }, + + // Element/input types + "header": function( elem ) { + return rheader.test( elem.nodeName ); + }, + + "input": function( elem ) { + return rinputs.test( elem.nodeName ); + }, + + "button": function( elem ) { + var name = elem.nodeName.toLowerCase(); + return name === "input" && elem.type === "button" || name === "button"; + }, + + "text": function( elem ) { + var attr; + return elem.nodeName.toLowerCase() === "input" && + elem.type === "text" && + + // Support: IE<8 + // New HTML5 attribute values (e.g., "search") appear with elem.type === "text" + ( ( attr = elem.getAttribute( "type" ) ) == null || + attr.toLowerCase() === "text" ); + }, + + // Position-in-collection + "first": createPositionalPseudo( function() { + return [ 0 ]; + } ), + + "last": createPositionalPseudo( function( _matchIndexes, length ) { + return [ length - 1 ]; + } ), + + "eq": createPositionalPseudo( function( _matchIndexes, length, argument ) { + return [ argument < 0 ? argument + length : argument ]; + } ), + + "even": createPositionalPseudo( function( matchIndexes, length ) { + var i = 0; + for ( ; i < length; i += 2 ) { + matchIndexes.push( i ); + } + return matchIndexes; + } ), + + "odd": createPositionalPseudo( function( matchIndexes, length ) { + var i = 1; + for ( ; i < length; i += 2 ) { + matchIndexes.push( i ); + } + return matchIndexes; + } ), + + "lt": createPositionalPseudo( function( matchIndexes, length, argument ) { + var i = argument < 0 ? + argument + length : + argument > length ? + length : + argument; + for ( ; --i >= 0; ) { + matchIndexes.push( i ); + } + return matchIndexes; + } ), + + "gt": createPositionalPseudo( function( matchIndexes, length, argument ) { + var i = argument < 0 ? argument + length : argument; + for ( ; ++i < length; ) { + matchIndexes.push( i ); + } + return matchIndexes; + } ) + } +}; + +Expr.pseudos[ "nth" ] = Expr.pseudos[ "eq" ]; + +// Add button/input type pseudos +for ( i in { radio: true, checkbox: true, file: true, password: true, image: true } ) { + Expr.pseudos[ i ] = createInputPseudo( i ); +} +for ( i in { submit: true, reset: true } ) { + Expr.pseudos[ i ] = createButtonPseudo( i ); +} + +// Easy API for creating new setFilters +function setFilters() {} +setFilters.prototype = Expr.filters = Expr.pseudos; +Expr.setFilters = new setFilters(); + +tokenize = Sizzle.tokenize = function( selector, parseOnly ) { + var matched, match, tokens, type, + soFar, groups, preFilters, + cached = tokenCache[ selector + " " ]; + + if ( cached ) { + return parseOnly ? 0 : cached.slice( 0 ); + } + + soFar = selector; + groups = []; + preFilters = Expr.preFilter; + + while ( soFar ) { + + // Comma and first run + if ( !matched || ( match = rcomma.exec( soFar ) ) ) { + if ( match ) { + + // Don't consume trailing commas as valid + soFar = soFar.slice( match[ 0 ].length ) || soFar; + } + groups.push( ( tokens = [] ) ); + } + + matched = false; + + // Combinators + if ( ( match = rcombinators.exec( soFar ) ) ) { + matched = match.shift(); + tokens.push( { + value: matched, + + // Cast descendant combinators to space + type: match[ 0 ].replace( rtrim, " " ) + } ); + soFar = soFar.slice( matched.length ); + } + + // Filters + for ( type in Expr.filter ) { + if ( ( match = matchExpr[ type ].exec( soFar ) ) && ( !preFilters[ type ] || + ( match = preFilters[ type ]( match ) ) ) ) { + matched = match.shift(); + tokens.push( { + value: matched, + type: type, + matches: match + } ); + soFar = soFar.slice( matched.length ); + } + } + + if ( !matched ) { + break; + } + } + + // Return the length of the invalid excess + // if we're just parsing + // Otherwise, throw an error or return tokens + return parseOnly ? + soFar.length : + soFar ? + Sizzle.error( selector ) : + + // Cache the tokens + tokenCache( selector, groups ).slice( 0 ); +}; + +function toSelector( tokens ) { + var i = 0, + len = tokens.length, + selector = ""; + for ( ; i < len; i++ ) { + selector += tokens[ i ].value; + } + return selector; +} + +function addCombinator( matcher, combinator, base ) { + var dir = combinator.dir, + skip = combinator.next, + key = skip || dir, + checkNonElements = base && key === "parentNode", + doneName = done++; + + return combinator.first ? + + // Check against closest ancestor/preceding element + function( elem, context, xml ) { + while ( ( elem = elem[ dir ] ) ) { + if ( elem.nodeType === 1 || checkNonElements ) { + return matcher( elem, context, xml ); + } + } + return false; + } : + + // Check against all ancestor/preceding elements + function( elem, context, xml ) { + var oldCache, uniqueCache, outerCache, + newCache = [ dirruns, doneName ]; + + // We can't set arbitrary data on XML nodes, so they don't benefit from combinator caching + if ( xml ) { + while ( ( elem = elem[ dir ] ) ) { + if ( elem.nodeType === 1 || checkNonElements ) { + if ( matcher( elem, context, xml ) ) { + return true; + } + } + } + } else { + while ( ( elem = elem[ dir ] ) ) { + if ( elem.nodeType === 1 || checkNonElements ) { + outerCache = elem[ expando ] || ( elem[ expando ] = {} ); + + // Support: IE <9 only + // Defend against cloned attroperties (jQuery gh-1709) + uniqueCache = outerCache[ elem.uniqueID ] || + ( outerCache[ elem.uniqueID ] = {} ); + + if ( skip && skip === elem.nodeName.toLowerCase() ) { + elem = elem[ dir ] || elem; + } else if ( ( oldCache = uniqueCache[ key ] ) && + oldCache[ 0 ] === dirruns && oldCache[ 1 ] === doneName ) { + + // Assign to newCache so results back-propagate to previous elements + return ( newCache[ 2 ] = oldCache[ 2 ] ); + } else { + + // Reuse newcache so results back-propagate to previous elements + uniqueCache[ key ] = newCache; + + // A match means we're done; a fail means we have to keep checking + if ( ( newCache[ 2 ] = matcher( elem, context, xml ) ) ) { + return true; + } + } + } + } + } + return false; + }; +} + +function elementMatcher( matchers ) { + return matchers.length > 1 ? + function( elem, context, xml ) { + var i = matchers.length; + while ( i-- ) { + if ( !matchers[ i ]( elem, context, xml ) ) { + return false; + } + } + return true; + } : + matchers[ 0 ]; +} + +function multipleContexts( selector, contexts, results ) { + var i = 0, + len = contexts.length; + for ( ; i < len; i++ ) { + Sizzle( selector, contexts[ i ], results ); + } + return results; +} + +function condense( unmatched, map, filter, context, xml ) { + var elem, + newUnmatched = [], + i = 0, + len = unmatched.length, + mapped = map != null; + + for ( ; i < len; i++ ) { + if ( ( elem = unmatched[ i ] ) ) { + if ( !filter || filter( elem, context, xml ) ) { + newUnmatched.push( elem ); + if ( mapped ) { + map.push( i ); + } + } + } + } + + return newUnmatched; +} + +function setMatcher( preFilter, selector, matcher, postFilter, postFinder, postSelector ) { + if ( postFilter && !postFilter[ expando ] ) { + postFilter = setMatcher( postFilter ); + } + if ( postFinder && !postFinder[ expando ] ) { + postFinder = setMatcher( postFinder, postSelector ); + } + return markFunction( function( seed, results, context, xml ) { + var temp, i, elem, + preMap = [], + postMap = [], + preexisting = results.length, + + // Get initial elements from seed or context + elems = seed || multipleContexts( + selector || "*", + context.nodeType ? [ context ] : context, + [] + ), + + // Prefilter to get matcher input, preserving a map for seed-results synchronization + matcherIn = preFilter && ( seed || !selector ) ? + condense( elems, preMap, preFilter, context, xml ) : + elems, + + matcherOut = matcher ? + + // If we have a postFinder, or filtered seed, or non-seed postFilter or preexisting results, + postFinder || ( seed ? preFilter : preexisting || postFilter ) ? + + // ...intermediate processing is necessary + [] : + + // ...otherwise use results directly + results : + matcherIn; + + // Find primary matches + if ( matcher ) { + matcher( matcherIn, matcherOut, context, xml ); + } + + // Apply postFilter + if ( postFilter ) { + temp = condense( matcherOut, postMap ); + postFilter( temp, [], context, xml ); + + // Un-match failing elements by moving them back to matcherIn + i = temp.length; + while ( i-- ) { + if ( ( elem = temp[ i ] ) ) { + matcherOut[ postMap[ i ] ] = !( matcherIn[ postMap[ i ] ] = elem ); + } + } + } + + if ( seed ) { + if ( postFinder || preFilter ) { + if ( postFinder ) { + + // Get the final matcherOut by condensing this intermediate into postFinder contexts + temp = []; + i = matcherOut.length; + while ( i-- ) { + if ( ( elem = matcherOut[ i ] ) ) { + + // Restore matcherIn since elem is not yet a final match + temp.push( ( matcherIn[ i ] = elem ) ); + } + } + postFinder( null, ( matcherOut = [] ), temp, xml ); + } + + // Move matched elements from seed to results to keep them synchronized + i = matcherOut.length; + while ( i-- ) { + if ( ( elem = matcherOut[ i ] ) && + ( temp = postFinder ? indexOf( seed, elem ) : preMap[ i ] ) > -1 ) { + + seed[ temp ] = !( results[ temp ] = elem ); + } + } + } + + // Add elements to results, through postFinder if defined + } else { + matcherOut = condense( + matcherOut === results ? + matcherOut.splice( preexisting, matcherOut.length ) : + matcherOut + ); + if ( postFinder ) { + postFinder( null, results, matcherOut, xml ); + } else { + push.apply( results, matcherOut ); + } + } + } ); +} + +function matcherFromTokens( tokens ) { + var checkContext, matcher, j, + len = tokens.length, + leadingRelative = Expr.relative[ tokens[ 0 ].type ], + implicitRelative = leadingRelative || Expr.relative[ " " ], + i = leadingRelative ? 1 : 0, + + // The foundational matcher ensures that elements are reachable from top-level context(s) + matchContext = addCombinator( function( elem ) { + return elem === checkContext; + }, implicitRelative, true ), + matchAnyContext = addCombinator( function( elem ) { + return indexOf( checkContext, elem ) > -1; + }, implicitRelative, true ), + matchers = [ function( elem, context, xml ) { + var ret = ( !leadingRelative && ( xml || context !== outermostContext ) ) || ( + ( checkContext = context ).nodeType ? + matchContext( elem, context, xml ) : + matchAnyContext( elem, context, xml ) ); + + // Avoid hanging onto element (issue #299) + checkContext = null; + return ret; + } ]; + + for ( ; i < len; i++ ) { + if ( ( matcher = Expr.relative[ tokens[ i ].type ] ) ) { + matchers = [ addCombinator( elementMatcher( matchers ), matcher ) ]; + } else { + matcher = Expr.filter[ tokens[ i ].type ].apply( null, tokens[ i ].matches ); + + // Return special upon seeing a positional matcher + if ( matcher[ expando ] ) { + + // Find the next relative operator (if any) for proper handling + j = ++i; + for ( ; j < len; j++ ) { + if ( Expr.relative[ tokens[ j ].type ] ) { + break; + } + } + return setMatcher( + i > 1 && elementMatcher( matchers ), + i > 1 && toSelector( + + // If the preceding token was a descendant combinator, insert an implicit any-element `*` + tokens + .slice( 0, i - 1 ) + .concat( { value: tokens[ i - 2 ].type === " " ? "*" : "" } ) + ).replace( rtrim, "$1" ), + matcher, + i < j && matcherFromTokens( tokens.slice( i, j ) ), + j < len && matcherFromTokens( ( tokens = tokens.slice( j ) ) ), + j < len && toSelector( tokens ) + ); + } + matchers.push( matcher ); + } + } + + return elementMatcher( matchers ); +} + +function matcherFromGroupMatchers( elementMatchers, setMatchers ) { + var bySet = setMatchers.length > 0, + byElement = elementMatchers.length > 0, + superMatcher = function( seed, context, xml, results, outermost ) { + var elem, j, matcher, + matchedCount = 0, + i = "0", + unmatched = seed && [], + setMatched = [], + contextBackup = outermostContext, + + // We must always have either seed elements or outermost context + elems = seed || byElement && Expr.find[ "TAG" ]( "*", outermost ), + + // Use integer dirruns iff this is the outermost matcher + dirrunsUnique = ( dirruns += contextBackup == null ? 1 : Math.random() || 0.1 ), + len = elems.length; + + if ( outermost ) { + + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + outermostContext = context == document || context || outermost; + } + + // Add elements passing elementMatchers directly to results + // Support: IE<9, Safari + // Tolerate NodeList properties (IE: "length"; Safari: ) matching elements by id + for ( ; i !== len && ( elem = elems[ i ] ) != null; i++ ) { + if ( byElement && elem ) { + j = 0; + + // Support: IE 11+, Edge 17 - 18+ + // IE/Edge sometimes throw a "Permission denied" error when strict-comparing + // two documents; shallow comparisons work. + // eslint-disable-next-line eqeqeq + if ( !context && elem.ownerDocument != document ) { + setDocument( elem ); + xml = !documentIsHTML; + } + while ( ( matcher = elementMatchers[ j++ ] ) ) { + if ( matcher( elem, context || document, xml ) ) { + results.push( elem ); + break; + } + } + if ( outermost ) { + dirruns = dirrunsUnique; + } + } + + // Track unmatched elements for set filters + if ( bySet ) { + + // They will have gone through all possible matchers + if ( ( elem = !matcher && elem ) ) { + matchedCount--; + } + + // Lengthen the array for every element, matched or not + if ( seed ) { + unmatched.push( elem ); + } + } + } + + // `i` is now the count of elements visited above, and adding it to `matchedCount` + // makes the latter nonnegative. + matchedCount += i; + + // Apply set filters to unmatched elements + // NOTE: This can be skipped if there are no unmatched elements (i.e., `matchedCount` + // equals `i`), unless we didn't visit _any_ elements in the above loop because we have + // no element matchers and no seed. + // Incrementing an initially-string "0" `i` allows `i` to remain a string only in that + // case, which will result in a "00" `matchedCount` that differs from `i` but is also + // numerically zero. + if ( bySet && i !== matchedCount ) { + j = 0; + while ( ( matcher = setMatchers[ j++ ] ) ) { + matcher( unmatched, setMatched, context, xml ); + } + + if ( seed ) { + + // Reintegrate element matches to eliminate the need for sorting + if ( matchedCount > 0 ) { + while ( i-- ) { + if ( !( unmatched[ i ] || setMatched[ i ] ) ) { + setMatched[ i ] = pop.call( results ); + } + } + } + + // Discard index placeholder values to get only actual matches + setMatched = condense( setMatched ); + } + + // Add matches to results + push.apply( results, setMatched ); + + // Seedless set matches succeeding multiple successful matchers stipulate sorting + if ( outermost && !seed && setMatched.length > 0 && + ( matchedCount + setMatchers.length ) > 1 ) { + + Sizzle.uniqueSort( results ); + } + } + + // Override manipulation of globals by nested matchers + if ( outermost ) { + dirruns = dirrunsUnique; + outermostContext = contextBackup; + } + + return unmatched; + }; + + return bySet ? + markFunction( superMatcher ) : + superMatcher; +} + +compile = Sizzle.compile = function( selector, match /* Internal Use Only */ ) { + var i, + setMatchers = [], + elementMatchers = [], + cached = compilerCache[ selector + " " ]; + + if ( !cached ) { + + // Generate a function of recursive functions that can be used to check each element + if ( !match ) { + match = tokenize( selector ); + } + i = match.length; + while ( i-- ) { + cached = matcherFromTokens( match[ i ] ); + if ( cached[ expando ] ) { + setMatchers.push( cached ); + } else { + elementMatchers.push( cached ); + } + } + + // Cache the compiled function + cached = compilerCache( + selector, + matcherFromGroupMatchers( elementMatchers, setMatchers ) + ); + + // Save selector and tokenization + cached.selector = selector; + } + return cached; +}; + +/** + * A low-level selection function that works with Sizzle's compiled + * selector functions + * @param {String|Function} selector A selector or a pre-compiled + * selector function built with Sizzle.compile + * @param {Element} context + * @param {Array} [results] + * @param {Array} [seed] A set of elements to match against + */ +select = Sizzle.select = function( selector, context, results, seed ) { + var i, tokens, token, type, find, + compiled = typeof selector === "function" && selector, + match = !seed && tokenize( ( selector = compiled.selector || selector ) ); + + results = results || []; + + // Try to minimize operations if there is only one selector in the list and no seed + // (the latter of which guarantees us context) + if ( match.length === 1 ) { + + // Reduce context if the leading compound selector is an ID + tokens = match[ 0 ] = match[ 0 ].slice( 0 ); + if ( tokens.length > 2 && ( token = tokens[ 0 ] ).type === "ID" && + context.nodeType === 9 && documentIsHTML && Expr.relative[ tokens[ 1 ].type ] ) { + + context = ( Expr.find[ "ID" ]( token.matches[ 0 ] + .replace( runescape, funescape ), context ) || [] )[ 0 ]; + if ( !context ) { + return results; + + // Precompiled matchers will still verify ancestry, so step up a level + } else if ( compiled ) { + context = context.parentNode; + } + + selector = selector.slice( tokens.shift().value.length ); + } + + // Fetch a seed set for right-to-left matching + i = matchExpr[ "needsContext" ].test( selector ) ? 0 : tokens.length; + while ( i-- ) { + token = tokens[ i ]; + + // Abort if we hit a combinator + if ( Expr.relative[ ( type = token.type ) ] ) { + break; + } + if ( ( find = Expr.find[ type ] ) ) { + + // Search, expanding context for leading sibling combinators + if ( ( seed = find( + token.matches[ 0 ].replace( runescape, funescape ), + rsibling.test( tokens[ 0 ].type ) && testContext( context.parentNode ) || + context + ) ) ) { + + // If seed is empty or no tokens remain, we can return early + tokens.splice( i, 1 ); + selector = seed.length && toSelector( tokens ); + if ( !selector ) { + push.apply( results, seed ); + return results; + } + + break; + } + } + } + } + + // Compile and execute a filtering function if one is not provided + // Provide `match` to avoid retokenization if we modified the selector above + ( compiled || compile( selector, match ) )( + seed, + context, + !documentIsHTML, + results, + !context || rsibling.test( selector ) && testContext( context.parentNode ) || context + ); + return results; +}; + +// One-time assignments + +// Sort stability +support.sortStable = expando.split( "" ).sort( sortOrder ).join( "" ) === expando; + +// Support: Chrome 14-35+ +// Always assume duplicates if they aren't passed to the comparison function +support.detectDuplicates = !!hasDuplicate; + +// Initialize against the default document +setDocument(); + +// Support: Webkit<537.32 - Safari 6.0.3/Chrome 25 (fixed in Chrome 27) +// Detached nodes confoundingly follow *each other* +support.sortDetached = assert( function( el ) { + + // Should return 1, but returns 4 (following) + return el.compareDocumentPosition( document.createElement( "fieldset" ) ) & 1; +} ); + +// Support: IE<8 +// Prevent attribute/property "interpolation" +// https://msdn.microsoft.com/en-us/library/ms536429%28VS.85%29.aspx +if ( !assert( function( el ) { + el.innerHTML = ""; + return el.firstChild.getAttribute( "href" ) === "#"; +} ) ) { + addHandle( "type|href|height|width", function( elem, name, isXML ) { + if ( !isXML ) { + return elem.getAttribute( name, name.toLowerCase() === "type" ? 1 : 2 ); + } + } ); +} + +// Support: IE<9 +// Use defaultValue in place of getAttribute("value") +if ( !support.attributes || !assert( function( el ) { + el.innerHTML = ""; + el.firstChild.setAttribute( "value", "" ); + return el.firstChild.getAttribute( "value" ) === ""; +} ) ) { + addHandle( "value", function( elem, _name, isXML ) { + if ( !isXML && elem.nodeName.toLowerCase() === "input" ) { + return elem.defaultValue; + } + } ); +} + +// Support: IE<9 +// Use getAttributeNode to fetch booleans when getAttribute lies +if ( !assert( function( el ) { + return el.getAttribute( "disabled" ) == null; +} ) ) { + addHandle( booleans, function( elem, name, isXML ) { + var val; + if ( !isXML ) { + return elem[ name ] === true ? name.toLowerCase() : + ( val = elem.getAttributeNode( name ) ) && val.specified ? + val.value : + null; + } + } ); +} + +return Sizzle; + +} )( window ); + + + +jQuery.find = Sizzle; +jQuery.expr = Sizzle.selectors; + +// Deprecated +jQuery.expr[ ":" ] = jQuery.expr.pseudos; +jQuery.uniqueSort = jQuery.unique = Sizzle.uniqueSort; +jQuery.text = Sizzle.getText; +jQuery.isXMLDoc = Sizzle.isXML; +jQuery.contains = Sizzle.contains; +jQuery.escapeSelector = Sizzle.escape; + + + + +var dir = function( elem, dir, until ) { + var matched = [], + truncate = until !== undefined; + + while ( ( elem = elem[ dir ] ) && elem.nodeType !== 9 ) { + if ( elem.nodeType === 1 ) { + if ( truncate && jQuery( elem ).is( until ) ) { + break; + } + matched.push( elem ); + } + } + return matched; +}; + + +var siblings = function( n, elem ) { + var matched = []; + + for ( ; n; n = n.nextSibling ) { + if ( n.nodeType === 1 && n !== elem ) { + matched.push( n ); + } + } + + return matched; +}; + + +var rneedsContext = jQuery.expr.match.needsContext; + + + +function nodeName( elem, name ) { + + return elem.nodeName && elem.nodeName.toLowerCase() === name.toLowerCase(); + +} +var rsingleTag = ( /^<([a-z][^\/\0>:\x20\t\r\n\f]*)[\x20\t\r\n\f]*\/?>(?:<\/\1>|)$/i ); + + + +// Implement the identical functionality for filter and not +function winnow( elements, qualifier, not ) { + if ( isFunction( qualifier ) ) { + return jQuery.grep( elements, function( elem, i ) { + return !!qualifier.call( elem, i, elem ) !== not; + } ); + } + + // Single element + if ( qualifier.nodeType ) { + return jQuery.grep( elements, function( elem ) { + return ( elem === qualifier ) !== not; + } ); + } + + // Arraylike of elements (jQuery, arguments, Array) + if ( typeof qualifier !== "string" ) { + return jQuery.grep( elements, function( elem ) { + return ( indexOf.call( qualifier, elem ) > -1 ) !== not; + } ); + } + + // Filtered directly for both simple and complex selectors + return jQuery.filter( qualifier, elements, not ); +} + +jQuery.filter = function( expr, elems, not ) { + var elem = elems[ 0 ]; + + if ( not ) { + expr = ":not(" + expr + ")"; + } + + if ( elems.length === 1 && elem.nodeType === 1 ) { + return jQuery.find.matchesSelector( elem, expr ) ? [ elem ] : []; + } + + return jQuery.find.matches( expr, jQuery.grep( elems, function( elem ) { + return elem.nodeType === 1; + } ) ); +}; + +jQuery.fn.extend( { + find: function( selector ) { + var i, ret, + len = this.length, + self = this; + + if ( typeof selector !== "string" ) { + return this.pushStack( jQuery( selector ).filter( function() { + for ( i = 0; i < len; i++ ) { + if ( jQuery.contains( self[ i ], this ) ) { + return true; + } + } + } ) ); + } + + ret = this.pushStack( [] ); + + for ( i = 0; i < len; i++ ) { + jQuery.find( selector, self[ i ], ret ); + } + + return len > 1 ? jQuery.uniqueSort( ret ) : ret; + }, + filter: function( selector ) { + return this.pushStack( winnow( this, selector || [], false ) ); + }, + not: function( selector ) { + return this.pushStack( winnow( this, selector || [], true ) ); + }, + is: function( selector ) { + return !!winnow( + this, + + // If this is a positional/relative selector, check membership in the returned set + // so $("p:first").is("p:last") won't return true for a doc with two "p". + typeof selector === "string" && rneedsContext.test( selector ) ? + jQuery( selector ) : + selector || [], + false + ).length; + } +} ); + + +// Initialize a jQuery object + + +// A central reference to the root jQuery(document) +var rootjQuery, + + // A simple way to check for HTML strings + // Prioritize #id over to avoid XSS via location.hash (#9521) + // Strict HTML recognition (#11290: must start with <) + // Shortcut simple #id case for speed + rquickExpr = /^(?:\s*(<[\w\W]+>)[^>]*|#([\w-]+))$/, + + init = jQuery.fn.init = function( selector, context, root ) { + var match, elem; + + // HANDLE: $(""), $(null), $(undefined), $(false) + if ( !selector ) { + return this; + } + + // Method init() accepts an alternate rootjQuery + // so migrate can support jQuery.sub (gh-2101) + root = root || rootjQuery; + + // Handle HTML strings + if ( typeof selector === "string" ) { + if ( selector[ 0 ] === "<" && + selector[ selector.length - 1 ] === ">" && + selector.length >= 3 ) { + + // Assume that strings that start and end with <> are HTML and skip the regex check + match = [ null, selector, null ]; + + } else { + match = rquickExpr.exec( selector ); + } + + // Match html or make sure no context is specified for #id + if ( match && ( match[ 1 ] || !context ) ) { + + // HANDLE: $(html) -> $(array) + if ( match[ 1 ] ) { + context = context instanceof jQuery ? context[ 0 ] : context; + + // Option to run scripts is true for back-compat + // Intentionally let the error be thrown if parseHTML is not present + jQuery.merge( this, jQuery.parseHTML( + match[ 1 ], + context && context.nodeType ? context.ownerDocument || context : document, + true + ) ); + + // HANDLE: $(html, props) + if ( rsingleTag.test( match[ 1 ] ) && jQuery.isPlainObject( context ) ) { + for ( match in context ) { + + // Properties of context are called as methods if possible + if ( isFunction( this[ match ] ) ) { + this[ match ]( context[ match ] ); + + // ...and otherwise set as attributes + } else { + this.attr( match, context[ match ] ); + } + } + } + + return this; + + // HANDLE: $(#id) + } else { + elem = document.getElementById( match[ 2 ] ); + + if ( elem ) { + + // Inject the element directly into the jQuery object + this[ 0 ] = elem; + this.length = 1; + } + return this; + } + + // HANDLE: $(expr, $(...)) + } else if ( !context || context.jquery ) { + return ( context || root ).find( selector ); + + // HANDLE: $(expr, context) + // (which is just equivalent to: $(context).find(expr) + } else { + return this.constructor( context ).find( selector ); + } + + // HANDLE: $(DOMElement) + } else if ( selector.nodeType ) { + this[ 0 ] = selector; + this.length = 1; + return this; + + // HANDLE: $(function) + // Shortcut for document ready + } else if ( isFunction( selector ) ) { + return root.ready !== undefined ? + root.ready( selector ) : + + // Execute immediately if ready is not present + selector( jQuery ); + } + + return jQuery.makeArray( selector, this ); + }; + +// Give the init function the jQuery prototype for later instantiation +init.prototype = jQuery.fn; + +// Initialize central reference +rootjQuery = jQuery( document ); + + +var rparentsprev = /^(?:parents|prev(?:Until|All))/, + + // Methods guaranteed to produce a unique set when starting from a unique set + guaranteedUnique = { + children: true, + contents: true, + next: true, + prev: true + }; + +jQuery.fn.extend( { + has: function( target ) { + var targets = jQuery( target, this ), + l = targets.length; + + return this.filter( function() { + var i = 0; + for ( ; i < l; i++ ) { + if ( jQuery.contains( this, targets[ i ] ) ) { + return true; + } + } + } ); + }, + + closest: function( selectors, context ) { + var cur, + i = 0, + l = this.length, + matched = [], + targets = typeof selectors !== "string" && jQuery( selectors ); + + // Positional selectors never match, since there's no _selection_ context + if ( !rneedsContext.test( selectors ) ) { + for ( ; i < l; i++ ) { + for ( cur = this[ i ]; cur && cur !== context; cur = cur.parentNode ) { + + // Always skip document fragments + if ( cur.nodeType < 11 && ( targets ? + targets.index( cur ) > -1 : + + // Don't pass non-elements to Sizzle + cur.nodeType === 1 && + jQuery.find.matchesSelector( cur, selectors ) ) ) { + + matched.push( cur ); + break; + } + } + } + } + + return this.pushStack( matched.length > 1 ? jQuery.uniqueSort( matched ) : matched ); + }, + + // Determine the position of an element within the set + index: function( elem ) { + + // No argument, return index in parent + if ( !elem ) { + return ( this[ 0 ] && this[ 0 ].parentNode ) ? this.first().prevAll().length : -1; + } + + // Index in selector + if ( typeof elem === "string" ) { + return indexOf.call( jQuery( elem ), this[ 0 ] ); + } + + // Locate the position of the desired element + return indexOf.call( this, + + // If it receives a jQuery object, the first element is used + elem.jquery ? elem[ 0 ] : elem + ); + }, + + add: function( selector, context ) { + return this.pushStack( + jQuery.uniqueSort( + jQuery.merge( this.get(), jQuery( selector, context ) ) + ) + ); + }, + + addBack: function( selector ) { + return this.add( selector == null ? + this.prevObject : this.prevObject.filter( selector ) + ); + } +} ); + +function sibling( cur, dir ) { + while ( ( cur = cur[ dir ] ) && cur.nodeType !== 1 ) {} + return cur; +} + +jQuery.each( { + parent: function( elem ) { + var parent = elem.parentNode; + return parent && parent.nodeType !== 11 ? parent : null; + }, + parents: function( elem ) { + return dir( elem, "parentNode" ); + }, + parentsUntil: function( elem, _i, until ) { + return dir( elem, "parentNode", until ); + }, + next: function( elem ) { + return sibling( elem, "nextSibling" ); + }, + prev: function( elem ) { + return sibling( elem, "previousSibling" ); + }, + nextAll: function( elem ) { + return dir( elem, "nextSibling" ); + }, + prevAll: function( elem ) { + return dir( elem, "previousSibling" ); + }, + nextUntil: function( elem, _i, until ) { + return dir( elem, "nextSibling", until ); + }, + prevUntil: function( elem, _i, until ) { + return dir( elem, "previousSibling", until ); + }, + siblings: function( elem ) { + return siblings( ( elem.parentNode || {} ).firstChild, elem ); + }, + children: function( elem ) { + return siblings( elem.firstChild ); + }, + contents: function( elem ) { + if ( elem.contentDocument != null && + + // Support: IE 11+ + // elements with no `data` attribute has an object + // `contentDocument` with a `null` prototype. + getProto( elem.contentDocument ) ) { + + return elem.contentDocument; + } + + // Support: IE 9 - 11 only, iOS 7 only, Android Browser <=4.3 only + // Treat the template element as a regular one in browsers that + // don't support it. + if ( nodeName( elem, "template" ) ) { + elem = elem.content || elem; + } + + return jQuery.merge( [], elem.childNodes ); + } +}, function( name, fn ) { + jQuery.fn[ name ] = function( until, selector ) { + var matched = jQuery.map( this, fn, until ); + + if ( name.slice( -5 ) !== "Until" ) { + selector = until; + } + + if ( selector && typeof selector === "string" ) { + matched = jQuery.filter( selector, matched ); + } + + if ( this.length > 1 ) { + + // Remove duplicates + if ( !guaranteedUnique[ name ] ) { + jQuery.uniqueSort( matched ); + } + + // Reverse order for parents* and prev-derivatives + if ( rparentsprev.test( name ) ) { + matched.reverse(); + } + } + + return this.pushStack( matched ); + }; +} ); +var rnothtmlwhite = ( /[^\x20\t\r\n\f]+/g ); + + + +// Convert String-formatted options into Object-formatted ones +function createOptions( options ) { + var object = {}; + jQuery.each( options.match( rnothtmlwhite ) || [], function( _, flag ) { + object[ flag ] = true; + } ); + return object; +} + +/* + * Create a callback list using the following parameters: + * + * options: an optional list of space-separated options that will change how + * the callback list behaves or a more traditional option object + * + * By default a callback list will act like an event callback list and can be + * "fired" multiple times. + * + * Possible options: + * + * once: will ensure the callback list can only be fired once (like a Deferred) + * + * memory: will keep track of previous values and will call any callback added + * after the list has been fired right away with the latest "memorized" + * values (like a Deferred) + * + * unique: will ensure a callback can only be added once (no duplicate in the list) + * + * stopOnFalse: interrupt callings when a callback returns false + * + */ +jQuery.Callbacks = function( options ) { + + // Convert options from String-formatted to Object-formatted if needed + // (we check in cache first) + options = typeof options === "string" ? + createOptions( options ) : + jQuery.extend( {}, options ); + + var // Flag to know if list is currently firing + firing, + + // Last fire value for non-forgettable lists + memory, + + // Flag to know if list was already fired + fired, + + // Flag to prevent firing + locked, + + // Actual callback list + list = [], + + // Queue of execution data for repeatable lists + queue = [], + + // Index of currently firing callback (modified by add/remove as needed) + firingIndex = -1, + + // Fire callbacks + fire = function() { + + // Enforce single-firing + locked = locked || options.once; + + // Execute callbacks for all pending executions, + // respecting firingIndex overrides and runtime changes + fired = firing = true; + for ( ; queue.length; firingIndex = -1 ) { + memory = queue.shift(); + while ( ++firingIndex < list.length ) { + + // Run callback and check for early termination + if ( list[ firingIndex ].apply( memory[ 0 ], memory[ 1 ] ) === false && + options.stopOnFalse ) { + + // Jump to end and forget the data so .add doesn't re-fire + firingIndex = list.length; + memory = false; + } + } + } + + // Forget the data if we're done with it + if ( !options.memory ) { + memory = false; + } + + firing = false; + + // Clean up if we're done firing for good + if ( locked ) { + + // Keep an empty list if we have data for future add calls + if ( memory ) { + list = []; + + // Otherwise, this object is spent + } else { + list = ""; + } + } + }, + + // Actual Callbacks object + self = { + + // Add a callback or a collection of callbacks to the list + add: function() { + if ( list ) { + + // If we have memory from a past run, we should fire after adding + if ( memory && !firing ) { + firingIndex = list.length - 1; + queue.push( memory ); + } + + ( function add( args ) { + jQuery.each( args, function( _, arg ) { + if ( isFunction( arg ) ) { + if ( !options.unique || !self.has( arg ) ) { + list.push( arg ); + } + } else if ( arg && arg.length && toType( arg ) !== "string" ) { + + // Inspect recursively + add( arg ); + } + } ); + } )( arguments ); + + if ( memory && !firing ) { + fire(); + } + } + return this; + }, + + // Remove a callback from the list + remove: function() { + jQuery.each( arguments, function( _, arg ) { + var index; + while ( ( index = jQuery.inArray( arg, list, index ) ) > -1 ) { + list.splice( index, 1 ); + + // Handle firing indexes + if ( index <= firingIndex ) { + firingIndex--; + } + } + } ); + return this; + }, + + // Check if a given callback is in the list. + // If no argument is given, return whether or not list has callbacks attached. + has: function( fn ) { + return fn ? + jQuery.inArray( fn, list ) > -1 : + list.length > 0; + }, + + // Remove all callbacks from the list + empty: function() { + if ( list ) { + list = []; + } + return this; + }, + + // Disable .fire and .add + // Abort any current/pending executions + // Clear all callbacks and values + disable: function() { + locked = queue = []; + list = memory = ""; + return this; + }, + disabled: function() { + return !list; + }, + + // Disable .fire + // Also disable .add unless we have memory (since it would have no effect) + // Abort any pending executions + lock: function() { + locked = queue = []; + if ( !memory && !firing ) { + list = memory = ""; + } + return this; + }, + locked: function() { + return !!locked; + }, + + // Call all callbacks with the given context and arguments + fireWith: function( context, args ) { + if ( !locked ) { + args = args || []; + args = [ context, args.slice ? args.slice() : args ]; + queue.push( args ); + if ( !firing ) { + fire(); + } + } + return this; + }, + + // Call all the callbacks with the given arguments + fire: function() { + self.fireWith( this, arguments ); + return this; + }, + + // To know if the callbacks have already been called at least once + fired: function() { + return !!fired; + } + }; + + return self; +}; + + +function Identity( v ) { + return v; +} +function Thrower( ex ) { + throw ex; +} + +function adoptValue( value, resolve, reject, noValue ) { + var method; + + try { + + // Check for promise aspect first to privilege synchronous behavior + if ( value && isFunction( ( method = value.promise ) ) ) { + method.call( value ).done( resolve ).fail( reject ); + + // Other thenables + } else if ( value && isFunction( ( method = value.then ) ) ) { + method.call( value, resolve, reject ); + + // Other non-thenables + } else { + + // Control `resolve` arguments by letting Array#slice cast boolean `noValue` to integer: + // * false: [ value ].slice( 0 ) => resolve( value ) + // * true: [ value ].slice( 1 ) => resolve() + resolve.apply( undefined, [ value ].slice( noValue ) ); + } + + // For Promises/A+, convert exceptions into rejections + // Since jQuery.when doesn't unwrap thenables, we can skip the extra checks appearing in + // Deferred#then to conditionally suppress rejection. + } catch ( value ) { + + // Support: Android 4.0 only + // Strict mode functions invoked without .call/.apply get global-object context + reject.apply( undefined, [ value ] ); + } +} + +jQuery.extend( { + + Deferred: function( func ) { + var tuples = [ + + // action, add listener, callbacks, + // ... .then handlers, argument index, [final state] + [ "notify", "progress", jQuery.Callbacks( "memory" ), + jQuery.Callbacks( "memory" ), 2 ], + [ "resolve", "done", jQuery.Callbacks( "once memory" ), + jQuery.Callbacks( "once memory" ), 0, "resolved" ], + [ "reject", "fail", jQuery.Callbacks( "once memory" ), + jQuery.Callbacks( "once memory" ), 1, "rejected" ] + ], + state = "pending", + promise = { + state: function() { + return state; + }, + always: function() { + deferred.done( arguments ).fail( arguments ); + return this; + }, + "catch": function( fn ) { + return promise.then( null, fn ); + }, + + // Keep pipe for back-compat + pipe: function( /* fnDone, fnFail, fnProgress */ ) { + var fns = arguments; + + return jQuery.Deferred( function( newDefer ) { + jQuery.each( tuples, function( _i, tuple ) { + + // Map tuples (progress, done, fail) to arguments (done, fail, progress) + var fn = isFunction( fns[ tuple[ 4 ] ] ) && fns[ tuple[ 4 ] ]; + + // deferred.progress(function() { bind to newDefer or newDefer.notify }) + // deferred.done(function() { bind to newDefer or newDefer.resolve }) + // deferred.fail(function() { bind to newDefer or newDefer.reject }) + deferred[ tuple[ 1 ] ]( function() { + var returned = fn && fn.apply( this, arguments ); + if ( returned && isFunction( returned.promise ) ) { + returned.promise() + .progress( newDefer.notify ) + .done( newDefer.resolve ) + .fail( newDefer.reject ); + } else { + newDefer[ tuple[ 0 ] + "With" ]( + this, + fn ? [ returned ] : arguments + ); + } + } ); + } ); + fns = null; + } ).promise(); + }, + then: function( onFulfilled, onRejected, onProgress ) { + var maxDepth = 0; + function resolve( depth, deferred, handler, special ) { + return function() { + var that = this, + args = arguments, + mightThrow = function() { + var returned, then; + + // Support: Promises/A+ section 2.3.3.3.3 + // https://promisesaplus.com/#point-59 + // Ignore double-resolution attempts + if ( depth < maxDepth ) { + return; + } + + returned = handler.apply( that, args ); + + // Support: Promises/A+ section 2.3.1 + // https://promisesaplus.com/#point-48 + if ( returned === deferred.promise() ) { + throw new TypeError( "Thenable self-resolution" ); + } + + // Support: Promises/A+ sections 2.3.3.1, 3.5 + // https://promisesaplus.com/#point-54 + // https://promisesaplus.com/#point-75 + // Retrieve `then` only once + then = returned && + + // Support: Promises/A+ section 2.3.4 + // https://promisesaplus.com/#point-64 + // Only check objects and functions for thenability + ( typeof returned === "object" || + typeof returned === "function" ) && + returned.then; + + // Handle a returned thenable + if ( isFunction( then ) ) { + + // Special processors (notify) just wait for resolution + if ( special ) { + then.call( + returned, + resolve( maxDepth, deferred, Identity, special ), + resolve( maxDepth, deferred, Thrower, special ) + ); + + // Normal processors (resolve) also hook into progress + } else { + + // ...and disregard older resolution values + maxDepth++; + + then.call( + returned, + resolve( maxDepth, deferred, Identity, special ), + resolve( maxDepth, deferred, Thrower, special ), + resolve( maxDepth, deferred, Identity, + deferred.notifyWith ) + ); + } + + // Handle all other returned values + } else { + + // Only substitute handlers pass on context + // and multiple values (non-spec behavior) + if ( handler !== Identity ) { + that = undefined; + args = [ returned ]; + } + + // Process the value(s) + // Default process is resolve + ( special || deferred.resolveWith )( that, args ); + } + }, + + // Only normal processors (resolve) catch and reject exceptions + process = special ? + mightThrow : + function() { + try { + mightThrow(); + } catch ( e ) { + + if ( jQuery.Deferred.exceptionHook ) { + jQuery.Deferred.exceptionHook( e, + process.stackTrace ); + } + + // Support: Promises/A+ section 2.3.3.3.4.1 + // https://promisesaplus.com/#point-61 + // Ignore post-resolution exceptions + if ( depth + 1 >= maxDepth ) { + + // Only substitute handlers pass on context + // and multiple values (non-spec behavior) + if ( handler !== Thrower ) { + that = undefined; + args = [ e ]; + } + + deferred.rejectWith( that, args ); + } + } + }; + + // Support: Promises/A+ section 2.3.3.3.1 + // https://promisesaplus.com/#point-57 + // Re-resolve promises immediately to dodge false rejection from + // subsequent errors + if ( depth ) { + process(); + } else { + + // Call an optional hook to record the stack, in case of exception + // since it's otherwise lost when execution goes async + if ( jQuery.Deferred.getStackHook ) { + process.stackTrace = jQuery.Deferred.getStackHook(); + } + window.setTimeout( process ); + } + }; + } + + return jQuery.Deferred( function( newDefer ) { + + // progress_handlers.add( ... ) + tuples[ 0 ][ 3 ].add( + resolve( + 0, + newDefer, + isFunction( onProgress ) ? + onProgress : + Identity, + newDefer.notifyWith + ) + ); + + // fulfilled_handlers.add( ... ) + tuples[ 1 ][ 3 ].add( + resolve( + 0, + newDefer, + isFunction( onFulfilled ) ? + onFulfilled : + Identity + ) + ); + + // rejected_handlers.add( ... ) + tuples[ 2 ][ 3 ].add( + resolve( + 0, + newDefer, + isFunction( onRejected ) ? + onRejected : + Thrower + ) + ); + } ).promise(); + }, + + // Get a promise for this deferred + // If obj is provided, the promise aspect is added to the object + promise: function( obj ) { + return obj != null ? jQuery.extend( obj, promise ) : promise; + } + }, + deferred = {}; + + // Add list-specific methods + jQuery.each( tuples, function( i, tuple ) { + var list = tuple[ 2 ], + stateString = tuple[ 5 ]; + + // promise.progress = list.add + // promise.done = list.add + // promise.fail = list.add + promise[ tuple[ 1 ] ] = list.add; + + // Handle state + if ( stateString ) { + list.add( + function() { + + // state = "resolved" (i.e., fulfilled) + // state = "rejected" + state = stateString; + }, + + // rejected_callbacks.disable + // fulfilled_callbacks.disable + tuples[ 3 - i ][ 2 ].disable, + + // rejected_handlers.disable + // fulfilled_handlers.disable + tuples[ 3 - i ][ 3 ].disable, + + // progress_callbacks.lock + tuples[ 0 ][ 2 ].lock, + + // progress_handlers.lock + tuples[ 0 ][ 3 ].lock + ); + } + + // progress_handlers.fire + // fulfilled_handlers.fire + // rejected_handlers.fire + list.add( tuple[ 3 ].fire ); + + // deferred.notify = function() { deferred.notifyWith(...) } + // deferred.resolve = function() { deferred.resolveWith(...) } + // deferred.reject = function() { deferred.rejectWith(...) } + deferred[ tuple[ 0 ] ] = function() { + deferred[ tuple[ 0 ] + "With" ]( this === deferred ? undefined : this, arguments ); + return this; + }; + + // deferred.notifyWith = list.fireWith + // deferred.resolveWith = list.fireWith + // deferred.rejectWith = list.fireWith + deferred[ tuple[ 0 ] + "With" ] = list.fireWith; + } ); + + // Make the deferred a promise + promise.promise( deferred ); + + // Call given func if any + if ( func ) { + func.call( deferred, deferred ); + } + + // All done! + return deferred; + }, + + // Deferred helper + when: function( singleValue ) { + var + + // count of uncompleted subordinates + remaining = arguments.length, + + // count of unprocessed arguments + i = remaining, + + // subordinate fulfillment data + resolveContexts = Array( i ), + resolveValues = slice.call( arguments ), + + // the primary Deferred + primary = jQuery.Deferred(), + + // subordinate callback factory + updateFunc = function( i ) { + return function( value ) { + resolveContexts[ i ] = this; + resolveValues[ i ] = arguments.length > 1 ? slice.call( arguments ) : value; + if ( !( --remaining ) ) { + primary.resolveWith( resolveContexts, resolveValues ); + } + }; + }; + + // Single- and empty arguments are adopted like Promise.resolve + if ( remaining <= 1 ) { + adoptValue( singleValue, primary.done( updateFunc( i ) ).resolve, primary.reject, + !remaining ); + + // Use .then() to unwrap secondary thenables (cf. gh-3000) + if ( primary.state() === "pending" || + isFunction( resolveValues[ i ] && resolveValues[ i ].then ) ) { + + return primary.then(); + } + } + + // Multiple arguments are aggregated like Promise.all array elements + while ( i-- ) { + adoptValue( resolveValues[ i ], updateFunc( i ), primary.reject ); + } + + return primary.promise(); + } +} ); + + +// These usually indicate a programmer mistake during development, +// warn about them ASAP rather than swallowing them by default. +var rerrorNames = /^(Eval|Internal|Range|Reference|Syntax|Type|URI)Error$/; + +jQuery.Deferred.exceptionHook = function( error, stack ) { + + // Support: IE 8 - 9 only + // Console exists when dev tools are open, which can happen at any time + if ( window.console && window.console.warn && error && rerrorNames.test( error.name ) ) { + window.console.warn( "jQuery.Deferred exception: " + error.message, error.stack, stack ); + } +}; + + + + +jQuery.readyException = function( error ) { + window.setTimeout( function() { + throw error; + } ); +}; + + + + +// The deferred used on DOM ready +var readyList = jQuery.Deferred(); + +jQuery.fn.ready = function( fn ) { + + readyList + .then( fn ) + + // Wrap jQuery.readyException in a function so that the lookup + // happens at the time of error handling instead of callback + // registration. + .catch( function( error ) { + jQuery.readyException( error ); + } ); + + return this; +}; + +jQuery.extend( { + + // Is the DOM ready to be used? Set to true once it occurs. + isReady: false, + + // A counter to track how many items to wait for before + // the ready event fires. See #6781 + readyWait: 1, + + // Handle when the DOM is ready + ready: function( wait ) { + + // Abort if there are pending holds or we're already ready + if ( wait === true ? --jQuery.readyWait : jQuery.isReady ) { + return; + } + + // Remember that the DOM is ready + jQuery.isReady = true; + + // If a normal DOM Ready event fired, decrement, and wait if need be + if ( wait !== true && --jQuery.readyWait > 0 ) { + return; + } + + // If there are functions bound, to execute + readyList.resolveWith( document, [ jQuery ] ); + } +} ); + +jQuery.ready.then = readyList.then; + +// The ready event handler and self cleanup method +function completed() { + document.removeEventListener( "DOMContentLoaded", completed ); + window.removeEventListener( "load", completed ); + jQuery.ready(); +} + +// Catch cases where $(document).ready() is called +// after the browser event has already occurred. +// Support: IE <=9 - 10 only +// Older IE sometimes signals "interactive" too soon +if ( document.readyState === "complete" || + ( document.readyState !== "loading" && !document.documentElement.doScroll ) ) { + + // Handle it asynchronously to allow scripts the opportunity to delay ready + window.setTimeout( jQuery.ready ); + +} else { + + // Use the handy event callback + document.addEventListener( "DOMContentLoaded", completed ); + + // A fallback to window.onload, that will always work + window.addEventListener( "load", completed ); +} + + + + +// Multifunctional method to get and set values of a collection +// The value/s can optionally be executed if it's a function +var access = function( elems, fn, key, value, chainable, emptyGet, raw ) { + var i = 0, + len = elems.length, + bulk = key == null; + + // Sets many values + if ( toType( key ) === "object" ) { + chainable = true; + for ( i in key ) { + access( elems, fn, i, key[ i ], true, emptyGet, raw ); + } + + // Sets one value + } else if ( value !== undefined ) { + chainable = true; + + if ( !isFunction( value ) ) { + raw = true; + } + + if ( bulk ) { + + // Bulk operations run against the entire set + if ( raw ) { + fn.call( elems, value ); + fn = null; + + // ...except when executing function values + } else { + bulk = fn; + fn = function( elem, _key, value ) { + return bulk.call( jQuery( elem ), value ); + }; + } + } + + if ( fn ) { + for ( ; i < len; i++ ) { + fn( + elems[ i ], key, raw ? + value : + value.call( elems[ i ], i, fn( elems[ i ], key ) ) + ); + } + } + } + + if ( chainable ) { + return elems; + } + + // Gets + if ( bulk ) { + return fn.call( elems ); + } + + return len ? fn( elems[ 0 ], key ) : emptyGet; +}; + + +// Matches dashed string for camelizing +var rmsPrefix = /^-ms-/, + rdashAlpha = /-([a-z])/g; + +// Used by camelCase as callback to replace() +function fcamelCase( _all, letter ) { + return letter.toUpperCase(); +} + +// Convert dashed to camelCase; used by the css and data modules +// Support: IE <=9 - 11, Edge 12 - 15 +// Microsoft forgot to hump their vendor prefix (#9572) +function camelCase( string ) { + return string.replace( rmsPrefix, "ms-" ).replace( rdashAlpha, fcamelCase ); +} +var acceptData = function( owner ) { + + // Accepts only: + // - Node + // - Node.ELEMENT_NODE + // - Node.DOCUMENT_NODE + // - Object + // - Any + return owner.nodeType === 1 || owner.nodeType === 9 || !( +owner.nodeType ); +}; + + + + +function Data() { + this.expando = jQuery.expando + Data.uid++; +} + +Data.uid = 1; + +Data.prototype = { + + cache: function( owner ) { + + // Check if the owner object already has a cache + var value = owner[ this.expando ]; + + // If not, create one + if ( !value ) { + value = {}; + + // We can accept data for non-element nodes in modern browsers, + // but we should not, see #8335. + // Always return an empty object. + if ( acceptData( owner ) ) { + + // If it is a node unlikely to be stringify-ed or looped over + // use plain assignment + if ( owner.nodeType ) { + owner[ this.expando ] = value; + + // Otherwise secure it in a non-enumerable property + // configurable must be true to allow the property to be + // deleted when data is removed + } else { + Object.defineProperty( owner, this.expando, { + value: value, + configurable: true + } ); + } + } + } + + return value; + }, + set: function( owner, data, value ) { + var prop, + cache = this.cache( owner ); + + // Handle: [ owner, key, value ] args + // Always use camelCase key (gh-2257) + if ( typeof data === "string" ) { + cache[ camelCase( data ) ] = value; + + // Handle: [ owner, { properties } ] args + } else { + + // Copy the properties one-by-one to the cache object + for ( prop in data ) { + cache[ camelCase( prop ) ] = data[ prop ]; + } + } + return cache; + }, + get: function( owner, key ) { + return key === undefined ? + this.cache( owner ) : + + // Always use camelCase key (gh-2257) + owner[ this.expando ] && owner[ this.expando ][ camelCase( key ) ]; + }, + access: function( owner, key, value ) { + + // In cases where either: + // + // 1. No key was specified + // 2. A string key was specified, but no value provided + // + // Take the "read" path and allow the get method to determine + // which value to return, respectively either: + // + // 1. The entire cache object + // 2. The data stored at the key + // + if ( key === undefined || + ( ( key && typeof key === "string" ) && value === undefined ) ) { + + return this.get( owner, key ); + } + + // When the key is not a string, or both a key and value + // are specified, set or extend (existing objects) with either: + // + // 1. An object of properties + // 2. A key and value + // + this.set( owner, key, value ); + + // Since the "set" path can have two possible entry points + // return the expected data based on which path was taken[*] + return value !== undefined ? value : key; + }, + remove: function( owner, key ) { + var i, + cache = owner[ this.expando ]; + + if ( cache === undefined ) { + return; + } + + if ( key !== undefined ) { + + // Support array or space separated string of keys + if ( Array.isArray( key ) ) { + + // If key is an array of keys... + // We always set camelCase keys, so remove that. + key = key.map( camelCase ); + } else { + key = camelCase( key ); + + // If a key with the spaces exists, use it. + // Otherwise, create an array by matching non-whitespace + key = key in cache ? + [ key ] : + ( key.match( rnothtmlwhite ) || [] ); + } + + i = key.length; + + while ( i-- ) { + delete cache[ key[ i ] ]; + } + } + + // Remove the expando if there's no more data + if ( key === undefined || jQuery.isEmptyObject( cache ) ) { + + // Support: Chrome <=35 - 45 + // Webkit & Blink performance suffers when deleting properties + // from DOM nodes, so set to undefined instead + // https://bugs.chromium.org/p/chromium/issues/detail?id=378607 (bug restricted) + if ( owner.nodeType ) { + owner[ this.expando ] = undefined; + } else { + delete owner[ this.expando ]; + } + } + }, + hasData: function( owner ) { + var cache = owner[ this.expando ]; + return cache !== undefined && !jQuery.isEmptyObject( cache ); + } +}; +var dataPriv = new Data(); + +var dataUser = new Data(); + + + +// Implementation Summary +// +// 1. Enforce API surface and semantic compatibility with 1.9.x branch +// 2. Improve the module's maintainability by reducing the storage +// paths to a single mechanism. +// 3. Use the same single mechanism to support "private" and "user" data. +// 4. _Never_ expose "private" data to user code (TODO: Drop _data, _removeData) +// 5. Avoid exposing implementation details on user objects (eg. expando properties) +// 6. Provide a clear path for implementation upgrade to WeakMap in 2014 + +var rbrace = /^(?:\{[\w\W]*\}|\[[\w\W]*\])$/, + rmultiDash = /[A-Z]/g; + +function getData( data ) { + if ( data === "true" ) { + return true; + } + + if ( data === "false" ) { + return false; + } + + if ( data === "null" ) { + return null; + } + + // Only convert to a number if it doesn't change the string + if ( data === +data + "" ) { + return +data; + } + + if ( rbrace.test( data ) ) { + return JSON.parse( data ); + } + + return data; +} + +function dataAttr( elem, key, data ) { + var name; + + // If nothing was found internally, try to fetch any + // data from the HTML5 data-* attribute + if ( data === undefined && elem.nodeType === 1 ) { + name = "data-" + key.replace( rmultiDash, "-$&" ).toLowerCase(); + data = elem.getAttribute( name ); + + if ( typeof data === "string" ) { + try { + data = getData( data ); + } catch ( e ) {} + + // Make sure we set the data so it isn't changed later + dataUser.set( elem, key, data ); + } else { + data = undefined; + } + } + return data; +} + +jQuery.extend( { + hasData: function( elem ) { + return dataUser.hasData( elem ) || dataPriv.hasData( elem ); + }, + + data: function( elem, name, data ) { + return dataUser.access( elem, name, data ); + }, + + removeData: function( elem, name ) { + dataUser.remove( elem, name ); + }, + + // TODO: Now that all calls to _data and _removeData have been replaced + // with direct calls to dataPriv methods, these can be deprecated. + _data: function( elem, name, data ) { + return dataPriv.access( elem, name, data ); + }, + + _removeData: function( elem, name ) { + dataPriv.remove( elem, name ); + } +} ); + +jQuery.fn.extend( { + data: function( key, value ) { + var i, name, data, + elem = this[ 0 ], + attrs = elem && elem.attributes; + + // Gets all values + if ( key === undefined ) { + if ( this.length ) { + data = dataUser.get( elem ); + + if ( elem.nodeType === 1 && !dataPriv.get( elem, "hasDataAttrs" ) ) { + i = attrs.length; + while ( i-- ) { + + // Support: IE 11 only + // The attrs elements can be null (#14894) + if ( attrs[ i ] ) { + name = attrs[ i ].name; + if ( name.indexOf( "data-" ) === 0 ) { + name = camelCase( name.slice( 5 ) ); + dataAttr( elem, name, data[ name ] ); + } + } + } + dataPriv.set( elem, "hasDataAttrs", true ); + } + } + + return data; + } + + // Sets multiple values + if ( typeof key === "object" ) { + return this.each( function() { + dataUser.set( this, key ); + } ); + } + + return access( this, function( value ) { + var data; + + // The calling jQuery object (element matches) is not empty + // (and therefore has an element appears at this[ 0 ]) and the + // `value` parameter was not undefined. An empty jQuery object + // will result in `undefined` for elem = this[ 0 ] which will + // throw an exception if an attempt to read a data cache is made. + if ( elem && value === undefined ) { + + // Attempt to get data from the cache + // The key will always be camelCased in Data + data = dataUser.get( elem, key ); + if ( data !== undefined ) { + return data; + } + + // Attempt to "discover" the data in + // HTML5 custom data-* attrs + data = dataAttr( elem, key ); + if ( data !== undefined ) { + return data; + } + + // We tried really hard, but the data doesn't exist. + return; + } + + // Set the data... + this.each( function() { + + // We always store the camelCased key + dataUser.set( this, key, value ); + } ); + }, null, value, arguments.length > 1, null, true ); + }, + + removeData: function( key ) { + return this.each( function() { + dataUser.remove( this, key ); + } ); + } +} ); + + +jQuery.extend( { + queue: function( elem, type, data ) { + var queue; + + if ( elem ) { + type = ( type || "fx" ) + "queue"; + queue = dataPriv.get( elem, type ); + + // Speed up dequeue by getting out quickly if this is just a lookup + if ( data ) { + if ( !queue || Array.isArray( data ) ) { + queue = dataPriv.access( elem, type, jQuery.makeArray( data ) ); + } else { + queue.push( data ); + } + } + return queue || []; + } + }, + + dequeue: function( elem, type ) { + type = type || "fx"; + + var queue = jQuery.queue( elem, type ), + startLength = queue.length, + fn = queue.shift(), + hooks = jQuery._queueHooks( elem, type ), + next = function() { + jQuery.dequeue( elem, type ); + }; + + // If the fx queue is dequeued, always remove the progress sentinel + if ( fn === "inprogress" ) { + fn = queue.shift(); + startLength--; + } + + if ( fn ) { + + // Add a progress sentinel to prevent the fx queue from being + // automatically dequeued + if ( type === "fx" ) { + queue.unshift( "inprogress" ); + } + + // Clear up the last queue stop function + delete hooks.stop; + fn.call( elem, next, hooks ); + } + + if ( !startLength && hooks ) { + hooks.empty.fire(); + } + }, + + // Not public - generate a queueHooks object, or return the current one + _queueHooks: function( elem, type ) { + var key = type + "queueHooks"; + return dataPriv.get( elem, key ) || dataPriv.access( elem, key, { + empty: jQuery.Callbacks( "once memory" ).add( function() { + dataPriv.remove( elem, [ type + "queue", key ] ); + } ) + } ); + } +} ); + +jQuery.fn.extend( { + queue: function( type, data ) { + var setter = 2; + + if ( typeof type !== "string" ) { + data = type; + type = "fx"; + setter--; + } + + if ( arguments.length < setter ) { + return jQuery.queue( this[ 0 ], type ); + } + + return data === undefined ? + this : + this.each( function() { + var queue = jQuery.queue( this, type, data ); + + // Ensure a hooks for this queue + jQuery._queueHooks( this, type ); + + if ( type === "fx" && queue[ 0 ] !== "inprogress" ) { + jQuery.dequeue( this, type ); + } + } ); + }, + dequeue: function( type ) { + return this.each( function() { + jQuery.dequeue( this, type ); + } ); + }, + clearQueue: function( type ) { + return this.queue( type || "fx", [] ); + }, + + // Get a promise resolved when queues of a certain type + // are emptied (fx is the type by default) + promise: function( type, obj ) { + var tmp, + count = 1, + defer = jQuery.Deferred(), + elements = this, + i = this.length, + resolve = function() { + if ( !( --count ) ) { + defer.resolveWith( elements, [ elements ] ); + } + }; + + if ( typeof type !== "string" ) { + obj = type; + type = undefined; + } + type = type || "fx"; + + while ( i-- ) { + tmp = dataPriv.get( elements[ i ], type + "queueHooks" ); + if ( tmp && tmp.empty ) { + count++; + tmp.empty.add( resolve ); + } + } + resolve(); + return defer.promise( obj ); + } +} ); +var pnum = ( /[+-]?(?:\d*\.|)\d+(?:[eE][+-]?\d+|)/ ).source; + +var rcssNum = new RegExp( "^(?:([+-])=|)(" + pnum + ")([a-z%]*)$", "i" ); + + +var cssExpand = [ "Top", "Right", "Bottom", "Left" ]; + +var documentElement = document.documentElement; + + + + var isAttached = function( elem ) { + return jQuery.contains( elem.ownerDocument, elem ); + }, + composed = { composed: true }; + + // Support: IE 9 - 11+, Edge 12 - 18+, iOS 10.0 - 10.2 only + // Check attachment across shadow DOM boundaries when possible (gh-3504) + // Support: iOS 10.0-10.2 only + // Early iOS 10 versions support `attachShadow` but not `getRootNode`, + // leading to errors. We need to check for `getRootNode`. + if ( documentElement.getRootNode ) { + isAttached = function( elem ) { + return jQuery.contains( elem.ownerDocument, elem ) || + elem.getRootNode( composed ) === elem.ownerDocument; + }; + } +var isHiddenWithinTree = function( elem, el ) { + + // isHiddenWithinTree might be called from jQuery#filter function; + // in that case, element will be second argument + elem = el || elem; + + // Inline style trumps all + return elem.style.display === "none" || + elem.style.display === "" && + + // Otherwise, check computed style + // Support: Firefox <=43 - 45 + // Disconnected elements can have computed display: none, so first confirm that elem is + // in the document. + isAttached( elem ) && + + jQuery.css( elem, "display" ) === "none"; + }; + + + +function adjustCSS( elem, prop, valueParts, tween ) { + var adjusted, scale, + maxIterations = 20, + currentValue = tween ? + function() { + return tween.cur(); + } : + function() { + return jQuery.css( elem, prop, "" ); + }, + initial = currentValue(), + unit = valueParts && valueParts[ 3 ] || ( jQuery.cssNumber[ prop ] ? "" : "px" ), + + // Starting value computation is required for potential unit mismatches + initialInUnit = elem.nodeType && + ( jQuery.cssNumber[ prop ] || unit !== "px" && +initial ) && + rcssNum.exec( jQuery.css( elem, prop ) ); + + if ( initialInUnit && initialInUnit[ 3 ] !== unit ) { + + // Support: Firefox <=54 + // Halve the iteration target value to prevent interference from CSS upper bounds (gh-2144) + initial = initial / 2; + + // Trust units reported by jQuery.css + unit = unit || initialInUnit[ 3 ]; + + // Iteratively approximate from a nonzero starting point + initialInUnit = +initial || 1; + + while ( maxIterations-- ) { + + // Evaluate and update our best guess (doubling guesses that zero out). + // Finish if the scale equals or crosses 1 (making the old*new product non-positive). + jQuery.style( elem, prop, initialInUnit + unit ); + if ( ( 1 - scale ) * ( 1 - ( scale = currentValue() / initial || 0.5 ) ) <= 0 ) { + maxIterations = 0; + } + initialInUnit = initialInUnit / scale; + + } + + initialInUnit = initialInUnit * 2; + jQuery.style( elem, prop, initialInUnit + unit ); + + // Make sure we update the tween properties later on + valueParts = valueParts || []; + } + + if ( valueParts ) { + initialInUnit = +initialInUnit || +initial || 0; + + // Apply relative offset (+=/-=) if specified + adjusted = valueParts[ 1 ] ? + initialInUnit + ( valueParts[ 1 ] + 1 ) * valueParts[ 2 ] : + +valueParts[ 2 ]; + if ( tween ) { + tween.unit = unit; + tween.start = initialInUnit; + tween.end = adjusted; + } + } + return adjusted; +} + + +var defaultDisplayMap = {}; + +function getDefaultDisplay( elem ) { + var temp, + doc = elem.ownerDocument, + nodeName = elem.nodeName, + display = defaultDisplayMap[ nodeName ]; + + if ( display ) { + return display; + } + + temp = doc.body.appendChild( doc.createElement( nodeName ) ); + display = jQuery.css( temp, "display" ); + + temp.parentNode.removeChild( temp ); + + if ( display === "none" ) { + display = "block"; + } + defaultDisplayMap[ nodeName ] = display; + + return display; +} + +function showHide( elements, show ) { + var display, elem, + values = [], + index = 0, + length = elements.length; + + // Determine new display value for elements that need to change + for ( ; index < length; index++ ) { + elem = elements[ index ]; + if ( !elem.style ) { + continue; + } + + display = elem.style.display; + if ( show ) { + + // Since we force visibility upon cascade-hidden elements, an immediate (and slow) + // check is required in this first loop unless we have a nonempty display value (either + // inline or about-to-be-restored) + if ( display === "none" ) { + values[ index ] = dataPriv.get( elem, "display" ) || null; + if ( !values[ index ] ) { + elem.style.display = ""; + } + } + if ( elem.style.display === "" && isHiddenWithinTree( elem ) ) { + values[ index ] = getDefaultDisplay( elem ); + } + } else { + if ( display !== "none" ) { + values[ index ] = "none"; + + // Remember what we're overwriting + dataPriv.set( elem, "display", display ); + } + } + } + + // Set the display of the elements in a second loop to avoid constant reflow + for ( index = 0; index < length; index++ ) { + if ( values[ index ] != null ) { + elements[ index ].style.display = values[ index ]; + } + } + + return elements; +} + +jQuery.fn.extend( { + show: function() { + return showHide( this, true ); + }, + hide: function() { + return showHide( this ); + }, + toggle: function( state ) { + if ( typeof state === "boolean" ) { + return state ? this.show() : this.hide(); + } + + return this.each( function() { + if ( isHiddenWithinTree( this ) ) { + jQuery( this ).show(); + } else { + jQuery( this ).hide(); + } + } ); + } +} ); +var rcheckableType = ( /^(?:checkbox|radio)$/i ); + +var rtagName = ( /<([a-z][^\/\0>\x20\t\r\n\f]*)/i ); + +var rscriptType = ( /^$|^module$|\/(?:java|ecma)script/i ); + + + +( function() { + var fragment = document.createDocumentFragment(), + div = fragment.appendChild( document.createElement( "div" ) ), + input = document.createElement( "input" ); + + // Support: Android 4.0 - 4.3 only + // Check state lost if the name is set (#11217) + // Support: Windows Web Apps (WWA) + // `name` and `type` must use .setAttribute for WWA (#14901) + input.setAttribute( "type", "radio" ); + input.setAttribute( "checked", "checked" ); + input.setAttribute( "name", "t" ); + + div.appendChild( input ); + + // Support: Android <=4.1 only + // Older WebKit doesn't clone checked state correctly in fragments + support.checkClone = div.cloneNode( true ).cloneNode( true ).lastChild.checked; + + // Support: IE <=11 only + // Make sure textarea (and checkbox) defaultValue is properly cloned + div.innerHTML = ""; + support.noCloneChecked = !!div.cloneNode( true ).lastChild.defaultValue; + + // Support: IE <=9 only + // IE <=9 replaces
", "
" ], + col: [ 2, "", "
" ], + tr: [ 2, "", "
" ], + td: [ 3, "", "
" ], + + _default: [ 0, "", "" ] +}; + +wrapMap.tbody = wrapMap.tfoot = wrapMap.colgroup = wrapMap.caption = wrapMap.thead; +wrapMap.th = wrapMap.td; + +// Support: IE <=9 only +if ( !support.option ) { + wrapMap.optgroup = wrapMap.option = [ 1, "" ]; +} + + +function getAll( context, tag ) { + + // Support: IE <=9 - 11 only + // Use typeof to avoid zero-argument method invocation on host objects (#15151) + var ret; + + if ( typeof context.getElementsByTagName !== "undefined" ) { + ret = context.getElementsByTagName( tag || "*" ); + + } else if ( typeof context.querySelectorAll !== "undefined" ) { + ret = context.querySelectorAll( tag || "*" ); + + } else { + ret = []; + } + + if ( tag === undefined || tag && nodeName( context, tag ) ) { + return jQuery.merge( [ context ], ret ); + } + + return ret; +} + + +// Mark scripts as having already been evaluated +function setGlobalEval( elems, refElements ) { + var i = 0, + l = elems.length; + + for ( ; i < l; i++ ) { + dataPriv.set( + elems[ i ], + "globalEval", + !refElements || dataPriv.get( refElements[ i ], "globalEval" ) + ); + } +} + + +var rhtml = /<|&#?\w+;/; + +function buildFragment( elems, context, scripts, selection, ignored ) { + var elem, tmp, tag, wrap, attached, j, + fragment = context.createDocumentFragment(), + nodes = [], + i = 0, + l = elems.length; + + for ( ; i < l; i++ ) { + elem = elems[ i ]; + + if ( elem || elem === 0 ) { + + // Add nodes directly + if ( toType( elem ) === "object" ) { + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + jQuery.merge( nodes, elem.nodeType ? [ elem ] : elem ); + + // Convert non-html into a text node + } else if ( !rhtml.test( elem ) ) { + nodes.push( context.createTextNode( elem ) ); + + // Convert html into DOM nodes + } else { + tmp = tmp || fragment.appendChild( context.createElement( "div" ) ); + + // Deserialize a standard representation + tag = ( rtagName.exec( elem ) || [ "", "" ] )[ 1 ].toLowerCase(); + wrap = wrapMap[ tag ] || wrapMap._default; + tmp.innerHTML = wrap[ 1 ] + jQuery.htmlPrefilter( elem ) + wrap[ 2 ]; + + // Descend through wrappers to the right content + j = wrap[ 0 ]; + while ( j-- ) { + tmp = tmp.lastChild; + } + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + jQuery.merge( nodes, tmp.childNodes ); + + // Remember the top-level container + tmp = fragment.firstChild; + + // Ensure the created nodes are orphaned (#12392) + tmp.textContent = ""; + } + } + } + + // Remove wrapper from fragment + fragment.textContent = ""; + + i = 0; + while ( ( elem = nodes[ i++ ] ) ) { + + // Skip elements already in the context collection (trac-4087) + if ( selection && jQuery.inArray( elem, selection ) > -1 ) { + if ( ignored ) { + ignored.push( elem ); + } + continue; + } + + attached = isAttached( elem ); + + // Append to fragment + tmp = getAll( fragment.appendChild( elem ), "script" ); + + // Preserve script evaluation history + if ( attached ) { + setGlobalEval( tmp ); + } + + // Capture executables + if ( scripts ) { + j = 0; + while ( ( elem = tmp[ j++ ] ) ) { + if ( rscriptType.test( elem.type || "" ) ) { + scripts.push( elem ); + } + } + } + } + + return fragment; +} + + +var rtypenamespace = /^([^.]*)(?:\.(.+)|)/; + +function returnTrue() { + return true; +} + +function returnFalse() { + return false; +} + +// Support: IE <=9 - 11+ +// focus() and blur() are asynchronous, except when they are no-op. +// So expect focus to be synchronous when the element is already active, +// and blur to be synchronous when the element is not already active. +// (focus and blur are always synchronous in other supported browsers, +// this just defines when we can count on it). +function expectSync( elem, type ) { + return ( elem === safeActiveElement() ) === ( type === "focus" ); +} + +// Support: IE <=9 only +// Accessing document.activeElement can throw unexpectedly +// https://bugs.jquery.com/ticket/13393 +function safeActiveElement() { + try { + return document.activeElement; + } catch ( err ) { } +} + +function on( elem, types, selector, data, fn, one ) { + var origFn, type; + + // Types can be a map of types/handlers + if ( typeof types === "object" ) { + + // ( types-Object, selector, data ) + if ( typeof selector !== "string" ) { + + // ( types-Object, data ) + data = data || selector; + selector = undefined; + } + for ( type in types ) { + on( elem, type, selector, data, types[ type ], one ); + } + return elem; + } + + if ( data == null && fn == null ) { + + // ( types, fn ) + fn = selector; + data = selector = undefined; + } else if ( fn == null ) { + if ( typeof selector === "string" ) { + + // ( types, selector, fn ) + fn = data; + data = undefined; + } else { + + // ( types, data, fn ) + fn = data; + data = selector; + selector = undefined; + } + } + if ( fn === false ) { + fn = returnFalse; + } else if ( !fn ) { + return elem; + } + + if ( one === 1 ) { + origFn = fn; + fn = function( event ) { + + // Can use an empty set, since event contains the info + jQuery().off( event ); + return origFn.apply( this, arguments ); + }; + + // Use same guid so caller can remove using origFn + fn.guid = origFn.guid || ( origFn.guid = jQuery.guid++ ); + } + return elem.each( function() { + jQuery.event.add( this, types, fn, data, selector ); + } ); +} + +/* + * Helper functions for managing events -- not part of the public interface. + * Props to Dean Edwards' addEvent library for many of the ideas. + */ +jQuery.event = { + + global: {}, + + add: function( elem, types, handler, data, selector ) { + + var handleObjIn, eventHandle, tmp, + events, t, handleObj, + special, handlers, type, namespaces, origType, + elemData = dataPriv.get( elem ); + + // Only attach events to objects that accept data + if ( !acceptData( elem ) ) { + return; + } + + // Caller can pass in an object of custom data in lieu of the handler + if ( handler.handler ) { + handleObjIn = handler; + handler = handleObjIn.handler; + selector = handleObjIn.selector; + } + + // Ensure that invalid selectors throw exceptions at attach time + // Evaluate against documentElement in case elem is a non-element node (e.g., document) + if ( selector ) { + jQuery.find.matchesSelector( documentElement, selector ); + } + + // Make sure that the handler has a unique ID, used to find/remove it later + if ( !handler.guid ) { + handler.guid = jQuery.guid++; + } + + // Init the element's event structure and main handler, if this is the first + if ( !( events = elemData.events ) ) { + events = elemData.events = Object.create( null ); + } + if ( !( eventHandle = elemData.handle ) ) { + eventHandle = elemData.handle = function( e ) { + + // Discard the second event of a jQuery.event.trigger() and + // when an event is called after a page has unloaded + return typeof jQuery !== "undefined" && jQuery.event.triggered !== e.type ? + jQuery.event.dispatch.apply( elem, arguments ) : undefined; + }; + } + + // Handle multiple events separated by a space + types = ( types || "" ).match( rnothtmlwhite ) || [ "" ]; + t = types.length; + while ( t-- ) { + tmp = rtypenamespace.exec( types[ t ] ) || []; + type = origType = tmp[ 1 ]; + namespaces = ( tmp[ 2 ] || "" ).split( "." ).sort(); + + // There *must* be a type, no attaching namespace-only handlers + if ( !type ) { + continue; + } + + // If event changes its type, use the special event handlers for the changed type + special = jQuery.event.special[ type ] || {}; + + // If selector defined, determine special event api type, otherwise given type + type = ( selector ? special.delegateType : special.bindType ) || type; + + // Update special based on newly reset type + special = jQuery.event.special[ type ] || {}; + + // handleObj is passed to all event handlers + handleObj = jQuery.extend( { + type: type, + origType: origType, + data: data, + handler: handler, + guid: handler.guid, + selector: selector, + needsContext: selector && jQuery.expr.match.needsContext.test( selector ), + namespace: namespaces.join( "." ) + }, handleObjIn ); + + // Init the event handler queue if we're the first + if ( !( handlers = events[ type ] ) ) { + handlers = events[ type ] = []; + handlers.delegateCount = 0; + + // Only use addEventListener if the special events handler returns false + if ( !special.setup || + special.setup.call( elem, data, namespaces, eventHandle ) === false ) { + + if ( elem.addEventListener ) { + elem.addEventListener( type, eventHandle ); + } + } + } + + if ( special.add ) { + special.add.call( elem, handleObj ); + + if ( !handleObj.handler.guid ) { + handleObj.handler.guid = handler.guid; + } + } + + // Add to the element's handler list, delegates in front + if ( selector ) { + handlers.splice( handlers.delegateCount++, 0, handleObj ); + } else { + handlers.push( handleObj ); + } + + // Keep track of which events have ever been used, for event optimization + jQuery.event.global[ type ] = true; + } + + }, + + // Detach an event or set of events from an element + remove: function( elem, types, handler, selector, mappedTypes ) { + + var j, origCount, tmp, + events, t, handleObj, + special, handlers, type, namespaces, origType, + elemData = dataPriv.hasData( elem ) && dataPriv.get( elem ); + + if ( !elemData || !( events = elemData.events ) ) { + return; + } + + // Once for each type.namespace in types; type may be omitted + types = ( types || "" ).match( rnothtmlwhite ) || [ "" ]; + t = types.length; + while ( t-- ) { + tmp = rtypenamespace.exec( types[ t ] ) || []; + type = origType = tmp[ 1 ]; + namespaces = ( tmp[ 2 ] || "" ).split( "." ).sort(); + + // Unbind all events (on this namespace, if provided) for the element + if ( !type ) { + for ( type in events ) { + jQuery.event.remove( elem, type + types[ t ], handler, selector, true ); + } + continue; + } + + special = jQuery.event.special[ type ] || {}; + type = ( selector ? special.delegateType : special.bindType ) || type; + handlers = events[ type ] || []; + tmp = tmp[ 2 ] && + new RegExp( "(^|\\.)" + namespaces.join( "\\.(?:.*\\.|)" ) + "(\\.|$)" ); + + // Remove matching events + origCount = j = handlers.length; + while ( j-- ) { + handleObj = handlers[ j ]; + + if ( ( mappedTypes || origType === handleObj.origType ) && + ( !handler || handler.guid === handleObj.guid ) && + ( !tmp || tmp.test( handleObj.namespace ) ) && + ( !selector || selector === handleObj.selector || + selector === "**" && handleObj.selector ) ) { + handlers.splice( j, 1 ); + + if ( handleObj.selector ) { + handlers.delegateCount--; + } + if ( special.remove ) { + special.remove.call( elem, handleObj ); + } + } + } + + // Remove generic event handler if we removed something and no more handlers exist + // (avoids potential for endless recursion during removal of special event handlers) + if ( origCount && !handlers.length ) { + if ( !special.teardown || + special.teardown.call( elem, namespaces, elemData.handle ) === false ) { + + jQuery.removeEvent( elem, type, elemData.handle ); + } + + delete events[ type ]; + } + } + + // Remove data and the expando if it's no longer used + if ( jQuery.isEmptyObject( events ) ) { + dataPriv.remove( elem, "handle events" ); + } + }, + + dispatch: function( nativeEvent ) { + + var i, j, ret, matched, handleObj, handlerQueue, + args = new Array( arguments.length ), + + // Make a writable jQuery.Event from the native event object + event = jQuery.event.fix( nativeEvent ), + + handlers = ( + dataPriv.get( this, "events" ) || Object.create( null ) + )[ event.type ] || [], + special = jQuery.event.special[ event.type ] || {}; + + // Use the fix-ed jQuery.Event rather than the (read-only) native event + args[ 0 ] = event; + + for ( i = 1; i < arguments.length; i++ ) { + args[ i ] = arguments[ i ]; + } + + event.delegateTarget = this; + + // Call the preDispatch hook for the mapped type, and let it bail if desired + if ( special.preDispatch && special.preDispatch.call( this, event ) === false ) { + return; + } + + // Determine handlers + handlerQueue = jQuery.event.handlers.call( this, event, handlers ); + + // Run delegates first; they may want to stop propagation beneath us + i = 0; + while ( ( matched = handlerQueue[ i++ ] ) && !event.isPropagationStopped() ) { + event.currentTarget = matched.elem; + + j = 0; + while ( ( handleObj = matched.handlers[ j++ ] ) && + !event.isImmediatePropagationStopped() ) { + + // If the event is namespaced, then each handler is only invoked if it is + // specially universal or its namespaces are a superset of the event's. + if ( !event.rnamespace || handleObj.namespace === false || + event.rnamespace.test( handleObj.namespace ) ) { + + event.handleObj = handleObj; + event.data = handleObj.data; + + ret = ( ( jQuery.event.special[ handleObj.origType ] || {} ).handle || + handleObj.handler ).apply( matched.elem, args ); + + if ( ret !== undefined ) { + if ( ( event.result = ret ) === false ) { + event.preventDefault(); + event.stopPropagation(); + } + } + } + } + } + + // Call the postDispatch hook for the mapped type + if ( special.postDispatch ) { + special.postDispatch.call( this, event ); + } + + return event.result; + }, + + handlers: function( event, handlers ) { + var i, handleObj, sel, matchedHandlers, matchedSelectors, + handlerQueue = [], + delegateCount = handlers.delegateCount, + cur = event.target; + + // Find delegate handlers + if ( delegateCount && + + // Support: IE <=9 + // Black-hole SVG instance trees (trac-13180) + cur.nodeType && + + // Support: Firefox <=42 + // Suppress spec-violating clicks indicating a non-primary pointer button (trac-3861) + // https://www.w3.org/TR/DOM-Level-3-Events/#event-type-click + // Support: IE 11 only + // ...but not arrow key "clicks" of radio inputs, which can have `button` -1 (gh-2343) + !( event.type === "click" && event.button >= 1 ) ) { + + for ( ; cur !== this; cur = cur.parentNode || this ) { + + // Don't check non-elements (#13208) + // Don't process clicks on disabled elements (#6911, #8165, #11382, #11764) + if ( cur.nodeType === 1 && !( event.type === "click" && cur.disabled === true ) ) { + matchedHandlers = []; + matchedSelectors = {}; + for ( i = 0; i < delegateCount; i++ ) { + handleObj = handlers[ i ]; + + // Don't conflict with Object.prototype properties (#13203) + sel = handleObj.selector + " "; + + if ( matchedSelectors[ sel ] === undefined ) { + matchedSelectors[ sel ] = handleObj.needsContext ? + jQuery( sel, this ).index( cur ) > -1 : + jQuery.find( sel, this, null, [ cur ] ).length; + } + if ( matchedSelectors[ sel ] ) { + matchedHandlers.push( handleObj ); + } + } + if ( matchedHandlers.length ) { + handlerQueue.push( { elem: cur, handlers: matchedHandlers } ); + } + } + } + } + + // Add the remaining (directly-bound) handlers + cur = this; + if ( delegateCount < handlers.length ) { + handlerQueue.push( { elem: cur, handlers: handlers.slice( delegateCount ) } ); + } + + return handlerQueue; + }, + + addProp: function( name, hook ) { + Object.defineProperty( jQuery.Event.prototype, name, { + enumerable: true, + configurable: true, + + get: isFunction( hook ) ? + function() { + if ( this.originalEvent ) { + return hook( this.originalEvent ); + } + } : + function() { + if ( this.originalEvent ) { + return this.originalEvent[ name ]; + } + }, + + set: function( value ) { + Object.defineProperty( this, name, { + enumerable: true, + configurable: true, + writable: true, + value: value + } ); + } + } ); + }, + + fix: function( originalEvent ) { + return originalEvent[ jQuery.expando ] ? + originalEvent : + new jQuery.Event( originalEvent ); + }, + + special: { + load: { + + // Prevent triggered image.load events from bubbling to window.load + noBubble: true + }, + click: { + + // Utilize native event to ensure correct state for checkable inputs + setup: function( data ) { + + // For mutual compressibility with _default, replace `this` access with a local var. + // `|| data` is dead code meant only to preserve the variable through minification. + var el = this || data; + + // Claim the first handler + if ( rcheckableType.test( el.type ) && + el.click && nodeName( el, "input" ) ) { + + // dataPriv.set( el, "click", ... ) + leverageNative( el, "click", returnTrue ); + } + + // Return false to allow normal processing in the caller + return false; + }, + trigger: function( data ) { + + // For mutual compressibility with _default, replace `this` access with a local var. + // `|| data` is dead code meant only to preserve the variable through minification. + var el = this || data; + + // Force setup before triggering a click + if ( rcheckableType.test( el.type ) && + el.click && nodeName( el, "input" ) ) { + + leverageNative( el, "click" ); + } + + // Return non-false to allow normal event-path propagation + return true; + }, + + // For cross-browser consistency, suppress native .click() on links + // Also prevent it if we're currently inside a leveraged native-event stack + _default: function( event ) { + var target = event.target; + return rcheckableType.test( target.type ) && + target.click && nodeName( target, "input" ) && + dataPriv.get( target, "click" ) || + nodeName( target, "a" ); + } + }, + + beforeunload: { + postDispatch: function( event ) { + + // Support: Firefox 20+ + // Firefox doesn't alert if the returnValue field is not set. + if ( event.result !== undefined && event.originalEvent ) { + event.originalEvent.returnValue = event.result; + } + } + } + } +}; + +// Ensure the presence of an event listener that handles manually-triggered +// synthetic events by interrupting progress until reinvoked in response to +// *native* events that it fires directly, ensuring that state changes have +// already occurred before other listeners are invoked. +function leverageNative( el, type, expectSync ) { + + // Missing expectSync indicates a trigger call, which must force setup through jQuery.event.add + if ( !expectSync ) { + if ( dataPriv.get( el, type ) === undefined ) { + jQuery.event.add( el, type, returnTrue ); + } + return; + } + + // Register the controller as a special universal handler for all event namespaces + dataPriv.set( el, type, false ); + jQuery.event.add( el, type, { + namespace: false, + handler: function( event ) { + var notAsync, result, + saved = dataPriv.get( this, type ); + + if ( ( event.isTrigger & 1 ) && this[ type ] ) { + + // Interrupt processing of the outer synthetic .trigger()ed event + // Saved data should be false in such cases, but might be a leftover capture object + // from an async native handler (gh-4350) + if ( !saved.length ) { + + // Store arguments for use when handling the inner native event + // There will always be at least one argument (an event object), so this array + // will not be confused with a leftover capture object. + saved = slice.call( arguments ); + dataPriv.set( this, type, saved ); + + // Trigger the native event and capture its result + // Support: IE <=9 - 11+ + // focus() and blur() are asynchronous + notAsync = expectSync( this, type ); + this[ type ](); + result = dataPriv.get( this, type ); + if ( saved !== result || notAsync ) { + dataPriv.set( this, type, false ); + } else { + result = {}; + } + if ( saved !== result ) { + + // Cancel the outer synthetic event + event.stopImmediatePropagation(); + event.preventDefault(); + + // Support: Chrome 86+ + // In Chrome, if an element having a focusout handler is blurred by + // clicking outside of it, it invokes the handler synchronously. If + // that handler calls `.remove()` on the element, the data is cleared, + // leaving `result` undefined. We need to guard against this. + return result && result.value; + } + + // If this is an inner synthetic event for an event with a bubbling surrogate + // (focus or blur), assume that the surrogate already propagated from triggering the + // native event and prevent that from happening again here. + // This technically gets the ordering wrong w.r.t. to `.trigger()` (in which the + // bubbling surrogate propagates *after* the non-bubbling base), but that seems + // less bad than duplication. + } else if ( ( jQuery.event.special[ type ] || {} ).delegateType ) { + event.stopPropagation(); + } + + // If this is a native event triggered above, everything is now in order + // Fire an inner synthetic event with the original arguments + } else if ( saved.length ) { + + // ...and capture the result + dataPriv.set( this, type, { + value: jQuery.event.trigger( + + // Support: IE <=9 - 11+ + // Extend with the prototype to reset the above stopImmediatePropagation() + jQuery.extend( saved[ 0 ], jQuery.Event.prototype ), + saved.slice( 1 ), + this + ) + } ); + + // Abort handling of the native event + event.stopImmediatePropagation(); + } + } + } ); +} + +jQuery.removeEvent = function( elem, type, handle ) { + + // This "if" is needed for plain objects + if ( elem.removeEventListener ) { + elem.removeEventListener( type, handle ); + } +}; + +jQuery.Event = function( src, props ) { + + // Allow instantiation without the 'new' keyword + if ( !( this instanceof jQuery.Event ) ) { + return new jQuery.Event( src, props ); + } + + // Event object + if ( src && src.type ) { + this.originalEvent = src; + this.type = src.type; + + // Events bubbling up the document may have been marked as prevented + // by a handler lower down the tree; reflect the correct value. + this.isDefaultPrevented = src.defaultPrevented || + src.defaultPrevented === undefined && + + // Support: Android <=2.3 only + src.returnValue === false ? + returnTrue : + returnFalse; + + // Create target properties + // Support: Safari <=6 - 7 only + // Target should not be a text node (#504, #13143) + this.target = ( src.target && src.target.nodeType === 3 ) ? + src.target.parentNode : + src.target; + + this.currentTarget = src.currentTarget; + this.relatedTarget = src.relatedTarget; + + // Event type + } else { + this.type = src; + } + + // Put explicitly provided properties onto the event object + if ( props ) { + jQuery.extend( this, props ); + } + + // Create a timestamp if incoming event doesn't have one + this.timeStamp = src && src.timeStamp || Date.now(); + + // Mark it as fixed + this[ jQuery.expando ] = true; +}; + +// jQuery.Event is based on DOM3 Events as specified by the ECMAScript Language Binding +// https://www.w3.org/TR/2003/WD-DOM-Level-3-Events-20030331/ecma-script-binding.html +jQuery.Event.prototype = { + constructor: jQuery.Event, + isDefaultPrevented: returnFalse, + isPropagationStopped: returnFalse, + isImmediatePropagationStopped: returnFalse, + isSimulated: false, + + preventDefault: function() { + var e = this.originalEvent; + + this.isDefaultPrevented = returnTrue; + + if ( e && !this.isSimulated ) { + e.preventDefault(); + } + }, + stopPropagation: function() { + var e = this.originalEvent; + + this.isPropagationStopped = returnTrue; + + if ( e && !this.isSimulated ) { + e.stopPropagation(); + } + }, + stopImmediatePropagation: function() { + var e = this.originalEvent; + + this.isImmediatePropagationStopped = returnTrue; + + if ( e && !this.isSimulated ) { + e.stopImmediatePropagation(); + } + + this.stopPropagation(); + } +}; + +// Includes all common event props including KeyEvent and MouseEvent specific props +jQuery.each( { + altKey: true, + bubbles: true, + cancelable: true, + changedTouches: true, + ctrlKey: true, + detail: true, + eventPhase: true, + metaKey: true, + pageX: true, + pageY: true, + shiftKey: true, + view: true, + "char": true, + code: true, + charCode: true, + key: true, + keyCode: true, + button: true, + buttons: true, + clientX: true, + clientY: true, + offsetX: true, + offsetY: true, + pointerId: true, + pointerType: true, + screenX: true, + screenY: true, + targetTouches: true, + toElement: true, + touches: true, + which: true +}, jQuery.event.addProp ); + +jQuery.each( { focus: "focusin", blur: "focusout" }, function( type, delegateType ) { + jQuery.event.special[ type ] = { + + // Utilize native event if possible so blur/focus sequence is correct + setup: function() { + + // Claim the first handler + // dataPriv.set( this, "focus", ... ) + // dataPriv.set( this, "blur", ... ) + leverageNative( this, type, expectSync ); + + // Return false to allow normal processing in the caller + return false; + }, + trigger: function() { + + // Force setup before trigger + leverageNative( this, type ); + + // Return non-false to allow normal event-path propagation + return true; + }, + + // Suppress native focus or blur as it's already being fired + // in leverageNative. + _default: function() { + return true; + }, + + delegateType: delegateType + }; +} ); + +// Create mouseenter/leave events using mouseover/out and event-time checks +// so that event delegation works in jQuery. +// Do the same for pointerenter/pointerleave and pointerover/pointerout +// +// Support: Safari 7 only +// Safari sends mouseenter too often; see: +// https://bugs.chromium.org/p/chromium/issues/detail?id=470258 +// for the description of the bug (it existed in older Chrome versions as well). +jQuery.each( { + mouseenter: "mouseover", + mouseleave: "mouseout", + pointerenter: "pointerover", + pointerleave: "pointerout" +}, function( orig, fix ) { + jQuery.event.special[ orig ] = { + delegateType: fix, + bindType: fix, + + handle: function( event ) { + var ret, + target = this, + related = event.relatedTarget, + handleObj = event.handleObj; + + // For mouseenter/leave call the handler if related is outside the target. + // NB: No relatedTarget if the mouse left/entered the browser window + if ( !related || ( related !== target && !jQuery.contains( target, related ) ) ) { + event.type = handleObj.origType; + ret = handleObj.handler.apply( this, arguments ); + event.type = fix; + } + return ret; + } + }; +} ); + +jQuery.fn.extend( { + + on: function( types, selector, data, fn ) { + return on( this, types, selector, data, fn ); + }, + one: function( types, selector, data, fn ) { + return on( this, types, selector, data, fn, 1 ); + }, + off: function( types, selector, fn ) { + var handleObj, type; + if ( types && types.preventDefault && types.handleObj ) { + + // ( event ) dispatched jQuery.Event + handleObj = types.handleObj; + jQuery( types.delegateTarget ).off( + handleObj.namespace ? + handleObj.origType + "." + handleObj.namespace : + handleObj.origType, + handleObj.selector, + handleObj.handler + ); + return this; + } + if ( typeof types === "object" ) { + + // ( types-object [, selector] ) + for ( type in types ) { + this.off( type, selector, types[ type ] ); + } + return this; + } + if ( selector === false || typeof selector === "function" ) { + + // ( types [, fn] ) + fn = selector; + selector = undefined; + } + if ( fn === false ) { + fn = returnFalse; + } + return this.each( function() { + jQuery.event.remove( this, types, fn, selector ); + } ); + } +} ); + + +var + + // Support: IE <=10 - 11, Edge 12 - 13 only + // In IE/Edge using regex groups here causes severe slowdowns. + // See https://connect.microsoft.com/IE/feedback/details/1736512/ + rnoInnerhtml = /\s*$/g; + +// Prefer a tbody over its parent table for containing new rows +function manipulationTarget( elem, content ) { + if ( nodeName( elem, "table" ) && + nodeName( content.nodeType !== 11 ? content : content.firstChild, "tr" ) ) { + + return jQuery( elem ).children( "tbody" )[ 0 ] || elem; + } + + return elem; +} + +// Replace/restore the type attribute of script elements for safe DOM manipulation +function disableScript( elem ) { + elem.type = ( elem.getAttribute( "type" ) !== null ) + "/" + elem.type; + return elem; +} +function restoreScript( elem ) { + if ( ( elem.type || "" ).slice( 0, 5 ) === "true/" ) { + elem.type = elem.type.slice( 5 ); + } else { + elem.removeAttribute( "type" ); + } + + return elem; +} + +function cloneCopyEvent( src, dest ) { + var i, l, type, pdataOld, udataOld, udataCur, events; + + if ( dest.nodeType !== 1 ) { + return; + } + + // 1. Copy private data: events, handlers, etc. + if ( dataPriv.hasData( src ) ) { + pdataOld = dataPriv.get( src ); + events = pdataOld.events; + + if ( events ) { + dataPriv.remove( dest, "handle events" ); + + for ( type in events ) { + for ( i = 0, l = events[ type ].length; i < l; i++ ) { + jQuery.event.add( dest, type, events[ type ][ i ] ); + } + } + } + } + + // 2. Copy user data + if ( dataUser.hasData( src ) ) { + udataOld = dataUser.access( src ); + udataCur = jQuery.extend( {}, udataOld ); + + dataUser.set( dest, udataCur ); + } +} + +// Fix IE bugs, see support tests +function fixInput( src, dest ) { + var nodeName = dest.nodeName.toLowerCase(); + + // Fails to persist the checked state of a cloned checkbox or radio button. + if ( nodeName === "input" && rcheckableType.test( src.type ) ) { + dest.checked = src.checked; + + // Fails to return the selected option to the default selected state when cloning options + } else if ( nodeName === "input" || nodeName === "textarea" ) { + dest.defaultValue = src.defaultValue; + } +} + +function domManip( collection, args, callback, ignored ) { + + // Flatten any nested arrays + args = flat( args ); + + var fragment, first, scripts, hasScripts, node, doc, + i = 0, + l = collection.length, + iNoClone = l - 1, + value = args[ 0 ], + valueIsFunction = isFunction( value ); + + // We can't cloneNode fragments that contain checked, in WebKit + if ( valueIsFunction || + ( l > 1 && typeof value === "string" && + !support.checkClone && rchecked.test( value ) ) ) { + return collection.each( function( index ) { + var self = collection.eq( index ); + if ( valueIsFunction ) { + args[ 0 ] = value.call( this, index, self.html() ); + } + domManip( self, args, callback, ignored ); + } ); + } + + if ( l ) { + fragment = buildFragment( args, collection[ 0 ].ownerDocument, false, collection, ignored ); + first = fragment.firstChild; + + if ( fragment.childNodes.length === 1 ) { + fragment = first; + } + + // Require either new content or an interest in ignored elements to invoke the callback + if ( first || ignored ) { + scripts = jQuery.map( getAll( fragment, "script" ), disableScript ); + hasScripts = scripts.length; + + // Use the original fragment for the last item + // instead of the first because it can end up + // being emptied incorrectly in certain situations (#8070). + for ( ; i < l; i++ ) { + node = fragment; + + if ( i !== iNoClone ) { + node = jQuery.clone( node, true, true ); + + // Keep references to cloned scripts for later restoration + if ( hasScripts ) { + + // Support: Android <=4.0 only, PhantomJS 1 only + // push.apply(_, arraylike) throws on ancient WebKit + jQuery.merge( scripts, getAll( node, "script" ) ); + } + } + + callback.call( collection[ i ], node, i ); + } + + if ( hasScripts ) { + doc = scripts[ scripts.length - 1 ].ownerDocument; + + // Reenable scripts + jQuery.map( scripts, restoreScript ); + + // Evaluate executable scripts on first document insertion + for ( i = 0; i < hasScripts; i++ ) { + node = scripts[ i ]; + if ( rscriptType.test( node.type || "" ) && + !dataPriv.access( node, "globalEval" ) && + jQuery.contains( doc, node ) ) { + + if ( node.src && ( node.type || "" ).toLowerCase() !== "module" ) { + + // Optional AJAX dependency, but won't run scripts if not present + if ( jQuery._evalUrl && !node.noModule ) { + jQuery._evalUrl( node.src, { + nonce: node.nonce || node.getAttribute( "nonce" ) + }, doc ); + } + } else { + DOMEval( node.textContent.replace( rcleanScript, "" ), node, doc ); + } + } + } + } + } + } + + return collection; +} + +function remove( elem, selector, keepData ) { + var node, + nodes = selector ? jQuery.filter( selector, elem ) : elem, + i = 0; + + for ( ; ( node = nodes[ i ] ) != null; i++ ) { + if ( !keepData && node.nodeType === 1 ) { + jQuery.cleanData( getAll( node ) ); + } + + if ( node.parentNode ) { + if ( keepData && isAttached( node ) ) { + setGlobalEval( getAll( node, "script" ) ); + } + node.parentNode.removeChild( node ); + } + } + + return elem; +} + +jQuery.extend( { + htmlPrefilter: function( html ) { + return html; + }, + + clone: function( elem, dataAndEvents, deepDataAndEvents ) { + var i, l, srcElements, destElements, + clone = elem.cloneNode( true ), + inPage = isAttached( elem ); + + // Fix IE cloning issues + if ( !support.noCloneChecked && ( elem.nodeType === 1 || elem.nodeType === 11 ) && + !jQuery.isXMLDoc( elem ) ) { + + // We eschew Sizzle here for performance reasons: https://jsperf.com/getall-vs-sizzle/2 + destElements = getAll( clone ); + srcElements = getAll( elem ); + + for ( i = 0, l = srcElements.length; i < l; i++ ) { + fixInput( srcElements[ i ], destElements[ i ] ); + } + } + + // Copy the events from the original to the clone + if ( dataAndEvents ) { + if ( deepDataAndEvents ) { + srcElements = srcElements || getAll( elem ); + destElements = destElements || getAll( clone ); + + for ( i = 0, l = srcElements.length; i < l; i++ ) { + cloneCopyEvent( srcElements[ i ], destElements[ i ] ); + } + } else { + cloneCopyEvent( elem, clone ); + } + } + + // Preserve script evaluation history + destElements = getAll( clone, "script" ); + if ( destElements.length > 0 ) { + setGlobalEval( destElements, !inPage && getAll( elem, "script" ) ); + } + + // Return the cloned set + return clone; + }, + + cleanData: function( elems ) { + var data, elem, type, + special = jQuery.event.special, + i = 0; + + for ( ; ( elem = elems[ i ] ) !== undefined; i++ ) { + if ( acceptData( elem ) ) { + if ( ( data = elem[ dataPriv.expando ] ) ) { + if ( data.events ) { + for ( type in data.events ) { + if ( special[ type ] ) { + jQuery.event.remove( elem, type ); + + // This is a shortcut to avoid jQuery.event.remove's overhead + } else { + jQuery.removeEvent( elem, type, data.handle ); + } + } + } + + // Support: Chrome <=35 - 45+ + // Assign undefined instead of using delete, see Data#remove + elem[ dataPriv.expando ] = undefined; + } + if ( elem[ dataUser.expando ] ) { + + // Support: Chrome <=35 - 45+ + // Assign undefined instead of using delete, see Data#remove + elem[ dataUser.expando ] = undefined; + } + } + } + } +} ); + +jQuery.fn.extend( { + detach: function( selector ) { + return remove( this, selector, true ); + }, + + remove: function( selector ) { + return remove( this, selector ); + }, + + text: function( value ) { + return access( this, function( value ) { + return value === undefined ? + jQuery.text( this ) : + this.empty().each( function() { + if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) { + this.textContent = value; + } + } ); + }, null, value, arguments.length ); + }, + + append: function() { + return domManip( this, arguments, function( elem ) { + if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) { + var target = manipulationTarget( this, elem ); + target.appendChild( elem ); + } + } ); + }, + + prepend: function() { + return domManip( this, arguments, function( elem ) { + if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) { + var target = manipulationTarget( this, elem ); + target.insertBefore( elem, target.firstChild ); + } + } ); + }, + + before: function() { + return domManip( this, arguments, function( elem ) { + if ( this.parentNode ) { + this.parentNode.insertBefore( elem, this ); + } + } ); + }, + + after: function() { + return domManip( this, arguments, function( elem ) { + if ( this.parentNode ) { + this.parentNode.insertBefore( elem, this.nextSibling ); + } + } ); + }, + + empty: function() { + var elem, + i = 0; + + for ( ; ( elem = this[ i ] ) != null; i++ ) { + if ( elem.nodeType === 1 ) { + + // Prevent memory leaks + jQuery.cleanData( getAll( elem, false ) ); + + // Remove any remaining nodes + elem.textContent = ""; + } + } + + return this; + }, + + clone: function( dataAndEvents, deepDataAndEvents ) { + dataAndEvents = dataAndEvents == null ? false : dataAndEvents; + deepDataAndEvents = deepDataAndEvents == null ? dataAndEvents : deepDataAndEvents; + + return this.map( function() { + return jQuery.clone( this, dataAndEvents, deepDataAndEvents ); + } ); + }, + + html: function( value ) { + return access( this, function( value ) { + var elem = this[ 0 ] || {}, + i = 0, + l = this.length; + + if ( value === undefined && elem.nodeType === 1 ) { + return elem.innerHTML; + } + + // See if we can take a shortcut and just use innerHTML + if ( typeof value === "string" && !rnoInnerhtml.test( value ) && + !wrapMap[ ( rtagName.exec( value ) || [ "", "" ] )[ 1 ].toLowerCase() ] ) { + + value = jQuery.htmlPrefilter( value ); + + try { + for ( ; i < l; i++ ) { + elem = this[ i ] || {}; + + // Remove element nodes and prevent memory leaks + if ( elem.nodeType === 1 ) { + jQuery.cleanData( getAll( elem, false ) ); + elem.innerHTML = value; + } + } + + elem = 0; + + // If using innerHTML throws an exception, use the fallback method + } catch ( e ) {} + } + + if ( elem ) { + this.empty().append( value ); + } + }, null, value, arguments.length ); + }, + + replaceWith: function() { + var ignored = []; + + // Make the changes, replacing each non-ignored context element with the new content + return domManip( this, arguments, function( elem ) { + var parent = this.parentNode; + + if ( jQuery.inArray( this, ignored ) < 0 ) { + jQuery.cleanData( getAll( this ) ); + if ( parent ) { + parent.replaceChild( elem, this ); + } + } + + // Force callback invocation + }, ignored ); + } +} ); + +jQuery.each( { + appendTo: "append", + prependTo: "prepend", + insertBefore: "before", + insertAfter: "after", + replaceAll: "replaceWith" +}, function( name, original ) { + jQuery.fn[ name ] = function( selector ) { + var elems, + ret = [], + insert = jQuery( selector ), + last = insert.length - 1, + i = 0; + + for ( ; i <= last; i++ ) { + elems = i === last ? this : this.clone( true ); + jQuery( insert[ i ] )[ original ]( elems ); + + // Support: Android <=4.0 only, PhantomJS 1 only + // .get() because push.apply(_, arraylike) throws on ancient WebKit + push.apply( ret, elems.get() ); + } + + return this.pushStack( ret ); + }; +} ); +var rnumnonpx = new RegExp( "^(" + pnum + ")(?!px)[a-z%]+$", "i" ); + +var getStyles = function( elem ) { + + // Support: IE <=11 only, Firefox <=30 (#15098, #14150) + // IE throws on elements created in popups + // FF meanwhile throws on frame elements through "defaultView.getComputedStyle" + var view = elem.ownerDocument.defaultView; + + if ( !view || !view.opener ) { + view = window; + } + + return view.getComputedStyle( elem ); + }; + +var swap = function( elem, options, callback ) { + var ret, name, + old = {}; + + // Remember the old values, and insert the new ones + for ( name in options ) { + old[ name ] = elem.style[ name ]; + elem.style[ name ] = options[ name ]; + } + + ret = callback.call( elem ); + + // Revert the old values + for ( name in options ) { + elem.style[ name ] = old[ name ]; + } + + return ret; +}; + + +var rboxStyle = new RegExp( cssExpand.join( "|" ), "i" ); + + + +( function() { + + // Executing both pixelPosition & boxSizingReliable tests require only one layout + // so they're executed at the same time to save the second computation. + function computeStyleTests() { + + // This is a singleton, we need to execute it only once + if ( !div ) { + return; + } + + container.style.cssText = "position:absolute;left:-11111px;width:60px;" + + "margin-top:1px;padding:0;border:0"; + div.style.cssText = + "position:relative;display:block;box-sizing:border-box;overflow:scroll;" + + "margin:auto;border:1px;padding:1px;" + + "width:60%;top:1%"; + documentElement.appendChild( container ).appendChild( div ); + + var divStyle = window.getComputedStyle( div ); + pixelPositionVal = divStyle.top !== "1%"; + + // Support: Android 4.0 - 4.3 only, Firefox <=3 - 44 + reliableMarginLeftVal = roundPixelMeasures( divStyle.marginLeft ) === 12; + + // Support: Android 4.0 - 4.3 only, Safari <=9.1 - 10.1, iOS <=7.0 - 9.3 + // Some styles come back with percentage values, even though they shouldn't + div.style.right = "60%"; + pixelBoxStylesVal = roundPixelMeasures( divStyle.right ) === 36; + + // Support: IE 9 - 11 only + // Detect misreporting of content dimensions for box-sizing:border-box elements + boxSizingReliableVal = roundPixelMeasures( divStyle.width ) === 36; + + // Support: IE 9 only + // Detect overflow:scroll screwiness (gh-3699) + // Support: Chrome <=64 + // Don't get tricked when zoom affects offsetWidth (gh-4029) + div.style.position = "absolute"; + scrollboxSizeVal = roundPixelMeasures( div.offsetWidth / 3 ) === 12; + + documentElement.removeChild( container ); + + // Nullify the div so it wouldn't be stored in the memory and + // it will also be a sign that checks already performed + div = null; + } + + function roundPixelMeasures( measure ) { + return Math.round( parseFloat( measure ) ); + } + + var pixelPositionVal, boxSizingReliableVal, scrollboxSizeVal, pixelBoxStylesVal, + reliableTrDimensionsVal, reliableMarginLeftVal, + container = document.createElement( "div" ), + div = document.createElement( "div" ); + + // Finish early in limited (non-browser) environments + if ( !div.style ) { + return; + } + + // Support: IE <=9 - 11 only + // Style of cloned element affects source element cloned (#8908) + div.style.backgroundClip = "content-box"; + div.cloneNode( true ).style.backgroundClip = ""; + support.clearCloneStyle = div.style.backgroundClip === "content-box"; + + jQuery.extend( support, { + boxSizingReliable: function() { + computeStyleTests(); + return boxSizingReliableVal; + }, + pixelBoxStyles: function() { + computeStyleTests(); + return pixelBoxStylesVal; + }, + pixelPosition: function() { + computeStyleTests(); + return pixelPositionVal; + }, + reliableMarginLeft: function() { + computeStyleTests(); + return reliableMarginLeftVal; + }, + scrollboxSize: function() { + computeStyleTests(); + return scrollboxSizeVal; + }, + + // Support: IE 9 - 11+, Edge 15 - 18+ + // IE/Edge misreport `getComputedStyle` of table rows with width/height + // set in CSS while `offset*` properties report correct values. + // Behavior in IE 9 is more subtle than in newer versions & it passes + // some versions of this test; make sure not to make it pass there! + // + // Support: Firefox 70+ + // Only Firefox includes border widths + // in computed dimensions. (gh-4529) + reliableTrDimensions: function() { + var table, tr, trChild, trStyle; + if ( reliableTrDimensionsVal == null ) { + table = document.createElement( "table" ); + tr = document.createElement( "tr" ); + trChild = document.createElement( "div" ); + + table.style.cssText = "position:absolute;left:-11111px;border-collapse:separate"; + tr.style.cssText = "border:1px solid"; + + // Support: Chrome 86+ + // Height set through cssText does not get applied. + // Computed height then comes back as 0. + tr.style.height = "1px"; + trChild.style.height = "9px"; + + // Support: Android 8 Chrome 86+ + // In our bodyBackground.html iframe, + // display for all div elements is set to "inline", + // which causes a problem only in Android 8 Chrome 86. + // Ensuring the div is display: block + // gets around this issue. + trChild.style.display = "block"; + + documentElement + .appendChild( table ) + .appendChild( tr ) + .appendChild( trChild ); + + trStyle = window.getComputedStyle( tr ); + reliableTrDimensionsVal = ( parseInt( trStyle.height, 10 ) + + parseInt( trStyle.borderTopWidth, 10 ) + + parseInt( trStyle.borderBottomWidth, 10 ) ) === tr.offsetHeight; + + documentElement.removeChild( table ); + } + return reliableTrDimensionsVal; + } + } ); +} )(); + + +function curCSS( elem, name, computed ) { + var width, minWidth, maxWidth, ret, + + // Support: Firefox 51+ + // Retrieving style before computed somehow + // fixes an issue with getting wrong values + // on detached elements + style = elem.style; + + computed = computed || getStyles( elem ); + + // getPropertyValue is needed for: + // .css('filter') (IE 9 only, #12537) + // .css('--customProperty) (#3144) + if ( computed ) { + ret = computed.getPropertyValue( name ) || computed[ name ]; + + if ( ret === "" && !isAttached( elem ) ) { + ret = jQuery.style( elem, name ); + } + + // A tribute to the "awesome hack by Dean Edwards" + // Android Browser returns percentage for some values, + // but width seems to be reliably pixels. + // This is against the CSSOM draft spec: + // https://drafts.csswg.org/cssom/#resolved-values + if ( !support.pixelBoxStyles() && rnumnonpx.test( ret ) && rboxStyle.test( name ) ) { + + // Remember the original values + width = style.width; + minWidth = style.minWidth; + maxWidth = style.maxWidth; + + // Put in the new values to get a computed value out + style.minWidth = style.maxWidth = style.width = ret; + ret = computed.width; + + // Revert the changed values + style.width = width; + style.minWidth = minWidth; + style.maxWidth = maxWidth; + } + } + + return ret !== undefined ? + + // Support: IE <=9 - 11 only + // IE returns zIndex value as an integer. + ret + "" : + ret; +} + + +function addGetHookIf( conditionFn, hookFn ) { + + // Define the hook, we'll check on the first run if it's really needed. + return { + get: function() { + if ( conditionFn() ) { + + // Hook not needed (or it's not possible to use it due + // to missing dependency), remove it. + delete this.get; + return; + } + + // Hook needed; redefine it so that the support test is not executed again. + return ( this.get = hookFn ).apply( this, arguments ); + } + }; +} + + +var cssPrefixes = [ "Webkit", "Moz", "ms" ], + emptyStyle = document.createElement( "div" ).style, + vendorProps = {}; + +// Return a vendor-prefixed property or undefined +function vendorPropName( name ) { + + // Check for vendor prefixed names + var capName = name[ 0 ].toUpperCase() + name.slice( 1 ), + i = cssPrefixes.length; + + while ( i-- ) { + name = cssPrefixes[ i ] + capName; + if ( name in emptyStyle ) { + return name; + } + } +} + +// Return a potentially-mapped jQuery.cssProps or vendor prefixed property +function finalPropName( name ) { + var final = jQuery.cssProps[ name ] || vendorProps[ name ]; + + if ( final ) { + return final; + } + if ( name in emptyStyle ) { + return name; + } + return vendorProps[ name ] = vendorPropName( name ) || name; +} + + +var + + // Swappable if display is none or starts with table + // except "table", "table-cell", or "table-caption" + // See here for display values: https://developer.mozilla.org/en-US/docs/CSS/display + rdisplayswap = /^(none|table(?!-c[ea]).+)/, + rcustomProp = /^--/, + cssShow = { position: "absolute", visibility: "hidden", display: "block" }, + cssNormalTransform = { + letterSpacing: "0", + fontWeight: "400" + }; + +function setPositiveNumber( _elem, value, subtract ) { + + // Any relative (+/-) values have already been + // normalized at this point + var matches = rcssNum.exec( value ); + return matches ? + + // Guard against undefined "subtract", e.g., when used as in cssHooks + Math.max( 0, matches[ 2 ] - ( subtract || 0 ) ) + ( matches[ 3 ] || "px" ) : + value; +} + +function boxModelAdjustment( elem, dimension, box, isBorderBox, styles, computedVal ) { + var i = dimension === "width" ? 1 : 0, + extra = 0, + delta = 0; + + // Adjustment may not be necessary + if ( box === ( isBorderBox ? "border" : "content" ) ) { + return 0; + } + + for ( ; i < 4; i += 2 ) { + + // Both box models exclude margin + if ( box === "margin" ) { + delta += jQuery.css( elem, box + cssExpand[ i ], true, styles ); + } + + // If we get here with a content-box, we're seeking "padding" or "border" or "margin" + if ( !isBorderBox ) { + + // Add padding + delta += jQuery.css( elem, "padding" + cssExpand[ i ], true, styles ); + + // For "border" or "margin", add border + if ( box !== "padding" ) { + delta += jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles ); + + // But still keep track of it otherwise + } else { + extra += jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles ); + } + + // If we get here with a border-box (content + padding + border), we're seeking "content" or + // "padding" or "margin" + } else { + + // For "content", subtract padding + if ( box === "content" ) { + delta -= jQuery.css( elem, "padding" + cssExpand[ i ], true, styles ); + } + + // For "content" or "padding", subtract border + if ( box !== "margin" ) { + delta -= jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles ); + } + } + } + + // Account for positive content-box scroll gutter when requested by providing computedVal + if ( !isBorderBox && computedVal >= 0 ) { + + // offsetWidth/offsetHeight is a rounded sum of content, padding, scroll gutter, and border + // Assuming integer scroll gutter, subtract the rest and round down + delta += Math.max( 0, Math.ceil( + elem[ "offset" + dimension[ 0 ].toUpperCase() + dimension.slice( 1 ) ] - + computedVal - + delta - + extra - + 0.5 + + // If offsetWidth/offsetHeight is unknown, then we can't determine content-box scroll gutter + // Use an explicit zero to avoid NaN (gh-3964) + ) ) || 0; + } + + return delta; +} + +function getWidthOrHeight( elem, dimension, extra ) { + + // Start with computed style + var styles = getStyles( elem ), + + // To avoid forcing a reflow, only fetch boxSizing if we need it (gh-4322). + // Fake content-box until we know it's needed to know the true value. + boxSizingNeeded = !support.boxSizingReliable() || extra, + isBorderBox = boxSizingNeeded && + jQuery.css( elem, "boxSizing", false, styles ) === "border-box", + valueIsBorderBox = isBorderBox, + + val = curCSS( elem, dimension, styles ), + offsetProp = "offset" + dimension[ 0 ].toUpperCase() + dimension.slice( 1 ); + + // Support: Firefox <=54 + // Return a confounding non-pixel value or feign ignorance, as appropriate. + if ( rnumnonpx.test( val ) ) { + if ( !extra ) { + return val; + } + val = "auto"; + } + + + // Support: IE 9 - 11 only + // Use offsetWidth/offsetHeight for when box sizing is unreliable. + // In those cases, the computed value can be trusted to be border-box. + if ( ( !support.boxSizingReliable() && isBorderBox || + + // Support: IE 10 - 11+, Edge 15 - 18+ + // IE/Edge misreport `getComputedStyle` of table rows with width/height + // set in CSS while `offset*` properties report correct values. + // Interestingly, in some cases IE 9 doesn't suffer from this issue. + !support.reliableTrDimensions() && nodeName( elem, "tr" ) || + + // Fall back to offsetWidth/offsetHeight when value is "auto" + // This happens for inline elements with no explicit setting (gh-3571) + val === "auto" || + + // Support: Android <=4.1 - 4.3 only + // Also use offsetWidth/offsetHeight for misreported inline dimensions (gh-3602) + !parseFloat( val ) && jQuery.css( elem, "display", false, styles ) === "inline" ) && + + // Make sure the element is visible & connected + elem.getClientRects().length ) { + + isBorderBox = jQuery.css( elem, "boxSizing", false, styles ) === "border-box"; + + // Where available, offsetWidth/offsetHeight approximate border box dimensions. + // Where not available (e.g., SVG), assume unreliable box-sizing and interpret the + // retrieved value as a content box dimension. + valueIsBorderBox = offsetProp in elem; + if ( valueIsBorderBox ) { + val = elem[ offsetProp ]; + } + } + + // Normalize "" and auto + val = parseFloat( val ) || 0; + + // Adjust for the element's box model + return ( val + + boxModelAdjustment( + elem, + dimension, + extra || ( isBorderBox ? "border" : "content" ), + valueIsBorderBox, + styles, + + // Provide the current computed size to request scroll gutter calculation (gh-3589) + val + ) + ) + "px"; +} + +jQuery.extend( { + + // Add in style property hooks for overriding the default + // behavior of getting and setting a style property + cssHooks: { + opacity: { + get: function( elem, computed ) { + if ( computed ) { + + // We should always get a number back from opacity + var ret = curCSS( elem, "opacity" ); + return ret === "" ? "1" : ret; + } + } + } + }, + + // Don't automatically add "px" to these possibly-unitless properties + cssNumber: { + "animationIterationCount": true, + "columnCount": true, + "fillOpacity": true, + "flexGrow": true, + "flexShrink": true, + "fontWeight": true, + "gridArea": true, + "gridColumn": true, + "gridColumnEnd": true, + "gridColumnStart": true, + "gridRow": true, + "gridRowEnd": true, + "gridRowStart": true, + "lineHeight": true, + "opacity": true, + "order": true, + "orphans": true, + "widows": true, + "zIndex": true, + "zoom": true + }, + + // Add in properties whose names you wish to fix before + // setting or getting the value + cssProps: {}, + + // Get and set the style property on a DOM Node + style: function( elem, name, value, extra ) { + + // Don't set styles on text and comment nodes + if ( !elem || elem.nodeType === 3 || elem.nodeType === 8 || !elem.style ) { + return; + } + + // Make sure that we're working with the right name + var ret, type, hooks, + origName = camelCase( name ), + isCustomProp = rcustomProp.test( name ), + style = elem.style; + + // Make sure that we're working with the right name. We don't + // want to query the value if it is a CSS custom property + // since they are user-defined. + if ( !isCustomProp ) { + name = finalPropName( origName ); + } + + // Gets hook for the prefixed version, then unprefixed version + hooks = jQuery.cssHooks[ name ] || jQuery.cssHooks[ origName ]; + + // Check if we're setting a value + if ( value !== undefined ) { + type = typeof value; + + // Convert "+=" or "-=" to relative numbers (#7345) + if ( type === "string" && ( ret = rcssNum.exec( value ) ) && ret[ 1 ] ) { + value = adjustCSS( elem, name, ret ); + + // Fixes bug #9237 + type = "number"; + } + + // Make sure that null and NaN values aren't set (#7116) + if ( value == null || value !== value ) { + return; + } + + // If a number was passed in, add the unit (except for certain CSS properties) + // The isCustomProp check can be removed in jQuery 4.0 when we only auto-append + // "px" to a few hardcoded values. + if ( type === "number" && !isCustomProp ) { + value += ret && ret[ 3 ] || ( jQuery.cssNumber[ origName ] ? "" : "px" ); + } + + // background-* props affect original clone's values + if ( !support.clearCloneStyle && value === "" && name.indexOf( "background" ) === 0 ) { + style[ name ] = "inherit"; + } + + // If a hook was provided, use that value, otherwise just set the specified value + if ( !hooks || !( "set" in hooks ) || + ( value = hooks.set( elem, value, extra ) ) !== undefined ) { + + if ( isCustomProp ) { + style.setProperty( name, value ); + } else { + style[ name ] = value; + } + } + + } else { + + // If a hook was provided get the non-computed value from there + if ( hooks && "get" in hooks && + ( ret = hooks.get( elem, false, extra ) ) !== undefined ) { + + return ret; + } + + // Otherwise just get the value from the style object + return style[ name ]; + } + }, + + css: function( elem, name, extra, styles ) { + var val, num, hooks, + origName = camelCase( name ), + isCustomProp = rcustomProp.test( name ); + + // Make sure that we're working with the right name. We don't + // want to modify the value if it is a CSS custom property + // since they are user-defined. + if ( !isCustomProp ) { + name = finalPropName( origName ); + } + + // Try prefixed name followed by the unprefixed name + hooks = jQuery.cssHooks[ name ] || jQuery.cssHooks[ origName ]; + + // If a hook was provided get the computed value from there + if ( hooks && "get" in hooks ) { + val = hooks.get( elem, true, extra ); + } + + // Otherwise, if a way to get the computed value exists, use that + if ( val === undefined ) { + val = curCSS( elem, name, styles ); + } + + // Convert "normal" to computed value + if ( val === "normal" && name in cssNormalTransform ) { + val = cssNormalTransform[ name ]; + } + + // Make numeric if forced or a qualifier was provided and val looks numeric + if ( extra === "" || extra ) { + num = parseFloat( val ); + return extra === true || isFinite( num ) ? num || 0 : val; + } + + return val; + } +} ); + +jQuery.each( [ "height", "width" ], function( _i, dimension ) { + jQuery.cssHooks[ dimension ] = { + get: function( elem, computed, extra ) { + if ( computed ) { + + // Certain elements can have dimension info if we invisibly show them + // but it must have a current display style that would benefit + return rdisplayswap.test( jQuery.css( elem, "display" ) ) && + + // Support: Safari 8+ + // Table columns in Safari have non-zero offsetWidth & zero + // getBoundingClientRect().width unless display is changed. + // Support: IE <=11 only + // Running getBoundingClientRect on a disconnected node + // in IE throws an error. + ( !elem.getClientRects().length || !elem.getBoundingClientRect().width ) ? + swap( elem, cssShow, function() { + return getWidthOrHeight( elem, dimension, extra ); + } ) : + getWidthOrHeight( elem, dimension, extra ); + } + }, + + set: function( elem, value, extra ) { + var matches, + styles = getStyles( elem ), + + // Only read styles.position if the test has a chance to fail + // to avoid forcing a reflow. + scrollboxSizeBuggy = !support.scrollboxSize() && + styles.position === "absolute", + + // To avoid forcing a reflow, only fetch boxSizing if we need it (gh-3991) + boxSizingNeeded = scrollboxSizeBuggy || extra, + isBorderBox = boxSizingNeeded && + jQuery.css( elem, "boxSizing", false, styles ) === "border-box", + subtract = extra ? + boxModelAdjustment( + elem, + dimension, + extra, + isBorderBox, + styles + ) : + 0; + + // Account for unreliable border-box dimensions by comparing offset* to computed and + // faking a content-box to get border and padding (gh-3699) + if ( isBorderBox && scrollboxSizeBuggy ) { + subtract -= Math.ceil( + elem[ "offset" + dimension[ 0 ].toUpperCase() + dimension.slice( 1 ) ] - + parseFloat( styles[ dimension ] ) - + boxModelAdjustment( elem, dimension, "border", false, styles ) - + 0.5 + ); + } + + // Convert to pixels if value adjustment is needed + if ( subtract && ( matches = rcssNum.exec( value ) ) && + ( matches[ 3 ] || "px" ) !== "px" ) { + + elem.style[ dimension ] = value; + value = jQuery.css( elem, dimension ); + } + + return setPositiveNumber( elem, value, subtract ); + } + }; +} ); + +jQuery.cssHooks.marginLeft = addGetHookIf( support.reliableMarginLeft, + function( elem, computed ) { + if ( computed ) { + return ( parseFloat( curCSS( elem, "marginLeft" ) ) || + elem.getBoundingClientRect().left - + swap( elem, { marginLeft: 0 }, function() { + return elem.getBoundingClientRect().left; + } ) + ) + "px"; + } + } +); + +// These hooks are used by animate to expand properties +jQuery.each( { + margin: "", + padding: "", + border: "Width" +}, function( prefix, suffix ) { + jQuery.cssHooks[ prefix + suffix ] = { + expand: function( value ) { + var i = 0, + expanded = {}, + + // Assumes a single number if not a string + parts = typeof value === "string" ? value.split( " " ) : [ value ]; + + for ( ; i < 4; i++ ) { + expanded[ prefix + cssExpand[ i ] + suffix ] = + parts[ i ] || parts[ i - 2 ] || parts[ 0 ]; + } + + return expanded; + } + }; + + if ( prefix !== "margin" ) { + jQuery.cssHooks[ prefix + suffix ].set = setPositiveNumber; + } +} ); + +jQuery.fn.extend( { + css: function( name, value ) { + return access( this, function( elem, name, value ) { + var styles, len, + map = {}, + i = 0; + + if ( Array.isArray( name ) ) { + styles = getStyles( elem ); + len = name.length; + + for ( ; i < len; i++ ) { + map[ name[ i ] ] = jQuery.css( elem, name[ i ], false, styles ); + } + + return map; + } + + return value !== undefined ? + jQuery.style( elem, name, value ) : + jQuery.css( elem, name ); + }, name, value, arguments.length > 1 ); + } +} ); + + +function Tween( elem, options, prop, end, easing ) { + return new Tween.prototype.init( elem, options, prop, end, easing ); +} +jQuery.Tween = Tween; + +Tween.prototype = { + constructor: Tween, + init: function( elem, options, prop, end, easing, unit ) { + this.elem = elem; + this.prop = prop; + this.easing = easing || jQuery.easing._default; + this.options = options; + this.start = this.now = this.cur(); + this.end = end; + this.unit = unit || ( jQuery.cssNumber[ prop ] ? "" : "px" ); + }, + cur: function() { + var hooks = Tween.propHooks[ this.prop ]; + + return hooks && hooks.get ? + hooks.get( this ) : + Tween.propHooks._default.get( this ); + }, + run: function( percent ) { + var eased, + hooks = Tween.propHooks[ this.prop ]; + + if ( this.options.duration ) { + this.pos = eased = jQuery.easing[ this.easing ]( + percent, this.options.duration * percent, 0, 1, this.options.duration + ); + } else { + this.pos = eased = percent; + } + this.now = ( this.end - this.start ) * eased + this.start; + + if ( this.options.step ) { + this.options.step.call( this.elem, this.now, this ); + } + + if ( hooks && hooks.set ) { + hooks.set( this ); + } else { + Tween.propHooks._default.set( this ); + } + return this; + } +}; + +Tween.prototype.init.prototype = Tween.prototype; + +Tween.propHooks = { + _default: { + get: function( tween ) { + var result; + + // Use a property on the element directly when it is not a DOM element, + // or when there is no matching style property that exists. + if ( tween.elem.nodeType !== 1 || + tween.elem[ tween.prop ] != null && tween.elem.style[ tween.prop ] == null ) { + return tween.elem[ tween.prop ]; + } + + // Passing an empty string as a 3rd parameter to .css will automatically + // attempt a parseFloat and fallback to a string if the parse fails. + // Simple values such as "10px" are parsed to Float; + // complex values such as "rotate(1rad)" are returned as-is. + result = jQuery.css( tween.elem, tween.prop, "" ); + + // Empty strings, null, undefined and "auto" are converted to 0. + return !result || result === "auto" ? 0 : result; + }, + set: function( tween ) { + + // Use step hook for back compat. + // Use cssHook if its there. + // Use .style if available and use plain properties where available. + if ( jQuery.fx.step[ tween.prop ] ) { + jQuery.fx.step[ tween.prop ]( tween ); + } else if ( tween.elem.nodeType === 1 && ( + jQuery.cssHooks[ tween.prop ] || + tween.elem.style[ finalPropName( tween.prop ) ] != null ) ) { + jQuery.style( tween.elem, tween.prop, tween.now + tween.unit ); + } else { + tween.elem[ tween.prop ] = tween.now; + } + } + } +}; + +// Support: IE <=9 only +// Panic based approach to setting things on disconnected nodes +Tween.propHooks.scrollTop = Tween.propHooks.scrollLeft = { + set: function( tween ) { + if ( tween.elem.nodeType && tween.elem.parentNode ) { + tween.elem[ tween.prop ] = tween.now; + } + } +}; + +jQuery.easing = { + linear: function( p ) { + return p; + }, + swing: function( p ) { + return 0.5 - Math.cos( p * Math.PI ) / 2; + }, + _default: "swing" +}; + +jQuery.fx = Tween.prototype.init; + +// Back compat <1.8 extension point +jQuery.fx.step = {}; + + + + +var + fxNow, inProgress, + rfxtypes = /^(?:toggle|show|hide)$/, + rrun = /queueHooks$/; + +function schedule() { + if ( inProgress ) { + if ( document.hidden === false && window.requestAnimationFrame ) { + window.requestAnimationFrame( schedule ); + } else { + window.setTimeout( schedule, jQuery.fx.interval ); + } + + jQuery.fx.tick(); + } +} + +// Animations created synchronously will run synchronously +function createFxNow() { + window.setTimeout( function() { + fxNow = undefined; + } ); + return ( fxNow = Date.now() ); +} + +// Generate parameters to create a standard animation +function genFx( type, includeWidth ) { + var which, + i = 0, + attrs = { height: type }; + + // If we include width, step value is 1 to do all cssExpand values, + // otherwise step value is 2 to skip over Left and Right + includeWidth = includeWidth ? 1 : 0; + for ( ; i < 4; i += 2 - includeWidth ) { + which = cssExpand[ i ]; + attrs[ "margin" + which ] = attrs[ "padding" + which ] = type; + } + + if ( includeWidth ) { + attrs.opacity = attrs.width = type; + } + + return attrs; +} + +function createTween( value, prop, animation ) { + var tween, + collection = ( Animation.tweeners[ prop ] || [] ).concat( Animation.tweeners[ "*" ] ), + index = 0, + length = collection.length; + for ( ; index < length; index++ ) { + if ( ( tween = collection[ index ].call( animation, prop, value ) ) ) { + + // We're done with this property + return tween; + } + } +} + +function defaultPrefilter( elem, props, opts ) { + var prop, value, toggle, hooks, oldfire, propTween, restoreDisplay, display, + isBox = "width" in props || "height" in props, + anim = this, + orig = {}, + style = elem.style, + hidden = elem.nodeType && isHiddenWithinTree( elem ), + dataShow = dataPriv.get( elem, "fxshow" ); + + // Queue-skipping animations hijack the fx hooks + if ( !opts.queue ) { + hooks = jQuery._queueHooks( elem, "fx" ); + if ( hooks.unqueued == null ) { + hooks.unqueued = 0; + oldfire = hooks.empty.fire; + hooks.empty.fire = function() { + if ( !hooks.unqueued ) { + oldfire(); + } + }; + } + hooks.unqueued++; + + anim.always( function() { + + // Ensure the complete handler is called before this completes + anim.always( function() { + hooks.unqueued--; + if ( !jQuery.queue( elem, "fx" ).length ) { + hooks.empty.fire(); + } + } ); + } ); + } + + // Detect show/hide animations + for ( prop in props ) { + value = props[ prop ]; + if ( rfxtypes.test( value ) ) { + delete props[ prop ]; + toggle = toggle || value === "toggle"; + if ( value === ( hidden ? "hide" : "show" ) ) { + + // Pretend to be hidden if this is a "show" and + // there is still data from a stopped show/hide + if ( value === "show" && dataShow && dataShow[ prop ] !== undefined ) { + hidden = true; + + // Ignore all other no-op show/hide data + } else { + continue; + } + } + orig[ prop ] = dataShow && dataShow[ prop ] || jQuery.style( elem, prop ); + } + } + + // Bail out if this is a no-op like .hide().hide() + propTween = !jQuery.isEmptyObject( props ); + if ( !propTween && jQuery.isEmptyObject( orig ) ) { + return; + } + + // Restrict "overflow" and "display" styles during box animations + if ( isBox && elem.nodeType === 1 ) { + + // Support: IE <=9 - 11, Edge 12 - 15 + // Record all 3 overflow attributes because IE does not infer the shorthand + // from identically-valued overflowX and overflowY and Edge just mirrors + // the overflowX value there. + opts.overflow = [ style.overflow, style.overflowX, style.overflowY ]; + + // Identify a display type, preferring old show/hide data over the CSS cascade + restoreDisplay = dataShow && dataShow.display; + if ( restoreDisplay == null ) { + restoreDisplay = dataPriv.get( elem, "display" ); + } + display = jQuery.css( elem, "display" ); + if ( display === "none" ) { + if ( restoreDisplay ) { + display = restoreDisplay; + } else { + + // Get nonempty value(s) by temporarily forcing visibility + showHide( [ elem ], true ); + restoreDisplay = elem.style.display || restoreDisplay; + display = jQuery.css( elem, "display" ); + showHide( [ elem ] ); + } + } + + // Animate inline elements as inline-block + if ( display === "inline" || display === "inline-block" && restoreDisplay != null ) { + if ( jQuery.css( elem, "float" ) === "none" ) { + + // Restore the original display value at the end of pure show/hide animations + if ( !propTween ) { + anim.done( function() { + style.display = restoreDisplay; + } ); + if ( restoreDisplay == null ) { + display = style.display; + restoreDisplay = display === "none" ? "" : display; + } + } + style.display = "inline-block"; + } + } + } + + if ( opts.overflow ) { + style.overflow = "hidden"; + anim.always( function() { + style.overflow = opts.overflow[ 0 ]; + style.overflowX = opts.overflow[ 1 ]; + style.overflowY = opts.overflow[ 2 ]; + } ); + } + + // Implement show/hide animations + propTween = false; + for ( prop in orig ) { + + // General show/hide setup for this element animation + if ( !propTween ) { + if ( dataShow ) { + if ( "hidden" in dataShow ) { + hidden = dataShow.hidden; + } + } else { + dataShow = dataPriv.access( elem, "fxshow", { display: restoreDisplay } ); + } + + // Store hidden/visible for toggle so `.stop().toggle()` "reverses" + if ( toggle ) { + dataShow.hidden = !hidden; + } + + // Show elements before animating them + if ( hidden ) { + showHide( [ elem ], true ); + } + + /* eslint-disable no-loop-func */ + + anim.done( function() { + + /* eslint-enable no-loop-func */ + + // The final step of a "hide" animation is actually hiding the element + if ( !hidden ) { + showHide( [ elem ] ); + } + dataPriv.remove( elem, "fxshow" ); + for ( prop in orig ) { + jQuery.style( elem, prop, orig[ prop ] ); + } + } ); + } + + // Per-property setup + propTween = createTween( hidden ? dataShow[ prop ] : 0, prop, anim ); + if ( !( prop in dataShow ) ) { + dataShow[ prop ] = propTween.start; + if ( hidden ) { + propTween.end = propTween.start; + propTween.start = 0; + } + } + } +} + +function propFilter( props, specialEasing ) { + var index, name, easing, value, hooks; + + // camelCase, specialEasing and expand cssHook pass + for ( index in props ) { + name = camelCase( index ); + easing = specialEasing[ name ]; + value = props[ index ]; + if ( Array.isArray( value ) ) { + easing = value[ 1 ]; + value = props[ index ] = value[ 0 ]; + } + + if ( index !== name ) { + props[ name ] = value; + delete props[ index ]; + } + + hooks = jQuery.cssHooks[ name ]; + if ( hooks && "expand" in hooks ) { + value = hooks.expand( value ); + delete props[ name ]; + + // Not quite $.extend, this won't overwrite existing keys. + // Reusing 'index' because we have the correct "name" + for ( index in value ) { + if ( !( index in props ) ) { + props[ index ] = value[ index ]; + specialEasing[ index ] = easing; + } + } + } else { + specialEasing[ name ] = easing; + } + } +} + +function Animation( elem, properties, options ) { + var result, + stopped, + index = 0, + length = Animation.prefilters.length, + deferred = jQuery.Deferred().always( function() { + + // Don't match elem in the :animated selector + delete tick.elem; + } ), + tick = function() { + if ( stopped ) { + return false; + } + var currentTime = fxNow || createFxNow(), + remaining = Math.max( 0, animation.startTime + animation.duration - currentTime ), + + // Support: Android 2.3 only + // Archaic crash bug won't allow us to use `1 - ( 0.5 || 0 )` (#12497) + temp = remaining / animation.duration || 0, + percent = 1 - temp, + index = 0, + length = animation.tweens.length; + + for ( ; index < length; index++ ) { + animation.tweens[ index ].run( percent ); + } + + deferred.notifyWith( elem, [ animation, percent, remaining ] ); + + // If there's more to do, yield + if ( percent < 1 && length ) { + return remaining; + } + + // If this was an empty animation, synthesize a final progress notification + if ( !length ) { + deferred.notifyWith( elem, [ animation, 1, 0 ] ); + } + + // Resolve the animation and report its conclusion + deferred.resolveWith( elem, [ animation ] ); + return false; + }, + animation = deferred.promise( { + elem: elem, + props: jQuery.extend( {}, properties ), + opts: jQuery.extend( true, { + specialEasing: {}, + easing: jQuery.easing._default + }, options ), + originalProperties: properties, + originalOptions: options, + startTime: fxNow || createFxNow(), + duration: options.duration, + tweens: [], + createTween: function( prop, end ) { + var tween = jQuery.Tween( elem, animation.opts, prop, end, + animation.opts.specialEasing[ prop ] || animation.opts.easing ); + animation.tweens.push( tween ); + return tween; + }, + stop: function( gotoEnd ) { + var index = 0, + + // If we are going to the end, we want to run all the tweens + // otherwise we skip this part + length = gotoEnd ? animation.tweens.length : 0; + if ( stopped ) { + return this; + } + stopped = true; + for ( ; index < length; index++ ) { + animation.tweens[ index ].run( 1 ); + } + + // Resolve when we played the last frame; otherwise, reject + if ( gotoEnd ) { + deferred.notifyWith( elem, [ animation, 1, 0 ] ); + deferred.resolveWith( elem, [ animation, gotoEnd ] ); + } else { + deferred.rejectWith( elem, [ animation, gotoEnd ] ); + } + return this; + } + } ), + props = animation.props; + + propFilter( props, animation.opts.specialEasing ); + + for ( ; index < length; index++ ) { + result = Animation.prefilters[ index ].call( animation, elem, props, animation.opts ); + if ( result ) { + if ( isFunction( result.stop ) ) { + jQuery._queueHooks( animation.elem, animation.opts.queue ).stop = + result.stop.bind( result ); + } + return result; + } + } + + jQuery.map( props, createTween, animation ); + + if ( isFunction( animation.opts.start ) ) { + animation.opts.start.call( elem, animation ); + } + + // Attach callbacks from options + animation + .progress( animation.opts.progress ) + .done( animation.opts.done, animation.opts.complete ) + .fail( animation.opts.fail ) + .always( animation.opts.always ); + + jQuery.fx.timer( + jQuery.extend( tick, { + elem: elem, + anim: animation, + queue: animation.opts.queue + } ) + ); + + return animation; +} + +jQuery.Animation = jQuery.extend( Animation, { + + tweeners: { + "*": [ function( prop, value ) { + var tween = this.createTween( prop, value ); + adjustCSS( tween.elem, prop, rcssNum.exec( value ), tween ); + return tween; + } ] + }, + + tweener: function( props, callback ) { + if ( isFunction( props ) ) { + callback = props; + props = [ "*" ]; + } else { + props = props.match( rnothtmlwhite ); + } + + var prop, + index = 0, + length = props.length; + + for ( ; index < length; index++ ) { + prop = props[ index ]; + Animation.tweeners[ prop ] = Animation.tweeners[ prop ] || []; + Animation.tweeners[ prop ].unshift( callback ); + } + }, + + prefilters: [ defaultPrefilter ], + + prefilter: function( callback, prepend ) { + if ( prepend ) { + Animation.prefilters.unshift( callback ); + } else { + Animation.prefilters.push( callback ); + } + } +} ); + +jQuery.speed = function( speed, easing, fn ) { + var opt = speed && typeof speed === "object" ? jQuery.extend( {}, speed ) : { + complete: fn || !fn && easing || + isFunction( speed ) && speed, + duration: speed, + easing: fn && easing || easing && !isFunction( easing ) && easing + }; + + // Go to the end state if fx are off + if ( jQuery.fx.off ) { + opt.duration = 0; + + } else { + if ( typeof opt.duration !== "number" ) { + if ( opt.duration in jQuery.fx.speeds ) { + opt.duration = jQuery.fx.speeds[ opt.duration ]; + + } else { + opt.duration = jQuery.fx.speeds._default; + } + } + } + + // Normalize opt.queue - true/undefined/null -> "fx" + if ( opt.queue == null || opt.queue === true ) { + opt.queue = "fx"; + } + + // Queueing + opt.old = opt.complete; + + opt.complete = function() { + if ( isFunction( opt.old ) ) { + opt.old.call( this ); + } + + if ( opt.queue ) { + jQuery.dequeue( this, opt.queue ); + } + }; + + return opt; +}; + +jQuery.fn.extend( { + fadeTo: function( speed, to, easing, callback ) { + + // Show any hidden elements after setting opacity to 0 + return this.filter( isHiddenWithinTree ).css( "opacity", 0 ).show() + + // Animate to the value specified + .end().animate( { opacity: to }, speed, easing, callback ); + }, + animate: function( prop, speed, easing, callback ) { + var empty = jQuery.isEmptyObject( prop ), + optall = jQuery.speed( speed, easing, callback ), + doAnimation = function() { + + // Operate on a copy of prop so per-property easing won't be lost + var anim = Animation( this, jQuery.extend( {}, prop ), optall ); + + // Empty animations, or finishing resolves immediately + if ( empty || dataPriv.get( this, "finish" ) ) { + anim.stop( true ); + } + }; + + doAnimation.finish = doAnimation; + + return empty || optall.queue === false ? + this.each( doAnimation ) : + this.queue( optall.queue, doAnimation ); + }, + stop: function( type, clearQueue, gotoEnd ) { + var stopQueue = function( hooks ) { + var stop = hooks.stop; + delete hooks.stop; + stop( gotoEnd ); + }; + + if ( typeof type !== "string" ) { + gotoEnd = clearQueue; + clearQueue = type; + type = undefined; + } + if ( clearQueue ) { + this.queue( type || "fx", [] ); + } + + return this.each( function() { + var dequeue = true, + index = type != null && type + "queueHooks", + timers = jQuery.timers, + data = dataPriv.get( this ); + + if ( index ) { + if ( data[ index ] && data[ index ].stop ) { + stopQueue( data[ index ] ); + } + } else { + for ( index in data ) { + if ( data[ index ] && data[ index ].stop && rrun.test( index ) ) { + stopQueue( data[ index ] ); + } + } + } + + for ( index = timers.length; index--; ) { + if ( timers[ index ].elem === this && + ( type == null || timers[ index ].queue === type ) ) { + + timers[ index ].anim.stop( gotoEnd ); + dequeue = false; + timers.splice( index, 1 ); + } + } + + // Start the next in the queue if the last step wasn't forced. + // Timers currently will call their complete callbacks, which + // will dequeue but only if they were gotoEnd. + if ( dequeue || !gotoEnd ) { + jQuery.dequeue( this, type ); + } + } ); + }, + finish: function( type ) { + if ( type !== false ) { + type = type || "fx"; + } + return this.each( function() { + var index, + data = dataPriv.get( this ), + queue = data[ type + "queue" ], + hooks = data[ type + "queueHooks" ], + timers = jQuery.timers, + length = queue ? queue.length : 0; + + // Enable finishing flag on private data + data.finish = true; + + // Empty the queue first + jQuery.queue( this, type, [] ); + + if ( hooks && hooks.stop ) { + hooks.stop.call( this, true ); + } + + // Look for any active animations, and finish them + for ( index = timers.length; index--; ) { + if ( timers[ index ].elem === this && timers[ index ].queue === type ) { + timers[ index ].anim.stop( true ); + timers.splice( index, 1 ); + } + } + + // Look for any animations in the old queue and finish them + for ( index = 0; index < length; index++ ) { + if ( queue[ index ] && queue[ index ].finish ) { + queue[ index ].finish.call( this ); + } + } + + // Turn off finishing flag + delete data.finish; + } ); + } +} ); + +jQuery.each( [ "toggle", "show", "hide" ], function( _i, name ) { + var cssFn = jQuery.fn[ name ]; + jQuery.fn[ name ] = function( speed, easing, callback ) { + return speed == null || typeof speed === "boolean" ? + cssFn.apply( this, arguments ) : + this.animate( genFx( name, true ), speed, easing, callback ); + }; +} ); + +// Generate shortcuts for custom animations +jQuery.each( { + slideDown: genFx( "show" ), + slideUp: genFx( "hide" ), + slideToggle: genFx( "toggle" ), + fadeIn: { opacity: "show" }, + fadeOut: { opacity: "hide" }, + fadeToggle: { opacity: "toggle" } +}, function( name, props ) { + jQuery.fn[ name ] = function( speed, easing, callback ) { + return this.animate( props, speed, easing, callback ); + }; +} ); + +jQuery.timers = []; +jQuery.fx.tick = function() { + var timer, + i = 0, + timers = jQuery.timers; + + fxNow = Date.now(); + + for ( ; i < timers.length; i++ ) { + timer = timers[ i ]; + + // Run the timer and safely remove it when done (allowing for external removal) + if ( !timer() && timers[ i ] === timer ) { + timers.splice( i--, 1 ); + } + } + + if ( !timers.length ) { + jQuery.fx.stop(); + } + fxNow = undefined; +}; + +jQuery.fx.timer = function( timer ) { + jQuery.timers.push( timer ); + jQuery.fx.start(); +}; + +jQuery.fx.interval = 13; +jQuery.fx.start = function() { + if ( inProgress ) { + return; + } + + inProgress = true; + schedule(); +}; + +jQuery.fx.stop = function() { + inProgress = null; +}; + +jQuery.fx.speeds = { + slow: 600, + fast: 200, + + // Default speed + _default: 400 +}; + + +// Based off of the plugin by Clint Helfers, with permission. +// https://web.archive.org/web/20100324014747/http://blindsignals.com/index.php/2009/07/jquery-delay/ +jQuery.fn.delay = function( time, type ) { + time = jQuery.fx ? jQuery.fx.speeds[ time ] || time : time; + type = type || "fx"; + + return this.queue( type, function( next, hooks ) { + var timeout = window.setTimeout( next, time ); + hooks.stop = function() { + window.clearTimeout( timeout ); + }; + } ); +}; + + +( function() { + var input = document.createElement( "input" ), + select = document.createElement( "select" ), + opt = select.appendChild( document.createElement( "option" ) ); + + input.type = "checkbox"; + + // Support: Android <=4.3 only + // Default value for a checkbox should be "on" + support.checkOn = input.value !== ""; + + // Support: IE <=11 only + // Must access selectedIndex to make default options select + support.optSelected = opt.selected; + + // Support: IE <=11 only + // An input loses its value after becoming a radio + input = document.createElement( "input" ); + input.value = "t"; + input.type = "radio"; + support.radioValue = input.value === "t"; +} )(); + + +var boolHook, + attrHandle = jQuery.expr.attrHandle; + +jQuery.fn.extend( { + attr: function( name, value ) { + return access( this, jQuery.attr, name, value, arguments.length > 1 ); + }, + + removeAttr: function( name ) { + return this.each( function() { + jQuery.removeAttr( this, name ); + } ); + } +} ); + +jQuery.extend( { + attr: function( elem, name, value ) { + var ret, hooks, + nType = elem.nodeType; + + // Don't get/set attributes on text, comment and attribute nodes + if ( nType === 3 || nType === 8 || nType === 2 ) { + return; + } + + // Fallback to prop when attributes are not supported + if ( typeof elem.getAttribute === "undefined" ) { + return jQuery.prop( elem, name, value ); + } + + // Attribute hooks are determined by the lowercase version + // Grab necessary hook if one is defined + if ( nType !== 1 || !jQuery.isXMLDoc( elem ) ) { + hooks = jQuery.attrHooks[ name.toLowerCase() ] || + ( jQuery.expr.match.bool.test( name ) ? boolHook : undefined ); + } + + if ( value !== undefined ) { + if ( value === null ) { + jQuery.removeAttr( elem, name ); + return; + } + + if ( hooks && "set" in hooks && + ( ret = hooks.set( elem, value, name ) ) !== undefined ) { + return ret; + } + + elem.setAttribute( name, value + "" ); + return value; + } + + if ( hooks && "get" in hooks && ( ret = hooks.get( elem, name ) ) !== null ) { + return ret; + } + + ret = jQuery.find.attr( elem, name ); + + // Non-existent attributes return null, we normalize to undefined + return ret == null ? undefined : ret; + }, + + attrHooks: { + type: { + set: function( elem, value ) { + if ( !support.radioValue && value === "radio" && + nodeName( elem, "input" ) ) { + var val = elem.value; + elem.setAttribute( "type", value ); + if ( val ) { + elem.value = val; + } + return value; + } + } + } + }, + + removeAttr: function( elem, value ) { + var name, + i = 0, + + // Attribute names can contain non-HTML whitespace characters + // https://html.spec.whatwg.org/multipage/syntax.html#attributes-2 + attrNames = value && value.match( rnothtmlwhite ); + + if ( attrNames && elem.nodeType === 1 ) { + while ( ( name = attrNames[ i++ ] ) ) { + elem.removeAttribute( name ); + } + } + } +} ); + +// Hooks for boolean attributes +boolHook = { + set: function( elem, value, name ) { + if ( value === false ) { + + // Remove boolean attributes when set to false + jQuery.removeAttr( elem, name ); + } else { + elem.setAttribute( name, name ); + } + return name; + } +}; + +jQuery.each( jQuery.expr.match.bool.source.match( /\w+/g ), function( _i, name ) { + var getter = attrHandle[ name ] || jQuery.find.attr; + + attrHandle[ name ] = function( elem, name, isXML ) { + var ret, handle, + lowercaseName = name.toLowerCase(); + + if ( !isXML ) { + + // Avoid an infinite loop by temporarily removing this function from the getter + handle = attrHandle[ lowercaseName ]; + attrHandle[ lowercaseName ] = ret; + ret = getter( elem, name, isXML ) != null ? + lowercaseName : + null; + attrHandle[ lowercaseName ] = handle; + } + return ret; + }; +} ); + + + + +var rfocusable = /^(?:input|select|textarea|button)$/i, + rclickable = /^(?:a|area)$/i; + +jQuery.fn.extend( { + prop: function( name, value ) { + return access( this, jQuery.prop, name, value, arguments.length > 1 ); + }, + + removeProp: function( name ) { + return this.each( function() { + delete this[ jQuery.propFix[ name ] || name ]; + } ); + } +} ); + +jQuery.extend( { + prop: function( elem, name, value ) { + var ret, hooks, + nType = elem.nodeType; + + // Don't get/set properties on text, comment and attribute nodes + if ( nType === 3 || nType === 8 || nType === 2 ) { + return; + } + + if ( nType !== 1 || !jQuery.isXMLDoc( elem ) ) { + + // Fix name and attach hooks + name = jQuery.propFix[ name ] || name; + hooks = jQuery.propHooks[ name ]; + } + + if ( value !== undefined ) { + if ( hooks && "set" in hooks && + ( ret = hooks.set( elem, value, name ) ) !== undefined ) { + return ret; + } + + return ( elem[ name ] = value ); + } + + if ( hooks && "get" in hooks && ( ret = hooks.get( elem, name ) ) !== null ) { + return ret; + } + + return elem[ name ]; + }, + + propHooks: { + tabIndex: { + get: function( elem ) { + + // Support: IE <=9 - 11 only + // elem.tabIndex doesn't always return the + // correct value when it hasn't been explicitly set + // https://web.archive.org/web/20141116233347/http://fluidproject.org/blog/2008/01/09/getting-setting-and-removing-tabindex-values-with-javascript/ + // Use proper attribute retrieval(#12072) + var tabindex = jQuery.find.attr( elem, "tabindex" ); + + if ( tabindex ) { + return parseInt( tabindex, 10 ); + } + + if ( + rfocusable.test( elem.nodeName ) || + rclickable.test( elem.nodeName ) && + elem.href + ) { + return 0; + } + + return -1; + } + } + }, + + propFix: { + "for": "htmlFor", + "class": "className" + } +} ); + +// Support: IE <=11 only +// Accessing the selectedIndex property +// forces the browser to respect setting selected +// on the option +// The getter ensures a default option is selected +// when in an optgroup +// eslint rule "no-unused-expressions" is disabled for this code +// since it considers such accessions noop +if ( !support.optSelected ) { + jQuery.propHooks.selected = { + get: function( elem ) { + + /* eslint no-unused-expressions: "off" */ + + var parent = elem.parentNode; + if ( parent && parent.parentNode ) { + parent.parentNode.selectedIndex; + } + return null; + }, + set: function( elem ) { + + /* eslint no-unused-expressions: "off" */ + + var parent = elem.parentNode; + if ( parent ) { + parent.selectedIndex; + + if ( parent.parentNode ) { + parent.parentNode.selectedIndex; + } + } + } + }; +} + +jQuery.each( [ + "tabIndex", + "readOnly", + "maxLength", + "cellSpacing", + "cellPadding", + "rowSpan", + "colSpan", + "useMap", + "frameBorder", + "contentEditable" +], function() { + jQuery.propFix[ this.toLowerCase() ] = this; +} ); + + + + + // Strip and collapse whitespace according to HTML spec + // https://infra.spec.whatwg.org/#strip-and-collapse-ascii-whitespace + function stripAndCollapse( value ) { + var tokens = value.match( rnothtmlwhite ) || []; + return tokens.join( " " ); + } + + +function getClass( elem ) { + return elem.getAttribute && elem.getAttribute( "class" ) || ""; +} + +function classesToArray( value ) { + if ( Array.isArray( value ) ) { + return value; + } + if ( typeof value === "string" ) { + return value.match( rnothtmlwhite ) || []; + } + return []; +} + +jQuery.fn.extend( { + addClass: function( value ) { + var classes, elem, cur, curValue, clazz, j, finalValue, + i = 0; + + if ( isFunction( value ) ) { + return this.each( function( j ) { + jQuery( this ).addClass( value.call( this, j, getClass( this ) ) ); + } ); + } + + classes = classesToArray( value ); + + if ( classes.length ) { + while ( ( elem = this[ i++ ] ) ) { + curValue = getClass( elem ); + cur = elem.nodeType === 1 && ( " " + stripAndCollapse( curValue ) + " " ); + + if ( cur ) { + j = 0; + while ( ( clazz = classes[ j++ ] ) ) { + if ( cur.indexOf( " " + clazz + " " ) < 0 ) { + cur += clazz + " "; + } + } + + // Only assign if different to avoid unneeded rendering. + finalValue = stripAndCollapse( cur ); + if ( curValue !== finalValue ) { + elem.setAttribute( "class", finalValue ); + } + } + } + } + + return this; + }, + + removeClass: function( value ) { + var classes, elem, cur, curValue, clazz, j, finalValue, + i = 0; + + if ( isFunction( value ) ) { + return this.each( function( j ) { + jQuery( this ).removeClass( value.call( this, j, getClass( this ) ) ); + } ); + } + + if ( !arguments.length ) { + return this.attr( "class", "" ); + } + + classes = classesToArray( value ); + + if ( classes.length ) { + while ( ( elem = this[ i++ ] ) ) { + curValue = getClass( elem ); + + // This expression is here for better compressibility (see addClass) + cur = elem.nodeType === 1 && ( " " + stripAndCollapse( curValue ) + " " ); + + if ( cur ) { + j = 0; + while ( ( clazz = classes[ j++ ] ) ) { + + // Remove *all* instances + while ( cur.indexOf( " " + clazz + " " ) > -1 ) { + cur = cur.replace( " " + clazz + " ", " " ); + } + } + + // Only assign if different to avoid unneeded rendering. + finalValue = stripAndCollapse( cur ); + if ( curValue !== finalValue ) { + elem.setAttribute( "class", finalValue ); + } + } + } + } + + return this; + }, + + toggleClass: function( value, stateVal ) { + var type = typeof value, + isValidValue = type === "string" || Array.isArray( value ); + + if ( typeof stateVal === "boolean" && isValidValue ) { + return stateVal ? this.addClass( value ) : this.removeClass( value ); + } + + if ( isFunction( value ) ) { + return this.each( function( i ) { + jQuery( this ).toggleClass( + value.call( this, i, getClass( this ), stateVal ), + stateVal + ); + } ); + } + + return this.each( function() { + var className, i, self, classNames; + + if ( isValidValue ) { + + // Toggle individual class names + i = 0; + self = jQuery( this ); + classNames = classesToArray( value ); + + while ( ( className = classNames[ i++ ] ) ) { + + // Check each className given, space separated list + if ( self.hasClass( className ) ) { + self.removeClass( className ); + } else { + self.addClass( className ); + } + } + + // Toggle whole class name + } else if ( value === undefined || type === "boolean" ) { + className = getClass( this ); + if ( className ) { + + // Store className if set + dataPriv.set( this, "__className__", className ); + } + + // If the element has a class name or if we're passed `false`, + // then remove the whole classname (if there was one, the above saved it). + // Otherwise bring back whatever was previously saved (if anything), + // falling back to the empty string if nothing was stored. + if ( this.setAttribute ) { + this.setAttribute( "class", + className || value === false ? + "" : + dataPriv.get( this, "__className__" ) || "" + ); + } + } + } ); + }, + + hasClass: function( selector ) { + var className, elem, + i = 0; + + className = " " + selector + " "; + while ( ( elem = this[ i++ ] ) ) { + if ( elem.nodeType === 1 && + ( " " + stripAndCollapse( getClass( elem ) ) + " " ).indexOf( className ) > -1 ) { + return true; + } + } + + return false; + } +} ); + + + + +var rreturn = /\r/g; + +jQuery.fn.extend( { + val: function( value ) { + var hooks, ret, valueIsFunction, + elem = this[ 0 ]; + + if ( !arguments.length ) { + if ( elem ) { + hooks = jQuery.valHooks[ elem.type ] || + jQuery.valHooks[ elem.nodeName.toLowerCase() ]; + + if ( hooks && + "get" in hooks && + ( ret = hooks.get( elem, "value" ) ) !== undefined + ) { + return ret; + } + + ret = elem.value; + + // Handle most common string cases + if ( typeof ret === "string" ) { + return ret.replace( rreturn, "" ); + } + + // Handle cases where value is null/undef or number + return ret == null ? "" : ret; + } + + return; + } + + valueIsFunction = isFunction( value ); + + return this.each( function( i ) { + var val; + + if ( this.nodeType !== 1 ) { + return; + } + + if ( valueIsFunction ) { + val = value.call( this, i, jQuery( this ).val() ); + } else { + val = value; + } + + // Treat null/undefined as ""; convert numbers to string + if ( val == null ) { + val = ""; + + } else if ( typeof val === "number" ) { + val += ""; + + } else if ( Array.isArray( val ) ) { + val = jQuery.map( val, function( value ) { + return value == null ? "" : value + ""; + } ); + } + + hooks = jQuery.valHooks[ this.type ] || jQuery.valHooks[ this.nodeName.toLowerCase() ]; + + // If set returns undefined, fall back to normal setting + if ( !hooks || !( "set" in hooks ) || hooks.set( this, val, "value" ) === undefined ) { + this.value = val; + } + } ); + } +} ); + +jQuery.extend( { + valHooks: { + option: { + get: function( elem ) { + + var val = jQuery.find.attr( elem, "value" ); + return val != null ? + val : + + // Support: IE <=10 - 11 only + // option.text throws exceptions (#14686, #14858) + // Strip and collapse whitespace + // https://html.spec.whatwg.org/#strip-and-collapse-whitespace + stripAndCollapse( jQuery.text( elem ) ); + } + }, + select: { + get: function( elem ) { + var value, option, i, + options = elem.options, + index = elem.selectedIndex, + one = elem.type === "select-one", + values = one ? null : [], + max = one ? index + 1 : options.length; + + if ( index < 0 ) { + i = max; + + } else { + i = one ? index : 0; + } + + // Loop through all the selected options + for ( ; i < max; i++ ) { + option = options[ i ]; + + // Support: IE <=9 only + // IE8-9 doesn't update selected after form reset (#2551) + if ( ( option.selected || i === index ) && + + // Don't return options that are disabled or in a disabled optgroup + !option.disabled && + ( !option.parentNode.disabled || + !nodeName( option.parentNode, "optgroup" ) ) ) { + + // Get the specific value for the option + value = jQuery( option ).val(); + + // We don't need an array for one selects + if ( one ) { + return value; + } + + // Multi-Selects return an array + values.push( value ); + } + } + + return values; + }, + + set: function( elem, value ) { + var optionSet, option, + options = elem.options, + values = jQuery.makeArray( value ), + i = options.length; + + while ( i-- ) { + option = options[ i ]; + + /* eslint-disable no-cond-assign */ + + if ( option.selected = + jQuery.inArray( jQuery.valHooks.option.get( option ), values ) > -1 + ) { + optionSet = true; + } + + /* eslint-enable no-cond-assign */ + } + + // Force browsers to behave consistently when non-matching value is set + if ( !optionSet ) { + elem.selectedIndex = -1; + } + return values; + } + } + } +} ); + +// Radios and checkboxes getter/setter +jQuery.each( [ "radio", "checkbox" ], function() { + jQuery.valHooks[ this ] = { + set: function( elem, value ) { + if ( Array.isArray( value ) ) { + return ( elem.checked = jQuery.inArray( jQuery( elem ).val(), value ) > -1 ); + } + } + }; + if ( !support.checkOn ) { + jQuery.valHooks[ this ].get = function( elem ) { + return elem.getAttribute( "value" ) === null ? "on" : elem.value; + }; + } +} ); + + + + +// Return jQuery for attributes-only inclusion + + +support.focusin = "onfocusin" in window; + + +var rfocusMorph = /^(?:focusinfocus|focusoutblur)$/, + stopPropagationCallback = function( e ) { + e.stopPropagation(); + }; + +jQuery.extend( jQuery.event, { + + trigger: function( event, data, elem, onlyHandlers ) { + + var i, cur, tmp, bubbleType, ontype, handle, special, lastElement, + eventPath = [ elem || document ], + type = hasOwn.call( event, "type" ) ? event.type : event, + namespaces = hasOwn.call( event, "namespace" ) ? event.namespace.split( "." ) : []; + + cur = lastElement = tmp = elem = elem || document; + + // Don't do events on text and comment nodes + if ( elem.nodeType === 3 || elem.nodeType === 8 ) { + return; + } + + // focus/blur morphs to focusin/out; ensure we're not firing them right now + if ( rfocusMorph.test( type + jQuery.event.triggered ) ) { + return; + } + + if ( type.indexOf( "." ) > -1 ) { + + // Namespaced trigger; create a regexp to match event type in handle() + namespaces = type.split( "." ); + type = namespaces.shift(); + namespaces.sort(); + } + ontype = type.indexOf( ":" ) < 0 && "on" + type; + + // Caller can pass in a jQuery.Event object, Object, or just an event type string + event = event[ jQuery.expando ] ? + event : + new jQuery.Event( type, typeof event === "object" && event ); + + // Trigger bitmask: & 1 for native handlers; & 2 for jQuery (always true) + event.isTrigger = onlyHandlers ? 2 : 3; + event.namespace = namespaces.join( "." ); + event.rnamespace = event.namespace ? + new RegExp( "(^|\\.)" + namespaces.join( "\\.(?:.*\\.|)" ) + "(\\.|$)" ) : + null; + + // Clean up the event in case it is being reused + event.result = undefined; + if ( !event.target ) { + event.target = elem; + } + + // Clone any incoming data and prepend the event, creating the handler arg list + data = data == null ? + [ event ] : + jQuery.makeArray( data, [ event ] ); + + // Allow special events to draw outside the lines + special = jQuery.event.special[ type ] || {}; + if ( !onlyHandlers && special.trigger && special.trigger.apply( elem, data ) === false ) { + return; + } + + // Determine event propagation path in advance, per W3C events spec (#9951) + // Bubble up to document, then to window; watch for a global ownerDocument var (#9724) + if ( !onlyHandlers && !special.noBubble && !isWindow( elem ) ) { + + bubbleType = special.delegateType || type; + if ( !rfocusMorph.test( bubbleType + type ) ) { + cur = cur.parentNode; + } + for ( ; cur; cur = cur.parentNode ) { + eventPath.push( cur ); + tmp = cur; + } + + // Only add window if we got to document (e.g., not plain obj or detached DOM) + if ( tmp === ( elem.ownerDocument || document ) ) { + eventPath.push( tmp.defaultView || tmp.parentWindow || window ); + } + } + + // Fire handlers on the event path + i = 0; + while ( ( cur = eventPath[ i++ ] ) && !event.isPropagationStopped() ) { + lastElement = cur; + event.type = i > 1 ? + bubbleType : + special.bindType || type; + + // jQuery handler + handle = ( dataPriv.get( cur, "events" ) || Object.create( null ) )[ event.type ] && + dataPriv.get( cur, "handle" ); + if ( handle ) { + handle.apply( cur, data ); + } + + // Native handler + handle = ontype && cur[ ontype ]; + if ( handle && handle.apply && acceptData( cur ) ) { + event.result = handle.apply( cur, data ); + if ( event.result === false ) { + event.preventDefault(); + } + } + } + event.type = type; + + // If nobody prevented the default action, do it now + if ( !onlyHandlers && !event.isDefaultPrevented() ) { + + if ( ( !special._default || + special._default.apply( eventPath.pop(), data ) === false ) && + acceptData( elem ) ) { + + // Call a native DOM method on the target with the same name as the event. + // Don't do default actions on window, that's where global variables be (#6170) + if ( ontype && isFunction( elem[ type ] ) && !isWindow( elem ) ) { + + // Don't re-trigger an onFOO event when we call its FOO() method + tmp = elem[ ontype ]; + + if ( tmp ) { + elem[ ontype ] = null; + } + + // Prevent re-triggering of the same event, since we already bubbled it above + jQuery.event.triggered = type; + + if ( event.isPropagationStopped() ) { + lastElement.addEventListener( type, stopPropagationCallback ); + } + + elem[ type ](); + + if ( event.isPropagationStopped() ) { + lastElement.removeEventListener( type, stopPropagationCallback ); + } + + jQuery.event.triggered = undefined; + + if ( tmp ) { + elem[ ontype ] = tmp; + } + } + } + } + + return event.result; + }, + + // Piggyback on a donor event to simulate a different one + // Used only for `focus(in | out)` events + simulate: function( type, elem, event ) { + var e = jQuery.extend( + new jQuery.Event(), + event, + { + type: type, + isSimulated: true + } + ); + + jQuery.event.trigger( e, null, elem ); + } + +} ); + +jQuery.fn.extend( { + + trigger: function( type, data ) { + return this.each( function() { + jQuery.event.trigger( type, data, this ); + } ); + }, + triggerHandler: function( type, data ) { + var elem = this[ 0 ]; + if ( elem ) { + return jQuery.event.trigger( type, data, elem, true ); + } + } +} ); + + +// Support: Firefox <=44 +// Firefox doesn't have focus(in | out) events +// Related ticket - https://bugzilla.mozilla.org/show_bug.cgi?id=687787 +// +// Support: Chrome <=48 - 49, Safari <=9.0 - 9.1 +// focus(in | out) events fire after focus & blur events, +// which is spec violation - http://www.w3.org/TR/DOM-Level-3-Events/#events-focusevent-event-order +// Related ticket - https://bugs.chromium.org/p/chromium/issues/detail?id=449857 +if ( !support.focusin ) { + jQuery.each( { focus: "focusin", blur: "focusout" }, function( orig, fix ) { + + // Attach a single capturing handler on the document while someone wants focusin/focusout + var handler = function( event ) { + jQuery.event.simulate( fix, event.target, jQuery.event.fix( event ) ); + }; + + jQuery.event.special[ fix ] = { + setup: function() { + + // Handle: regular nodes (via `this.ownerDocument`), window + // (via `this.document`) & document (via `this`). + var doc = this.ownerDocument || this.document || this, + attaches = dataPriv.access( doc, fix ); + + if ( !attaches ) { + doc.addEventListener( orig, handler, true ); + } + dataPriv.access( doc, fix, ( attaches || 0 ) + 1 ); + }, + teardown: function() { + var doc = this.ownerDocument || this.document || this, + attaches = dataPriv.access( doc, fix ) - 1; + + if ( !attaches ) { + doc.removeEventListener( orig, handler, true ); + dataPriv.remove( doc, fix ); + + } else { + dataPriv.access( doc, fix, attaches ); + } + } + }; + } ); +} +var location = window.location; + +var nonce = { guid: Date.now() }; + +var rquery = ( /\?/ ); + + + +// Cross-browser xml parsing +jQuery.parseXML = function( data ) { + var xml, parserErrorElem; + if ( !data || typeof data !== "string" ) { + return null; + } + + // Support: IE 9 - 11 only + // IE throws on parseFromString with invalid input. + try { + xml = ( new window.DOMParser() ).parseFromString( data, "text/xml" ); + } catch ( e ) {} + + parserErrorElem = xml && xml.getElementsByTagName( "parsererror" )[ 0 ]; + if ( !xml || parserErrorElem ) { + jQuery.error( "Invalid XML: " + ( + parserErrorElem ? + jQuery.map( parserErrorElem.childNodes, function( el ) { + return el.textContent; + } ).join( "\n" ) : + data + ) ); + } + return xml; +}; + + +var + rbracket = /\[\]$/, + rCRLF = /\r?\n/g, + rsubmitterTypes = /^(?:submit|button|image|reset|file)$/i, + rsubmittable = /^(?:input|select|textarea|keygen)/i; + +function buildParams( prefix, obj, traditional, add ) { + var name; + + if ( Array.isArray( obj ) ) { + + // Serialize array item. + jQuery.each( obj, function( i, v ) { + if ( traditional || rbracket.test( prefix ) ) { + + // Treat each array item as a scalar. + add( prefix, v ); + + } else { + + // Item is non-scalar (array or object), encode its numeric index. + buildParams( + prefix + "[" + ( typeof v === "object" && v != null ? i : "" ) + "]", + v, + traditional, + add + ); + } + } ); + + } else if ( !traditional && toType( obj ) === "object" ) { + + // Serialize object item. + for ( name in obj ) { + buildParams( prefix + "[" + name + "]", obj[ name ], traditional, add ); + } + + } else { + + // Serialize scalar item. + add( prefix, obj ); + } +} + +// Serialize an array of form elements or a set of +// key/values into a query string +jQuery.param = function( a, traditional ) { + var prefix, + s = [], + add = function( key, valueOrFunction ) { + + // If value is a function, invoke it and use its return value + var value = isFunction( valueOrFunction ) ? + valueOrFunction() : + valueOrFunction; + + s[ s.length ] = encodeURIComponent( key ) + "=" + + encodeURIComponent( value == null ? "" : value ); + }; + + if ( a == null ) { + return ""; + } + + // If an array was passed in, assume that it is an array of form elements. + if ( Array.isArray( a ) || ( a.jquery && !jQuery.isPlainObject( a ) ) ) { + + // Serialize the form elements + jQuery.each( a, function() { + add( this.name, this.value ); + } ); + + } else { + + // If traditional, encode the "old" way (the way 1.3.2 or older + // did it), otherwise encode params recursively. + for ( prefix in a ) { + buildParams( prefix, a[ prefix ], traditional, add ); + } + } + + // Return the resulting serialization + return s.join( "&" ); +}; + +jQuery.fn.extend( { + serialize: function() { + return jQuery.param( this.serializeArray() ); + }, + serializeArray: function() { + return this.map( function() { + + // Can add propHook for "elements" to filter or add form elements + var elements = jQuery.prop( this, "elements" ); + return elements ? jQuery.makeArray( elements ) : this; + } ).filter( function() { + var type = this.type; + + // Use .is( ":disabled" ) so that fieldset[disabled] works + return this.name && !jQuery( this ).is( ":disabled" ) && + rsubmittable.test( this.nodeName ) && !rsubmitterTypes.test( type ) && + ( this.checked || !rcheckableType.test( type ) ); + } ).map( function( _i, elem ) { + var val = jQuery( this ).val(); + + if ( val == null ) { + return null; + } + + if ( Array.isArray( val ) ) { + return jQuery.map( val, function( val ) { + return { name: elem.name, value: val.replace( rCRLF, "\r\n" ) }; + } ); + } + + return { name: elem.name, value: val.replace( rCRLF, "\r\n" ) }; + } ).get(); + } +} ); + + +var + r20 = /%20/g, + rhash = /#.*$/, + rantiCache = /([?&])_=[^&]*/, + rheaders = /^(.*?):[ \t]*([^\r\n]*)$/mg, + + // #7653, #8125, #8152: local protocol detection + rlocalProtocol = /^(?:about|app|app-storage|.+-extension|file|res|widget):$/, + rnoContent = /^(?:GET|HEAD)$/, + rprotocol = /^\/\//, + + /* Prefilters + * 1) They are useful to introduce custom dataTypes (see ajax/jsonp.js for an example) + * 2) These are called: + * - BEFORE asking for a transport + * - AFTER param serialization (s.data is a string if s.processData is true) + * 3) key is the dataType + * 4) the catchall symbol "*" can be used + * 5) execution will start with transport dataType and THEN continue down to "*" if needed + */ + prefilters = {}, + + /* Transports bindings + * 1) key is the dataType + * 2) the catchall symbol "*" can be used + * 3) selection will start with transport dataType and THEN go to "*" if needed + */ + transports = {}, + + // Avoid comment-prolog char sequence (#10098); must appease lint and evade compression + allTypes = "*/".concat( "*" ), + + // Anchor tag for parsing the document origin + originAnchor = document.createElement( "a" ); + +originAnchor.href = location.href; + +// Base "constructor" for jQuery.ajaxPrefilter and jQuery.ajaxTransport +function addToPrefiltersOrTransports( structure ) { + + // dataTypeExpression is optional and defaults to "*" + return function( dataTypeExpression, func ) { + + if ( typeof dataTypeExpression !== "string" ) { + func = dataTypeExpression; + dataTypeExpression = "*"; + } + + var dataType, + i = 0, + dataTypes = dataTypeExpression.toLowerCase().match( rnothtmlwhite ) || []; + + if ( isFunction( func ) ) { + + // For each dataType in the dataTypeExpression + while ( ( dataType = dataTypes[ i++ ] ) ) { + + // Prepend if requested + if ( dataType[ 0 ] === "+" ) { + dataType = dataType.slice( 1 ) || "*"; + ( structure[ dataType ] = structure[ dataType ] || [] ).unshift( func ); + + // Otherwise append + } else { + ( structure[ dataType ] = structure[ dataType ] || [] ).push( func ); + } + } + } + }; +} + +// Base inspection function for prefilters and transports +function inspectPrefiltersOrTransports( structure, options, originalOptions, jqXHR ) { + + var inspected = {}, + seekingTransport = ( structure === transports ); + + function inspect( dataType ) { + var selected; + inspected[ dataType ] = true; + jQuery.each( structure[ dataType ] || [], function( _, prefilterOrFactory ) { + var dataTypeOrTransport = prefilterOrFactory( options, originalOptions, jqXHR ); + if ( typeof dataTypeOrTransport === "string" && + !seekingTransport && !inspected[ dataTypeOrTransport ] ) { + + options.dataTypes.unshift( dataTypeOrTransport ); + inspect( dataTypeOrTransport ); + return false; + } else if ( seekingTransport ) { + return !( selected = dataTypeOrTransport ); + } + } ); + return selected; + } + + return inspect( options.dataTypes[ 0 ] ) || !inspected[ "*" ] && inspect( "*" ); +} + +// A special extend for ajax options +// that takes "flat" options (not to be deep extended) +// Fixes #9887 +function ajaxExtend( target, src ) { + var key, deep, + flatOptions = jQuery.ajaxSettings.flatOptions || {}; + + for ( key in src ) { + if ( src[ key ] !== undefined ) { + ( flatOptions[ key ] ? target : ( deep || ( deep = {} ) ) )[ key ] = src[ key ]; + } + } + if ( deep ) { + jQuery.extend( true, target, deep ); + } + + return target; +} + +/* Handles responses to an ajax request: + * - finds the right dataType (mediates between content-type and expected dataType) + * - returns the corresponding response + */ +function ajaxHandleResponses( s, jqXHR, responses ) { + + var ct, type, finalDataType, firstDataType, + contents = s.contents, + dataTypes = s.dataTypes; + + // Remove auto dataType and get content-type in the process + while ( dataTypes[ 0 ] === "*" ) { + dataTypes.shift(); + if ( ct === undefined ) { + ct = s.mimeType || jqXHR.getResponseHeader( "Content-Type" ); + } + } + + // Check if we're dealing with a known content-type + if ( ct ) { + for ( type in contents ) { + if ( contents[ type ] && contents[ type ].test( ct ) ) { + dataTypes.unshift( type ); + break; + } + } + } + + // Check to see if we have a response for the expected dataType + if ( dataTypes[ 0 ] in responses ) { + finalDataType = dataTypes[ 0 ]; + } else { + + // Try convertible dataTypes + for ( type in responses ) { + if ( !dataTypes[ 0 ] || s.converters[ type + " " + dataTypes[ 0 ] ] ) { + finalDataType = type; + break; + } + if ( !firstDataType ) { + firstDataType = type; + } + } + + // Or just use first one + finalDataType = finalDataType || firstDataType; + } + + // If we found a dataType + // We add the dataType to the list if needed + // and return the corresponding response + if ( finalDataType ) { + if ( finalDataType !== dataTypes[ 0 ] ) { + dataTypes.unshift( finalDataType ); + } + return responses[ finalDataType ]; + } +} + +/* Chain conversions given the request and the original response + * Also sets the responseXXX fields on the jqXHR instance + */ +function ajaxConvert( s, response, jqXHR, isSuccess ) { + var conv2, current, conv, tmp, prev, + converters = {}, + + // Work with a copy of dataTypes in case we need to modify it for conversion + dataTypes = s.dataTypes.slice(); + + // Create converters map with lowercased keys + if ( dataTypes[ 1 ] ) { + for ( conv in s.converters ) { + converters[ conv.toLowerCase() ] = s.converters[ conv ]; + } + } + + current = dataTypes.shift(); + + // Convert to each sequential dataType + while ( current ) { + + if ( s.responseFields[ current ] ) { + jqXHR[ s.responseFields[ current ] ] = response; + } + + // Apply the dataFilter if provided + if ( !prev && isSuccess && s.dataFilter ) { + response = s.dataFilter( response, s.dataType ); + } + + prev = current; + current = dataTypes.shift(); + + if ( current ) { + + // There's only work to do if current dataType is non-auto + if ( current === "*" ) { + + current = prev; + + // Convert response if prev dataType is non-auto and differs from current + } else if ( prev !== "*" && prev !== current ) { + + // Seek a direct converter + conv = converters[ prev + " " + current ] || converters[ "* " + current ]; + + // If none found, seek a pair + if ( !conv ) { + for ( conv2 in converters ) { + + // If conv2 outputs current + tmp = conv2.split( " " ); + if ( tmp[ 1 ] === current ) { + + // If prev can be converted to accepted input + conv = converters[ prev + " " + tmp[ 0 ] ] || + converters[ "* " + tmp[ 0 ] ]; + if ( conv ) { + + // Condense equivalence converters + if ( conv === true ) { + conv = converters[ conv2 ]; + + // Otherwise, insert the intermediate dataType + } else if ( converters[ conv2 ] !== true ) { + current = tmp[ 0 ]; + dataTypes.unshift( tmp[ 1 ] ); + } + break; + } + } + } + } + + // Apply converter (if not an equivalence) + if ( conv !== true ) { + + // Unless errors are allowed to bubble, catch and return them + if ( conv && s.throws ) { + response = conv( response ); + } else { + try { + response = conv( response ); + } catch ( e ) { + return { + state: "parsererror", + error: conv ? e : "No conversion from " + prev + " to " + current + }; + } + } + } + } + } + } + + return { state: "success", data: response }; +} + +jQuery.extend( { + + // Counter for holding the number of active queries + active: 0, + + // Last-Modified header cache for next request + lastModified: {}, + etag: {}, + + ajaxSettings: { + url: location.href, + type: "GET", + isLocal: rlocalProtocol.test( location.protocol ), + global: true, + processData: true, + async: true, + contentType: "application/x-www-form-urlencoded; charset=UTF-8", + + /* + timeout: 0, + data: null, + dataType: null, + username: null, + password: null, + cache: null, + throws: false, + traditional: false, + headers: {}, + */ + + accepts: { + "*": allTypes, + text: "text/plain", + html: "text/html", + xml: "application/xml, text/xml", + json: "application/json, text/javascript" + }, + + contents: { + xml: /\bxml\b/, + html: /\bhtml/, + json: /\bjson\b/ + }, + + responseFields: { + xml: "responseXML", + text: "responseText", + json: "responseJSON" + }, + + // Data converters + // Keys separate source (or catchall "*") and destination types with a single space + converters: { + + // Convert anything to text + "* text": String, + + // Text to html (true = no transformation) + "text html": true, + + // Evaluate text as a json expression + "text json": JSON.parse, + + // Parse text as xml + "text xml": jQuery.parseXML + }, + + // For options that shouldn't be deep extended: + // you can add your own custom options here if + // and when you create one that shouldn't be + // deep extended (see ajaxExtend) + flatOptions: { + url: true, + context: true + } + }, + + // Creates a full fledged settings object into target + // with both ajaxSettings and settings fields. + // If target is omitted, writes into ajaxSettings. + ajaxSetup: function( target, settings ) { + return settings ? + + // Building a settings object + ajaxExtend( ajaxExtend( target, jQuery.ajaxSettings ), settings ) : + + // Extending ajaxSettings + ajaxExtend( jQuery.ajaxSettings, target ); + }, + + ajaxPrefilter: addToPrefiltersOrTransports( prefilters ), + ajaxTransport: addToPrefiltersOrTransports( transports ), + + // Main method + ajax: function( url, options ) { + + // If url is an object, simulate pre-1.5 signature + if ( typeof url === "object" ) { + options = url; + url = undefined; + } + + // Force options to be an object + options = options || {}; + + var transport, + + // URL without anti-cache param + cacheURL, + + // Response headers + responseHeadersString, + responseHeaders, + + // timeout handle + timeoutTimer, + + // Url cleanup var + urlAnchor, + + // Request state (becomes false upon send and true upon completion) + completed, + + // To know if global events are to be dispatched + fireGlobals, + + // Loop variable + i, + + // uncached part of the url + uncached, + + // Create the final options object + s = jQuery.ajaxSetup( {}, options ), + + // Callbacks context + callbackContext = s.context || s, + + // Context for global events is callbackContext if it is a DOM node or jQuery collection + globalEventContext = s.context && + ( callbackContext.nodeType || callbackContext.jquery ) ? + jQuery( callbackContext ) : + jQuery.event, + + // Deferreds + deferred = jQuery.Deferred(), + completeDeferred = jQuery.Callbacks( "once memory" ), + + // Status-dependent callbacks + statusCode = s.statusCode || {}, + + // Headers (they are sent all at once) + requestHeaders = {}, + requestHeadersNames = {}, + + // Default abort message + strAbort = "canceled", + + // Fake xhr + jqXHR = { + readyState: 0, + + // Builds headers hashtable if needed + getResponseHeader: function( key ) { + var match; + if ( completed ) { + if ( !responseHeaders ) { + responseHeaders = {}; + while ( ( match = rheaders.exec( responseHeadersString ) ) ) { + responseHeaders[ match[ 1 ].toLowerCase() + " " ] = + ( responseHeaders[ match[ 1 ].toLowerCase() + " " ] || [] ) + .concat( match[ 2 ] ); + } + } + match = responseHeaders[ key.toLowerCase() + " " ]; + } + return match == null ? null : match.join( ", " ); + }, + + // Raw string + getAllResponseHeaders: function() { + return completed ? responseHeadersString : null; + }, + + // Caches the header + setRequestHeader: function( name, value ) { + if ( completed == null ) { + name = requestHeadersNames[ name.toLowerCase() ] = + requestHeadersNames[ name.toLowerCase() ] || name; + requestHeaders[ name ] = value; + } + return this; + }, + + // Overrides response content-type header + overrideMimeType: function( type ) { + if ( completed == null ) { + s.mimeType = type; + } + return this; + }, + + // Status-dependent callbacks + statusCode: function( map ) { + var code; + if ( map ) { + if ( completed ) { + + // Execute the appropriate callbacks + jqXHR.always( map[ jqXHR.status ] ); + } else { + + // Lazy-add the new callbacks in a way that preserves old ones + for ( code in map ) { + statusCode[ code ] = [ statusCode[ code ], map[ code ] ]; + } + } + } + return this; + }, + + // Cancel the request + abort: function( statusText ) { + var finalText = statusText || strAbort; + if ( transport ) { + transport.abort( finalText ); + } + done( 0, finalText ); + return this; + } + }; + + // Attach deferreds + deferred.promise( jqXHR ); + + // Add protocol if not provided (prefilters might expect it) + // Handle falsy url in the settings object (#10093: consistency with old signature) + // We also use the url parameter if available + s.url = ( ( url || s.url || location.href ) + "" ) + .replace( rprotocol, location.protocol + "//" ); + + // Alias method option to type as per ticket #12004 + s.type = options.method || options.type || s.method || s.type; + + // Extract dataTypes list + s.dataTypes = ( s.dataType || "*" ).toLowerCase().match( rnothtmlwhite ) || [ "" ]; + + // A cross-domain request is in order when the origin doesn't match the current origin. + if ( s.crossDomain == null ) { + urlAnchor = document.createElement( "a" ); + + // Support: IE <=8 - 11, Edge 12 - 15 + // IE throws exception on accessing the href property if url is malformed, + // e.g. http://example.com:80x/ + try { + urlAnchor.href = s.url; + + // Support: IE <=8 - 11 only + // Anchor's host property isn't correctly set when s.url is relative + urlAnchor.href = urlAnchor.href; + s.crossDomain = originAnchor.protocol + "//" + originAnchor.host !== + urlAnchor.protocol + "//" + urlAnchor.host; + } catch ( e ) { + + // If there is an error parsing the URL, assume it is crossDomain, + // it can be rejected by the transport if it is invalid + s.crossDomain = true; + } + } + + // Convert data if not already a string + if ( s.data && s.processData && typeof s.data !== "string" ) { + s.data = jQuery.param( s.data, s.traditional ); + } + + // Apply prefilters + inspectPrefiltersOrTransports( prefilters, s, options, jqXHR ); + + // If request was aborted inside a prefilter, stop there + if ( completed ) { + return jqXHR; + } + + // We can fire global events as of now if asked to + // Don't fire events if jQuery.event is undefined in an AMD-usage scenario (#15118) + fireGlobals = jQuery.event && s.global; + + // Watch for a new set of requests + if ( fireGlobals && jQuery.active++ === 0 ) { + jQuery.event.trigger( "ajaxStart" ); + } + + // Uppercase the type + s.type = s.type.toUpperCase(); + + // Determine if request has content + s.hasContent = !rnoContent.test( s.type ); + + // Save the URL in case we're toying with the If-Modified-Since + // and/or If-None-Match header later on + // Remove hash to simplify url manipulation + cacheURL = s.url.replace( rhash, "" ); + + // More options handling for requests with no content + if ( !s.hasContent ) { + + // Remember the hash so we can put it back + uncached = s.url.slice( cacheURL.length ); + + // If data is available and should be processed, append data to url + if ( s.data && ( s.processData || typeof s.data === "string" ) ) { + cacheURL += ( rquery.test( cacheURL ) ? "&" : "?" ) + s.data; + + // #9682: remove data so that it's not used in an eventual retry + delete s.data; + } + + // Add or update anti-cache param if needed + if ( s.cache === false ) { + cacheURL = cacheURL.replace( rantiCache, "$1" ); + uncached = ( rquery.test( cacheURL ) ? "&" : "?" ) + "_=" + ( nonce.guid++ ) + + uncached; + } + + // Put hash and anti-cache on the URL that will be requested (gh-1732) + s.url = cacheURL + uncached; + + // Change '%20' to '+' if this is encoded form body content (gh-2658) + } else if ( s.data && s.processData && + ( s.contentType || "" ).indexOf( "application/x-www-form-urlencoded" ) === 0 ) { + s.data = s.data.replace( r20, "+" ); + } + + // Set the If-Modified-Since and/or If-None-Match header, if in ifModified mode. + if ( s.ifModified ) { + if ( jQuery.lastModified[ cacheURL ] ) { + jqXHR.setRequestHeader( "If-Modified-Since", jQuery.lastModified[ cacheURL ] ); + } + if ( jQuery.etag[ cacheURL ] ) { + jqXHR.setRequestHeader( "If-None-Match", jQuery.etag[ cacheURL ] ); + } + } + + // Set the correct header, if data is being sent + if ( s.data && s.hasContent && s.contentType !== false || options.contentType ) { + jqXHR.setRequestHeader( "Content-Type", s.contentType ); + } + + // Set the Accepts header for the server, depending on the dataType + jqXHR.setRequestHeader( + "Accept", + s.dataTypes[ 0 ] && s.accepts[ s.dataTypes[ 0 ] ] ? + s.accepts[ s.dataTypes[ 0 ] ] + + ( s.dataTypes[ 0 ] !== "*" ? ", " + allTypes + "; q=0.01" : "" ) : + s.accepts[ "*" ] + ); + + // Check for headers option + for ( i in s.headers ) { + jqXHR.setRequestHeader( i, s.headers[ i ] ); + } + + // Allow custom headers/mimetypes and early abort + if ( s.beforeSend && + ( s.beforeSend.call( callbackContext, jqXHR, s ) === false || completed ) ) { + + // Abort if not done already and return + return jqXHR.abort(); + } + + // Aborting is no longer a cancellation + strAbort = "abort"; + + // Install callbacks on deferreds + completeDeferred.add( s.complete ); + jqXHR.done( s.success ); + jqXHR.fail( s.error ); + + // Get transport + transport = inspectPrefiltersOrTransports( transports, s, options, jqXHR ); + + // If no transport, we auto-abort + if ( !transport ) { + done( -1, "No Transport" ); + } else { + jqXHR.readyState = 1; + + // Send global event + if ( fireGlobals ) { + globalEventContext.trigger( "ajaxSend", [ jqXHR, s ] ); + } + + // If request was aborted inside ajaxSend, stop there + if ( completed ) { + return jqXHR; + } + + // Timeout + if ( s.async && s.timeout > 0 ) { + timeoutTimer = window.setTimeout( function() { + jqXHR.abort( "timeout" ); + }, s.timeout ); + } + + try { + completed = false; + transport.send( requestHeaders, done ); + } catch ( e ) { + + // Rethrow post-completion exceptions + if ( completed ) { + throw e; + } + + // Propagate others as results + done( -1, e ); + } + } + + // Callback for when everything is done + function done( status, nativeStatusText, responses, headers ) { + var isSuccess, success, error, response, modified, + statusText = nativeStatusText; + + // Ignore repeat invocations + if ( completed ) { + return; + } + + completed = true; + + // Clear timeout if it exists + if ( timeoutTimer ) { + window.clearTimeout( timeoutTimer ); + } + + // Dereference transport for early garbage collection + // (no matter how long the jqXHR object will be used) + transport = undefined; + + // Cache response headers + responseHeadersString = headers || ""; + + // Set readyState + jqXHR.readyState = status > 0 ? 4 : 0; + + // Determine if successful + isSuccess = status >= 200 && status < 300 || status === 304; + + // Get response data + if ( responses ) { + response = ajaxHandleResponses( s, jqXHR, responses ); + } + + // Use a noop converter for missing script but not if jsonp + if ( !isSuccess && + jQuery.inArray( "script", s.dataTypes ) > -1 && + jQuery.inArray( "json", s.dataTypes ) < 0 ) { + s.converters[ "text script" ] = function() {}; + } + + // Convert no matter what (that way responseXXX fields are always set) + response = ajaxConvert( s, response, jqXHR, isSuccess ); + + // If successful, handle type chaining + if ( isSuccess ) { + + // Set the If-Modified-Since and/or If-None-Match header, if in ifModified mode. + if ( s.ifModified ) { + modified = jqXHR.getResponseHeader( "Last-Modified" ); + if ( modified ) { + jQuery.lastModified[ cacheURL ] = modified; + } + modified = jqXHR.getResponseHeader( "etag" ); + if ( modified ) { + jQuery.etag[ cacheURL ] = modified; + } + } + + // if no content + if ( status === 204 || s.type === "HEAD" ) { + statusText = "nocontent"; + + // if not modified + } else if ( status === 304 ) { + statusText = "notmodified"; + + // If we have data, let's convert it + } else { + statusText = response.state; + success = response.data; + error = response.error; + isSuccess = !error; + } + } else { + + // Extract error from statusText and normalize for non-aborts + error = statusText; + if ( status || !statusText ) { + statusText = "error"; + if ( status < 0 ) { + status = 0; + } + } + } + + // Set data for the fake xhr object + jqXHR.status = status; + jqXHR.statusText = ( nativeStatusText || statusText ) + ""; + + // Success/Error + if ( isSuccess ) { + deferred.resolveWith( callbackContext, [ success, statusText, jqXHR ] ); + } else { + deferred.rejectWith( callbackContext, [ jqXHR, statusText, error ] ); + } + + // Status-dependent callbacks + jqXHR.statusCode( statusCode ); + statusCode = undefined; + + if ( fireGlobals ) { + globalEventContext.trigger( isSuccess ? "ajaxSuccess" : "ajaxError", + [ jqXHR, s, isSuccess ? success : error ] ); + } + + // Complete + completeDeferred.fireWith( callbackContext, [ jqXHR, statusText ] ); + + if ( fireGlobals ) { + globalEventContext.trigger( "ajaxComplete", [ jqXHR, s ] ); + + // Handle the global AJAX counter + if ( !( --jQuery.active ) ) { + jQuery.event.trigger( "ajaxStop" ); + } + } + } + + return jqXHR; + }, + + getJSON: function( url, data, callback ) { + return jQuery.get( url, data, callback, "json" ); + }, + + getScript: function( url, callback ) { + return jQuery.get( url, undefined, callback, "script" ); + } +} ); + +jQuery.each( [ "get", "post" ], function( _i, method ) { + jQuery[ method ] = function( url, data, callback, type ) { + + // Shift arguments if data argument was omitted + if ( isFunction( data ) ) { + type = type || callback; + callback = data; + data = undefined; + } + + // The url can be an options object (which then must have .url) + return jQuery.ajax( jQuery.extend( { + url: url, + type: method, + dataType: type, + data: data, + success: callback + }, jQuery.isPlainObject( url ) && url ) ); + }; +} ); + +jQuery.ajaxPrefilter( function( s ) { + var i; + for ( i in s.headers ) { + if ( i.toLowerCase() === "content-type" ) { + s.contentType = s.headers[ i ] || ""; + } + } +} ); + + +jQuery._evalUrl = function( url, options, doc ) { + return jQuery.ajax( { + url: url, + + // Make this explicit, since user can override this through ajaxSetup (#11264) + type: "GET", + dataType: "script", + cache: true, + async: false, + global: false, + + // Only evaluate the response if it is successful (gh-4126) + // dataFilter is not invoked for failure responses, so using it instead + // of the default converter is kludgy but it works. + converters: { + "text script": function() {} + }, + dataFilter: function( response ) { + jQuery.globalEval( response, options, doc ); + } + } ); +}; + + +jQuery.fn.extend( { + wrapAll: function( html ) { + var wrap; + + if ( this[ 0 ] ) { + if ( isFunction( html ) ) { + html = html.call( this[ 0 ] ); + } + + // The elements to wrap the target around + wrap = jQuery( html, this[ 0 ].ownerDocument ).eq( 0 ).clone( true ); + + if ( this[ 0 ].parentNode ) { + wrap.insertBefore( this[ 0 ] ); + } + + wrap.map( function() { + var elem = this; + + while ( elem.firstElementChild ) { + elem = elem.firstElementChild; + } + + return elem; + } ).append( this ); + } + + return this; + }, + + wrapInner: function( html ) { + if ( isFunction( html ) ) { + return this.each( function( i ) { + jQuery( this ).wrapInner( html.call( this, i ) ); + } ); + } + + return this.each( function() { + var self = jQuery( this ), + contents = self.contents(); + + if ( contents.length ) { + contents.wrapAll( html ); + + } else { + self.append( html ); + } + } ); + }, + + wrap: function( html ) { + var htmlIsFunction = isFunction( html ); + + return this.each( function( i ) { + jQuery( this ).wrapAll( htmlIsFunction ? html.call( this, i ) : html ); + } ); + }, + + unwrap: function( selector ) { + this.parent( selector ).not( "body" ).each( function() { + jQuery( this ).replaceWith( this.childNodes ); + } ); + return this; + } +} ); + + +jQuery.expr.pseudos.hidden = function( elem ) { + return !jQuery.expr.pseudos.visible( elem ); +}; +jQuery.expr.pseudos.visible = function( elem ) { + return !!( elem.offsetWidth || elem.offsetHeight || elem.getClientRects().length ); +}; + + + + +jQuery.ajaxSettings.xhr = function() { + try { + return new window.XMLHttpRequest(); + } catch ( e ) {} +}; + +var xhrSuccessStatus = { + + // File protocol always yields status code 0, assume 200 + 0: 200, + + // Support: IE <=9 only + // #1450: sometimes IE returns 1223 when it should be 204 + 1223: 204 + }, + xhrSupported = jQuery.ajaxSettings.xhr(); + +support.cors = !!xhrSupported && ( "withCredentials" in xhrSupported ); +support.ajax = xhrSupported = !!xhrSupported; + +jQuery.ajaxTransport( function( options ) { + var callback, errorCallback; + + // Cross domain only allowed if supported through XMLHttpRequest + if ( support.cors || xhrSupported && !options.crossDomain ) { + return { + send: function( headers, complete ) { + var i, + xhr = options.xhr(); + + xhr.open( + options.type, + options.url, + options.async, + options.username, + options.password + ); + + // Apply custom fields if provided + if ( options.xhrFields ) { + for ( i in options.xhrFields ) { + xhr[ i ] = options.xhrFields[ i ]; + } + } + + // Override mime type if needed + if ( options.mimeType && xhr.overrideMimeType ) { + xhr.overrideMimeType( options.mimeType ); + } + + // X-Requested-With header + // For cross-domain requests, seeing as conditions for a preflight are + // akin to a jigsaw puzzle, we simply never set it to be sure. + // (it can always be set on a per-request basis or even using ajaxSetup) + // For same-domain requests, won't change header if already provided. + if ( !options.crossDomain && !headers[ "X-Requested-With" ] ) { + headers[ "X-Requested-With" ] = "XMLHttpRequest"; + } + + // Set headers + for ( i in headers ) { + xhr.setRequestHeader( i, headers[ i ] ); + } + + // Callback + callback = function( type ) { + return function() { + if ( callback ) { + callback = errorCallback = xhr.onload = + xhr.onerror = xhr.onabort = xhr.ontimeout = + xhr.onreadystatechange = null; + + if ( type === "abort" ) { + xhr.abort(); + } else if ( type === "error" ) { + + // Support: IE <=9 only + // On a manual native abort, IE9 throws + // errors on any property access that is not readyState + if ( typeof xhr.status !== "number" ) { + complete( 0, "error" ); + } else { + complete( + + // File: protocol always yields status 0; see #8605, #14207 + xhr.status, + xhr.statusText + ); + } + } else { + complete( + xhrSuccessStatus[ xhr.status ] || xhr.status, + xhr.statusText, + + // Support: IE <=9 only + // IE9 has no XHR2 but throws on binary (trac-11426) + // For XHR2 non-text, let the caller handle it (gh-2498) + ( xhr.responseType || "text" ) !== "text" || + typeof xhr.responseText !== "string" ? + { binary: xhr.response } : + { text: xhr.responseText }, + xhr.getAllResponseHeaders() + ); + } + } + }; + }; + + // Listen to events + xhr.onload = callback(); + errorCallback = xhr.onerror = xhr.ontimeout = callback( "error" ); + + // Support: IE 9 only + // Use onreadystatechange to replace onabort + // to handle uncaught aborts + if ( xhr.onabort !== undefined ) { + xhr.onabort = errorCallback; + } else { + xhr.onreadystatechange = function() { + + // Check readyState before timeout as it changes + if ( xhr.readyState === 4 ) { + + // Allow onerror to be called first, + // but that will not handle a native abort + // Also, save errorCallback to a variable + // as xhr.onerror cannot be accessed + window.setTimeout( function() { + if ( callback ) { + errorCallback(); + } + } ); + } + }; + } + + // Create the abort callback + callback = callback( "abort" ); + + try { + + // Do send the request (this may raise an exception) + xhr.send( options.hasContent && options.data || null ); + } catch ( e ) { + + // #14683: Only rethrow if this hasn't been notified as an error yet + if ( callback ) { + throw e; + } + } + }, + + abort: function() { + if ( callback ) { + callback(); + } + } + }; + } +} ); + + + + +// Prevent auto-execution of scripts when no explicit dataType was provided (See gh-2432) +jQuery.ajaxPrefilter( function( s ) { + if ( s.crossDomain ) { + s.contents.script = false; + } +} ); + +// Install script dataType +jQuery.ajaxSetup( { + accepts: { + script: "text/javascript, application/javascript, " + + "application/ecmascript, application/x-ecmascript" + }, + contents: { + script: /\b(?:java|ecma)script\b/ + }, + converters: { + "text script": function( text ) { + jQuery.globalEval( text ); + return text; + } + } +} ); + +// Handle cache's special case and crossDomain +jQuery.ajaxPrefilter( "script", function( s ) { + if ( s.cache === undefined ) { + s.cache = false; + } + if ( s.crossDomain ) { + s.type = "GET"; + } +} ); + +// Bind script tag hack transport +jQuery.ajaxTransport( "script", function( s ) { + + // This transport only deals with cross domain or forced-by-attrs requests + if ( s.crossDomain || s.scriptAttrs ) { + var script, callback; + return { + send: function( _, complete ) { + script = jQuery( " + + + + + + + + + + + + +
+
+
+
+ +
+

Administration API

+
+

Get configuration data

+
GET /config  HTTP/1.1
+
+
+
+

Sample response

+
{
+  "autocomplete": "",
+  "categories": [
+    "map",
+    "it",
+    "images",
+  ],
+  "default_locale": "",
+  "default_theme": "simple",
+  "engines": [
+    {
+      "categories": [
+        "map"
+      ],
+      "enabled": true,
+      "name": "openstreetmap",
+      "shortcut": "osm"
+    },
+    {
+      "categories": [
+        "it"
+      ],
+      "enabled": true,
+      "name": "arch linux wiki",
+      "shortcut": "al"
+    },
+    {
+      "categories": [
+        "images"
+      ],
+      "enabled": true,
+      "name": "google images",
+      "shortcut": "goi"
+    },
+    {
+      "categories": [
+        "it"
+      ],
+      "enabled": false,
+      "name": "bitbucket",
+      "shortcut": "bb"
+    },
+  ],
+  "instance_name": "searx",
+  "locales": {
+    "de": "Deutsch (German)",
+    "en": "English",
+    "eo": "Esperanto (Esperanto)",
+  },
+  "plugins": [
+    {
+      "enabled": true,
+      "name": "HTTPS rewrite"
+    },
+    {
+      "enabled": false,
+      "name": "Vim-like hotkeys"
+    }
+  ],
+  "safe_search": 0
+}
+
+
+
+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/architecture.html b/admin/architecture.html new file mode 100644 index 00000000..5aa54049 --- /dev/null +++ b/admin/architecture.html @@ -0,0 +1,192 @@ + + + + + + + + + Architecture — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Architecture

+ +

Herein you will find some hints and suggestions about typical architectures of +SearXNG infrastructures.

+
+

uWSGI Setup

+

We start with a reference setup for public SearXNG instances which can be build +up and maintained by the scripts from our DevOps tooling box.

+
+arch_public.dot
+

Fig. 1 Reference architecture of a public SearXNG setup.

+
+
+

The reference installation activates server.limiter, server.image_proxy +and ui.static_use_hash (/etc/searxng/settings.yml)

+
# SearXNG settings
+
+use_default_settings: true
+
+general:
+  debug: false
+  instance_name: "SearXNG"
+
+search:
+  safe_search: 2
+  autocomplete: 'duckduckgo'
+
+server:
+  secret_key: "ultrasecretkey"
+  limiter: true
+  image_proxy: true
+
+redis:
+  url: unix:///usr/local/searxng-redis/run/redis.sock?db=0
+
+ui:
+  static_use_hash: true
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/buildhosts.html b/admin/buildhosts.html new file mode 100644 index 00000000..28faa270 --- /dev/null +++ b/admin/buildhosts.html @@ -0,0 +1,282 @@ + + + + + + + + + Buildhosts — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Buildhosts

+ + +

To get best results from build, its recommend to install additional packages +on build hosts (see utils/searxng.sh).:

+
sudo -H ./utils/searxng.sh install buildhost
+
+
+

This will install packages needed by searx:

+
+
$ sudo -H apt-get install -y \
+    python3-dev python3-babel python3-venv \
+    uwsgi uwsgi-plugin-python3 \
+    git build-essential libxslt-dev zlib1g-dev libffi-dev libssl-dev
+
+
+
+

and packages needed to build docuemtation and run tests:

+
+
$ sudo -H apt-get install -y \
+    firefox graphviz imagemagick texlive-xetex librsvg2-bin \
+    texlive-latex-recommended texlive-extra-utils fonts-dejavu \
+    latexmk shellcheck
+
+
+
+
+

Build docs

+ +

Most of the sphinx requirements are installed from git://setup.py and the +docs can be build from scratch with make docs.html. For better math and +image processing additional packages are needed. The XeTeX needed not only for +PDF creation, its also needed for Math equations when HTML output is build.

+

To be able to do Math support for HTML outputs in Sphinx without CDNs, the math are rendered +as images (sphinx.ext.imgmath extension).

+

Here is the extract from the git://docs/conf.py file, setting math renderer +to imgmath:

+
html_math_renderer = 'imgmath'
+imgmath_image_format = 'svg'
+imgmath_font_size = 14
+
+
+

If your docs build (make docs.html) shows warnings like this:

+
WARNING: dot(1) not found, for better output quality install \
+         graphviz from https://www.graphviz.org
+..
+WARNING: LaTeX command 'latex' cannot be run (needed for math \
+         display), check the imgmath_latex setting
+
+
+

you need to install additional packages on your build host, to get better HTML +output.

+
+
$ sudo apt install graphviz imagemagick texlive-xetex librsvg2-bin
+
+
+
+

For PDF output you also need:

+
+
$ sudo apt texlive-latex-recommended texlive-extra-utils ttf-dejavu
+
+
+
+
+
+

Lint shell scripts

+

To lint shell scripts, we use ShellCheck - A shell script static analysis tool.

+
+
$ sudo apt install shellcheck
+
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/engines/command-line-engines.html b/admin/engines/command-line-engines.html new file mode 100644 index 00000000..84db84f3 --- /dev/null +++ b/admin/engines/command-line-engines.html @@ -0,0 +1,218 @@ + + + + + + + + + Command Line Engines — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Command Line Engines

+ +

With command engines administrators can run engines to integrate arbitrary +shell commands.

+

When creating and enabling a command engine on a public instance, you must +be careful to avoid leaking private data. The easiest solution is to limit the +access by setting tokens as described in section Private Engines (tokens).

+

The engine base is flexible. Only your imagination can limit the power of this +engine (and maybe security concerns). The following options are available:

+
+
command:

A comma separated list of the elements of the command. A special token +{{QUERY}} tells where to put the search terms of the user. Example:

+
['ls', '-l', '-h', '{{QUERY}}']
+
+
+
+
delimiter:

A mapping containing a delimiter char and the titles of each element in +keys.

+
+
parse_regex:

A dict containing the regular expressions for each result key.

+
+
+

query_type:

+
+

The expected type of user search terms. Possible values: path and +enum.

+
+
path:

Checks if the user provided path is inside the working directory. If not, +the query is not executed.

+
+
enum:

Is a list of allowed search terms. If the user submits something which is +not included in the list, the query returns an error.

+
+
+
+
+
query_enum:

A list containing allowed search terms if query_type is set to enum.

+
+
+

working_dir:

+
+

The directory where the command has to be executed. Default: ./

+
+
+
result_separator:

The character that separates results. Default: \n

+
+
+

The example engine below can be used to find files with a specific name in the +configured working directory:

+
- name: find
+  engine: command
+  command: ['find', '.', '-name', '{{QUERY}}']
+  query_type: path
+  shortcut: fnd
+  delimiter:
+      chars: ' '
+      keys: ['line']
+
+
+
+

Acknowledgment

+

This development was sponsored by Search and Discovery Fund of NLnet Foundation.

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/engines/configured_engines.html b/admin/engines/configured_engines.html new file mode 100644 index 00000000..9ac2f2c0 --- /dev/null +++ b/admin/engines/configured_engines.html @@ -0,0 +1,2032 @@ + + + + + + + + + Configured Engines — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Configured Engines

+ +

Explanation of the General Engine Configuration shown in the table +Configured Engines.

+

SearXNG supports 139 search engines (of which 61 are enabled by default).

+
+

general search engines

+
+

web

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

bing

!bi

bing

y

3.0

1

y

y

brave

!brave

xpath

y

3.0

1

y

y

duckduckgo

!ddg

duckduckgo

3.0

1

y

y

y

gigablast

!gb

gigablast

y

4.0

1

y

google

!go

google

3.0

1

y

y

y

y

mojeek

!mjk

xpath

y

3.0

1

y

neeva

!nv

xpath

y

5.0

1

y

y

qwant

!qw

qwant

3.0

1

y

y

y

startpage

!sp

startpage

y

6.0

1

y

y

wiby

!wib

json_engine

y

3.0

1

yahoo

!yh

yahoo

y

3.0

1

y

y

y

seznam

!szn

seznam

y +(CZ)

3.0

1

goo

!goo

xpath

y +(JA)

4.0

1

y

naver

!nvr

xpath

y +(KO)

3.0

1

y

+
+
+

others

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

alexandria

!alx

json_engine

y

1.5

1

y

archive is

!ai

xpath

y

7.0

1

curlie

!cl

xpath

y

3.0

1

y

currency

!cc

currency_convert

3.0

100

not applicable (online_currency)

ddg definitions

!ddd

duckduckgo_definitions

y

3.0

2

y

dictzone

!dc

dictzone

3.0

100

not applicable (online_dictionary)

lingva

!lv

lingva

3.0

1

not applicable (online_dictionary)

marginalia

!mar

json_engine

y

1.5

1

petalsearch

!pts

xpath

y

3.0

1

y

tineye

!tin

tineye

9.0

1

not applicable (online_url_search)

wikibooks

!wb

mediawiki

y

3.0

1

y

wikidata

!wd

wikidata

3.0

2

y

wikipedia

!wp

wikipedia

3.0

1

y

wikiquote

!wq

mediawiki

y

3.0

1

y

wikisource

!ws

mediawiki

y

3.0

1

y

wikiversity

!wv

mediawiki

y

3.0

1

y

wikivoyage

!wy

mediawiki

y

3.0

1

y

yep

!yep

json_engine

y

3.0

1

wikimini

!wkmn

xpath

y +(FR)

3.0

1

+
+
+
+

images search engines

+
+

web

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

bing images

!bii

bing_images

3.0

1

y

y

y

y

duckduckgo images

!ddi

duckduckgo_images

y

3.0

1

y

y

y

google images

!goi

google_images

3.0

1

y

y

y

y

qwant images

!qwi

qwant

3.0

1

y

y

y

+
+
+

others

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

1x

!1x

www1x

y

3.0

1

artic

!arc

artic

4.0

1

y

deviantart

!da

deviantart

3.0

1

y

y

flickr

!fl

flickr_noapi

3.0

1

y

y

frinkiac

!frk

frinkiac

y

3.0

1

library of congress

!loc

loc

3.0

1

y

openverse

!opv

openverse

3.0

1

y

petalsearch images

!ptsi

petal_images

y

3.0

1

y

y

unsplash

!us

unsplash

3.0

1

y

+
+
+
+

videos search engines

+
+

web

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

bing videos

!biv

bing_videos

3.0

1

y

y

y

y

google videos

!gov

google_videos

3.0

1

y

y

y

qwant videos

!qwv

qwant

3.0

1

y

y

y

+
+
+

others

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

ccc-tv

!c3tv

xpath

y

3.0

1

dailymotion

!dm

dailymotion

3.0

1

y

y

y

y

google play movies

!gpm

xpath

y

3.0

1

invidious

!iv

invidious

y

3.0

1

y

y

peertube

!ptb

peertube

y

6.0

1

y

y

rumble

!ru

rumble

y

3.0

1

y

sepiasearch

!sep

sepiasearch

3.0

1

y

y

y

y

vimeo

!vm

vimeo

3.0

1

y

youtube

!yt

youtube_noapi

3.0

1

y

y

mediathekviewweb

!mvw

mediathekviewweb

y +(DE)

3.0

1

y

ina

!in

ina

y +(FR)

6.0

1

y

+
+
+
+

news search engines

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

bing news

!bin

bing_news

3.0

1

y

y

y

google news

!gon

google_news

3.0

1

y

y

petalsearch news

!ptsn

xpath

y

3.0

1

y

qwant news

!qwn

qwant

3.0

1

y

y

y

wikinews

!wn

mediawiki

y

3.0

1

y

yahoo news

!yhn

yahoo_news

3.0

1

y

+
+
+

map search engines

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

apple maps

!apm

apple_maps

y

5.0

1

openstreetmap

!osm

openstreetmap

3.0

1

photon

!ph

photon

3.0

1

y

+
+
+

music search engines

+
+

lyrics

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

azlyrics

!lyrics

xpath

y

4.0

1

y

genius

!gen

genius

3.0

1

y

+
+
+

others

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

bandcamp

!bc

bandcamp

3.0

1

y

deezer

!dz

deezer

y

3.0

1

y

gpodder

!gpod

json_engine

y

4.0

1

invidious

!iv

invidious

y

3.0

1

y

y

mixcloud

!mc

mixcloud

3.0

1

y

soundcloud

!sc

soundcloud

3.0

1

y

youtube

!yt

youtube_noapi

3.0

1

y

y

+
+
+
+

it search engines

+
+

packages

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

docker hub

!dh

docker_hub

3.0

1

y

hoogle

!ho

xpath

3.0

1

y

lib.rs

!lrs

xpath

y

3.0

1

metacpan

!cpan

metacpan

y

3.0

1

y

npm

!npm

json_engine

y

5.0

1

y

packagist

!pack

json_engine

y

5.0

1

y

pub.dev

!pd

xpath

y

3.0

1

y

pypi

!pypi

xpath

3.0

1

y

rubygems

!rbg

xpath

y

3.0

1

y

+
+
+

q&a

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

askubuntu

!ubuntu

stackexchange

3.0

1

y

stackoverflow

!st

stackexchange

3.0

1

y

superuser

!su

stackexchange

3.0

1

y

+
+
+

repos

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

bitbucket

!bb

xpath

y

4.0

1

y

codeberg

!cb

json_engine

y

3.0

1

github

!gh

github

3.0

1

gitlab

!gl

json_engine

y

10.0

1

y

sourcehut

!srht

xpath

y

3.0

1

y

+
+
+

software wikis

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

arch linux wiki

!al

archlinux

3.0

1

y

y

free software directory

!fsd

mediawiki

y

5.0

1

y

gentoo

!ge

gentoo

3.0

1

y

y

+
+
+

others

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

framalibre

!frl

framalibre

y

3.0

1

y

habrahabr

!habr

xpath

y

4.0

1

y

lobste.rs

!lo

xpath

y

5.0

1

mankier

!man

json_engine

3.0

1

searchcode code

!scc

searchcode_code

y

3.0

1

y

+
+
+
+

science search engines

+
+

scientific publications

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

arxiv

!arx

arxiv

4.0

1

y

crossref

!cr

crossref

y

30

1

y

google scholar

!gos

google_scholar

3.0

1

y

y

y

pubmed

!pub

pubmed

3.0

1

semantic scholar

!se

semantic_scholar

y

3.0

1

y

+
+
+

others

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

openairedatasets

!oad

json_engine

5.0

1

y

openairepublications

!oap

json_engine

5.0

1

y

pdbe

!pdb

pdbe

3.0

1

+
+
+
+

files search engines

+
+

apps

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

apk mirror

!apkm

apkmirror

y

4.0

1

y

apple app store

!aps

apple_app_store

y

3.0

1

y

fdroid

!fd

fdroid

y

3.0

1

y

google play apps

!gpa

google_play_apps

y

3.0

1

+
+
+

others

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

1337x

!1337x

1337x

y

3.0

1

y

btdigg

!bt

btdigg

3.0

1

y

kickass

!kc

kickass

y

4.0

1

y

library genesis

!lg

xpath

y

7.0

1

nyaa

!nt

nyaa

y

3.0

1

y

openrepos

!or

xpath

y

4.0

1

y

piratebay

!tpb

piratebay

3.0

1

solidtorrents

!solid

solidtorrents

4.0

1

y

tokyotoshokan

!tt

tokyotoshokan

y

6.0

1

y

+
+
+
+

social media search engines

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Engines configured by default (in settings.yml)

Supported features

Name

Shortcut

Module

Disabled

Timeout

Weight

Paging

Language

Safe search

Time range

9gag

!9g

9gag

y

3.0

1

y

reddit

!re

reddit

3.0

1

twitter

!tw

twitter

y

3.0

1

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/engines/index.html b/admin/engines/index.html new file mode 100644 index 00000000..81aa4f75 --- /dev/null +++ b/admin/engines/index.html @@ -0,0 +1,166 @@ + + + + + + + + + Engines & Settings — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Engines & Settings

+ +
+ +
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/engines/nosql-engines.html b/admin/engines/nosql-engines.html new file mode 100644 index 00000000..84376d47 --- /dev/null +++ b/admin/engines/nosql-engines.html @@ -0,0 +1,262 @@ + + + + + + + + + NoSQL databases — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

NoSQL databases

+ +

The following NoSQL databases are supported:

+ +

All of the engines above are just commented out in the settings.yml, as you have to set various options and install +dependencies before using them.

+

By default, the engines use the key-value template for displaying results / +see simple +theme. If you are not satisfied with the original result layout, you can use +your own template, set result_template attribute to {template_name} and +place the templates at:

+
searx/templates/{theme_name}/result_templates/{template_name}
+
+
+

Furthermore, if you do not wish to expose these engines on a public instance, you +can still add them and limit the access by setting tokens as described in +section Private Engines (tokens).

+
+

Configure the engines

+

NoSQL databases are used for storing arbitrary data without first defining +their structure.

+
+

Extra Dependencies

+

For using Redis Server or MongoDB you need to +install additional packages in Python’s Virtual Environment of your SearXNG +instance. To switch into the environment (Install SearXNG & dependencies) you can use +utils/searxng.sh:

+
$ sudo utils/searxng.sh instance cmd bash
+(searxng-pyenv)$ pip install ...
+
+
+
+
+

Redis Server

+ +

Redis is an open source (BSD licensed), in-memory data structure (key value +based) store. Before configuring the redis_server engine, you must install +the dependency redis.

+

Select a database to search in and set its index in the option db. You can +either look for exact matches or use partial keywords to find what you are +looking for by configuring exact_match_only. You find an example +configuration below:

+
# Required dependency: redis
+
+- name: myredis
+  shortcut : rds
+  engine: redis_server
+  exact_match_only: false
+  host: '127.0.0.1'
+  port: 6379
+  enable_http: true
+  password: ''
+  db: 0
+
+
+
+
+

MongoDB

+ +

MongoDB is a document based database program that handles JSON like data. +Before configuring the mongodb engine, you must install the dependency +redis.

+

In order to query MongoDB, you have to select a database and a +collection. Furthermore, you have to select a key that is going to be +searched. MongoDB also supports the option exact_match_only, so configure +it as you wish. Below is an example configuration for using a MongoDB +collection:

+
# MongoDB engine
+# Required dependency: pymongo
+
+- name: mymongo
+  engine: mongodb
+  shortcut: md
+  exact_match_only: false
+  host: '127.0.0.1'
+  port: 27017
+  enable_http: true
+  results_per_page: 20
+  database: 'business'
+  collection: 'reviews'  # name of the db collection
+  key: 'name'            # key in the collection to search for
+
+
+
+
+
+

Acknowledgment

+

This development was sponsored by Search and Discovery Fund of NLnet Foundation.

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/engines/private-engines.html b/admin/engines/private-engines.html new file mode 100644 index 00000000..15bbe039 --- /dev/null +++ b/admin/engines/private-engines.html @@ -0,0 +1,181 @@ + + + + + + + + + Private Engines (tokens) — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Private Engines (tokens)

+

Administrators might find themselves wanting to limit access to some of the +enabled engines on their instances. It might be because they do not want to +expose some private information through Offline Engines. Or they would +rather share engines only with their trusted friends or colleagues.

+

To solve this issue the concept of private engines exists.

+

A new option was added to engines named tokens. It expects a list of +strings. If the user making a request presents one of the tokens of an engine, +they can access information about the engine and make search requests.

+

Example configuration to restrict access to the Arch Linux Wiki engine:

+
- name: arch linux wiki
+  engine: archlinux
+  shortcut: al
+  tokens: [ 'my-secret-token' ]
+
+
+

Unless a user has configured the right token, the engine is going +to be hidden from him/her. It is not going to be included in the +list of engines on the Preferences page and in the output of +/config REST API call.

+

Tokens can be added to one’s configuration on the Preferences page +under “Engine tokens”. The input expects a comma separated list of +strings.

+

The distribution of the tokens from the administrator to the users +is not carved in stone. As providing access to such engines +implies that the admin knows and trusts the user, we do not see +necessary to come up with a strict process. Instead, +we would like to add guidelines to the documentation of the feature.

+
+

Acknowledgment

+

This development was sponsored by Search and Discovery Fund of NLnet Foundation.

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/engines/recoll.html b/admin/engines/recoll.html new file mode 100644 index 00000000..4869732f --- /dev/null +++ b/admin/engines/recoll.html @@ -0,0 +1,190 @@ + + + + + + + + + Recoll Engine — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Recoll Engine

+ +

Recoll is a desktop full-text search tool based on Xapian. By itself Recoll +does not offer WEB or API access, this can be achieved using recoll-webui

+
+

Configuration

+

You must configure the following settings:

+
+
base_url:

Location where recoll-webui can be reached.

+
+
mount_prefix:

Location where the file hierarchy is mounted on your local filesystem.

+
+
dl_prefix:

Location where the file hierarchy as indexed by recoll can be reached.

+
+
search_dir:

Part of the indexed file hierarchy to be search, if empty the full domain is +searched.

+
+
+
+
+

Example

+

Scenario:

+
    +

  1. Recoll indexes a local filesystem mounted in /export/documents/reference,

    2. +

  2. the Recoll search interface can be reached at https://recoll.example.org/ and

    3. +

  3. the contents of this filesystem can be reached though https://download.example.org/reference

    4. +
+
base_url: https://recoll.example.org/
+mount_prefix: /export/documents
+dl_prefix: https://download.example.org
+search_dir: ''
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/engines/search-indexer-engines.html b/admin/engines/search-indexer-engines.html new file mode 100644 index 00000000..ebcd34bb --- /dev/null +++ b/admin/engines/search-indexer-engines.html @@ -0,0 +1,264 @@ + + + + + + + + + Local Search Engines — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Local Search Engines

+ +

Administrators might find themselves wanting to integrate locally running search +engines. The following ones are supported for now:

+ +

Each search engine is powerful, capable of full-text search. All of the engines +above are added to settings.yml just commented out, as you have to +base_url for all them.

+

Please note that if you are not using HTTPS to access these engines, you have to enable +HTTP requests by setting enable_http to True.

+

Furthermore, if you do not want to expose these engines on a public instance, you +can still add them and limit the access by setting tokens as described in +section Private Engines (tokens).

+
+

MeiliSearch

+ +

MeiliSearch is aimed at individuals and small companies. It is designed for +small-scale (less than 10 million documents) data collections. E.g. it is great +for storing web pages you have visited and searching in the contents later.

+

The engine supports faceted search, so you can search in a subset of documents +of the collection. Furthermore, you can search in MeiliSearch instances that +require authentication by setting auth_token.

+

Here is a simple example to query a Meilisearch instance:

+
- name: meilisearch
+  engine: meilisearch
+  shortcut: mes
+  base_url: http://localhost:7700
+  index: my-index
+  enable_http: true
+
+
+
+
+

Elasticsearch

+ +

Elasticsearch supports numerous ways to query the data it is storing. At the +moment the engine supports the most popular search methods (query_type):

+
    +

  • match,

    • +

  • simple_query_string,

    • +

  • term and

    • +

  • terms.

    • +
+

If none of the methods fit your use case, you can select custom query type +and provide the JSON payload to submit to Elasticsearch in +custom_query_json.

+

The following is an example configuration for an Elasticsearch instance with +authentication configured to read from my-index index.

+
- name: elasticsearch
+  shortcut: es
+  engine: elasticsearch
+  base_url: http://localhost:9200
+  username: elastic
+  password: changeme
+  index: my-index
+  query_type: match
+  # custom_query_json: '{ ... }'
+  enable_http: true
+
+
+
+
+

Solr

+ +

Solr is a popular search engine based on Lucene, just like Elasticsearch. But +instead of searching in indices, you can search in collections.

+

This is an example configuration for searching in the collection +my-collection and get the results in ascending order.

+
- name: solr
+  engine: solr
+  shortcut: slr
+  base_url: http://localhost:8983
+  collection: my-collection
+  sort: asc
+  enable_http: true
+
+
+
+
+

Acknowledgment

+

This development was sponsored by Search and Discovery Fund of NLnet Foundation.

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/engines/searx.engines.xpath.html b/admin/engines/searx.engines.xpath.html new file mode 100644 index 00000000..0b779a03 --- /dev/null +++ b/admin/engines/searx.engines.xpath.html @@ -0,0 +1,339 @@ + + + + + + + + + XPath Engine — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

XPath Engine

+

The XPath engine is a generic engine with which it is possible to configure +engines in the settings.

+

Here is a simple example of a XPath engine configured in the +Engine settings section, further read Engine Overview.

+
- name : bitbucket
+  engine : xpath
+  paging : True
+  search_url : https://bitbucket.org/repo/all/{pageno}?name={query}
+  url_xpath : //article[@class="repo-summary"]//a[@class="repo-link"]/@href
+  title_xpath : //article[@class="repo-summary"]//a[@class="repo-link"]
+  content_xpath : //article[@class="repo-summary"]/p
+
+
+
+
+searx.engines.xpath.content_xpath = None
+

XPath selector of result’s content.

+
+ +
+
+searx.engines.xpath.first_page_num = 1
+

Number of the first page (usually 0 or 1).

+
+ +
+
+searx.engines.xpath.headers = {}
+

Some engines might offer different result based on cookies or headers. +Possible use-case: To set safesearch cookie or header to moderate.

+
+ +
+
+searx.engines.xpath.lang_all = 'en'
+

Replacement {lang} in search_url if language all is +selected.

+
+ +
+
+searx.engines.xpath.no_result_for_http_status = []
+

Return empty result for these HTTP status codes instead of throwing an error.

+
no_result_for_http_status: []
+
+
+
+ +
+
+searx.engines.xpath.page_size = 1
+

Number of results on each page. Only needed if the site requires not a page +number, but an offset.

+
+ +
+
+searx.engines.xpath.paging = False
+

Engine supports paging [True or False].

+
+ +
+
+searx.engines.xpath.request(query, params)[source]
+

Build request parameters (see Making a Request).

+
+ +
+
+searx.engines.xpath.response(resp)[source]
+

Scrap results from the response (see Media Types).

+
+ +
+
+searx.engines.xpath.results_xpath = ''
+

XPath selector for the list of result items

+
+ +
+
+searx.engines.xpath.safe_search_map = {0: '&filter=none', 1: '&filter=moderate', 2: '&filter=strict'}
+

Maps safe-search value to {safe_search} in search_url.

+
safesearch: true
+safes_search_map:
+  0: '&filter=none'
+  1: '&filter=moderate'
+  2: '&filter=strict'
+
+
+
+ +
+
+searx.engines.xpath.safe_search_support = False
+

Engine supports safe-search.

+
+ +
+
+searx.engines.xpath.search_url = None
+

Search URL of the engine. Example:

+
https://example.org/?search={query}&page={pageno}{time_range}{safe_search}
+
+
+

Replacements are:

+
+
{query}:

Search terms from user.

+
+
{pageno}:

Page number if engine supports pagging paging

+
+
{lang}:

ISO 639-1 language code (en, de, fr ..)

+
+
{time_range}:

URL parameter if engine supports time +range. The value for the parameter is taken from +time_range_map.

+
+
{safe_search}:

Safe-search URL parameter if engine +supports safe-search. The {safe_search} +replacement is taken from the safes_search_map. Filter results:

+
0: none, 1: moderate, 2:strict
+
+
+

If not supported, the URL parameter is an empty string.

+
+
+
+ +
+
+searx.engines.xpath.soft_max_redirects = 0
+

Maximum redirects, soft limit. Record an error but don’t stop the engine

+
+ +
+
+searx.engines.xpath.suggestion_xpath = ''
+

XPath selector of result’s suggestion.

+
+ +
+
+searx.engines.xpath.thumbnail_xpath = False
+

XPath selector of result’s img_src.

+
+ +
+
+searx.engines.xpath.time_range_map = {'day': 24, 'month': 720, 'week': 168, 'year': 8760}
+

Maps time range value from user to {time_range_val} in +time_range_url.

+
time_range_map:
+  day: 1
+  week: 7
+  month: 30
+  year: 365
+
+
+
+ +
+
+searx.engines.xpath.time_range_support = False
+

Engine supports search time range.

+
+ +
+
+searx.engines.xpath.time_range_url = '&hours={time_range_val}'
+

Time range URL parameter in the in search_url. If no time range is +requested by the user, the URL parameter is an empty string. The +{time_range_val} replacement is taken from the time_range_map.

+
time_range_url : '&days={time_range_val}'
+
+
+
+ +
+
+searx.engines.xpath.title_xpath = None
+

XPath selector of result’s title.

+
+ +
+
+searx.engines.xpath.url_xpath = None
+

XPath selector of result’s url.

+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/engines/settings.html b/admin/engines/settings.html new file mode 100644 index 00000000..0e202b06 --- /dev/null +++ b/admin/engines/settings.html @@ -0,0 +1,750 @@ + + + + + + + + + settings.yml — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

settings.yml

+

This page describe the options possibilities of the git://searx/settings.yml +file.

+ + +
+

settings.yml location

+

The initial settings.yml we be load from these locations:

+
    +

  1. the full path specified in the SEARXNG_SETTINGS_PATH environment variable.

    2. +

  2. /etc/searxng/settings.yml

    3. +
+

If these files don’t exist (or are empty or can’t be read), SearXNG uses the +git://searx/settings.yml file. Read use_default_settings to +see how you can simplify your user defined settings.yml.

+
+
+

Global Settings

+
+

brand:

+
brand:
+  issue_url: https://github.com/searxng/searxng/issues
+  docs_url: https://docs.searxng.org
+  public_instances: https://searx.space
+  wiki_url: https://github.com/searxng/searxng/wiki
+
+
+
+
issue_url :

If you host your own issue tracker change this URL.

+
+
docs_url :

If you host your own documentation change this URL.

+
+
public_instances :

If you host your own https://searx.space change this URL.

+
+
wiki_url :

Link to your wiki (or false)

+
+
+
+
+

general:

+
general:
+  debug: false
+  instance_name:  "SearXNG"
+  privacypolicy_url: false
+  donation_url: https://docs.searxng.org/donate.html
+  contact_url: false
+  enable_metrics: true
+
+
+
+
debug$SEARXNG_DEBUG

Allow a more detailed log if you run SearXNG directly. Display detailed error +messages in the browser too, so this must be deactivated in production.

+
+
donation_url :

At default the donation link points to the SearXNG project. Set value to true to use your +own donation page written in the searx/info/en/donate.md and use false to disable the donation link altogether.

+
+
privacypolicy_url:

Link to privacy policy.

+
+
contact_url:

Contact mailto: address or WEB form.

+
+
enable_metrics:

Enabled by default. Record various anonymous metrics availabled at /stats, +/stats/errors and /preferences.

+
+
+
+ +
+

server:

+
server:
+    base_url: false                # set custom base_url (or false)
+    port: 8888
+    bind_address: "127.0.0.1"      # address to listen on
+    secret_key: "ultrasecretkey"   # change this!
+    limiter: false
+    image_proxy: false             # proxying image results through SearXNG
+    default_http_headers:
+      X-Content-Type-Options : nosniff
+      X-XSS-Protection : 1; mode=block
+      X-Download-Options : noopen
+      X-Robots-Tag : noindex, nofollow
+      Referrer-Policy : no-referrer
+
+
+ +
+
base_urlbuildenv SEARXNG_URL

The base URL where SearXNG is deployed. Used to create correct inbound links. +If you change the value, don’t forget to rebuild instance’s environment +(utils/brand.env)

+
+
port & bind_address: buildenv SEARXNG_PORT & SEARXNG_BIND_ADDRESS

Port number and bind address of the SearXNG web application if you run it +directly using python searx/webapp.py. Doesn’t apply to SearXNG running on +Apache or Nginx.

+
+
secret_key$SEARXNG_SECRET

Used for cryptography purpose.

+
+
+
+
limiter :

Rate limit the number of request on the instance, block some bots. The +Limiter Plugin requires a redis: database.

+
+
+
+
image_proxy :

Allow your instance of SearXNG of being able to proxy images. Uses memory space.

+
+
+
+
default_http_headers :

Set additional HTTP headers, see #755

+
+
+
+
+

ui:

+
ui:
+  static_use_hash: false
+  default_locale: ""
+  query_in_title: false
+  infinite_scroll: false
+  center_alignment: false
+  cache_url: https://web.archive.org/web/
+  default_theme: simple
+  theme_args:
+    simple_style: auto
+
+
+
+
static_use_hash :

Enables cache busting of static files.

+
+
default_locale :

SearXNG interface language. If blank, the locale is detected by using the +browser language. If it doesn’t work, or you are deploying a language +specific instance of searx, a locale can be defined using an ISO language +code, like fr, en, de.

+
+
query_in_title :

When true, the result page’s titles contains the query it decreases the +privacy, since the browser can records the page titles.

+
+
infinite_scroll:

When true, automatically loads the next page when scrolling to bottom of the current page.

+
+
center_alignmentdefault false

When enabled, the results are centered instead of being in the left (or RTL) +side of the screen. This setting only affects the desktop layout +(min-width: @tablet)

+
+
+
+
cache_urlhttps://web.archive.org/web/

URL prefix of the internet archive or cache, don’t forgett trailing slash (if +needed). The default is https://web.archive.org/web/ alternatives are:

+ +
+
default_theme :

Name of the theme you want to use by default on your SearXNG instance.

+
+
theme_args.simple_style:

Style of simple theme: auto, light, dark

+
+
results_on_new_tab:

Open result links in a new tab by default.

+
+
+
+
+

redis:

+

A redis DB can be connected by an URL, in searx.redisdb you +will find a description to test your redis connection in SerXNG. When using +sockets, don’t forget to check the access rights on the socket:

+
ls -la /usr/local/searxng-redis/run/redis.sock
+srwxrwx--- 1 searxng-redis searxng-redis ... /usr/local/searxng-redis/run/redis.sock
+
+
+

In this example read/write access is given to the searxng-redis group. To get +access rights to redis instance (the socket), your SearXNG (or even your +developer) account needs to be added to the searxng-redis group.

+
+
url

URL to connect redis database, see Redis.from_url(url) & Redis DB:

+
redis://[[username]:[password]]@localhost:6379/0
+rediss://[[username]:[password]]@localhost:6379/0
+unix://[[username]:[password]]@/path/to/socket.sock?db=0
+
+
+
+
+
+

Tip for developers

+

To set up a local redis instance, first set the socket path of the Redis DB +in your YAML setting:

+
redis:
+  url: unix:///usr/local/searxng-redis/run/redis.sock?db=0
+
+
+

Then use the following commands to install the redis instance

+
$ ./manage redis.build
+$ sudo -H ./manage redis.install
+$ sudo -H ./manage redis.addgrp "${USER}"
+# don't forget to logout & login to get member of group
+
+
+
+
+
+

outgoing:

+

Communication with search engines.

+
outgoing:
+  request_timeout: 2.0       # default timeout in seconds, can be override by engine
+  max_request_timeout: 10.0  # the maximum timeout in seconds
+  useragent_suffix: ""       # information like an email address to the administrator
+  pool_connections: 100      # Maximum number of allowable connections, or null
+                             # for no limits. The default is 100.
+  pool_maxsize: 10           # Number of allowable keep-alive connections, or null
+                             # to always allow. The default is 10.
+  enable_http2: true         # See https://www.python-httpx.org/http2/
+  # uncomment below section if you want to use a custom server certificate
+  # see https://www.python-httpx.org/advanced/#changing-the-verification-defaults
+  # and https://www.python-httpx.org/compatibility/#ssl-configuration
+  #  verify: ~/.mitmproxy/mitmproxy-ca-cert.cer
+  #
+  # uncomment below section if you want to use a proxyq see: SOCKS proxies
+  #   https://2.python-requests.org/en/latest/user/advanced/#proxies
+  # are also supported: see
+  #   https://2.python-requests.org/en/latest/user/advanced/#socks
+  #
+  #  proxies:
+  #    all://:
+  #      - http://proxy1:8080
+  #      - http://proxy2:8080
+  #
+  #  using_tor_proxy: true
+  #
+  # Extra seconds to add in order to account for the time taken by the proxy
+  #
+  #  extra_proxy_timeout: 10.0
+  #
+
+
+
+
request_timeout :

Global timeout of the requests made to others engines in seconds. A bigger +timeout will allow to wait for answers from slow engines, but in consequence +will slow SearXNG reactivity (the result page may take the time specified in the +timeout to load). Can be override by Engine settings

+
+
useragent_suffix :

Suffix to the user-agent SearXNG uses to send requests to others engines. If an +engine wish to block you, a contact info here may be useful to avoid that.

+
+
keepalive_expiry :

Number of seconds to keep a connection in the pool. By default 5.0 seconds.

+
+
+
+
proxies :

Define one or more proxies you wish to use, see httpx proxies. +If there are more than one proxy for one protocol (http, https), +requests to the engines are distributed in a round-robin fashion.

+
+
source_ips :

If you use multiple network interfaces, define from which IP the requests must +be made. Example:

+
    +

  • 0.0.0.0 any local IPv4 address.

    • +

  • :: any local IPv6 address.

    • +

  • 192.168.0.1

    • +

  • [ 192.168.0.1, 192.168.0.2 ] these two specific IP addresses

    • +

  • fe80::60a2:1691:e5a2:ee1f

    • +

  • fe80::60a2:1691:e5a2:ee1f/126 all IP addresses in this network.

    • +

  • [ 192.168.0.1, fe80::/126 ]

    • +
+
+
retries :

Number of retry in case of an HTTP error. On each retry, SearXNG uses an +different proxy and source ip.

+
+
retry_on_http_error :

Retry request on some HTTP status code.

+

Example:

+
    +

  • true : on HTTP status code between 400 and 599.

    • +

  • 403 : on HTTP status code 403.

    • +

  • [403, 429]: on HTTP status code 403 and 429.

    • +
+
+
enable_http2 :

Enable by default. Set to false to disable HTTP/2.

+
+
+
+
verify:$SSL_CERT_FILE, $SSL_CERT_DIR

Allow to specify a path to certificate. +see httpx verification defaults.

+

In addition to verify, SearXNG supports the $SSL_CERT_FILE (for a file) and +$SSL_CERT_DIR (for a directory) OpenSSL variables. +see httpx ssl configuration.

+
+
max_redirects :

30 by default. Maximum redirect before it is an error.

+
+
+
+
+

categories_as_tabs:

+

A list of the categories that are displayed as tabs in the user interface. +Categories not listed here can still be searched with the Search syntax.

+
categories_as_tabs:
+  general:
+  images:
+  videos:
+  news:
+  map:
+  music:
+  it:
+  science:
+  files:
+  social media:
+
+
+
+
+
+

Engine settings

+ +

In the code example below a full fledged example of a YAML setup from a dummy +engine is shown. Most of the options have a default value or even are optional.

+
- name: example engine
+  engine: example
+  shortcut: demo
+  base_url: 'https://{language}.example.com/'
+  send_accept_language_header: false
+  categories: general
+  timeout: 3.0
+  api_key: 'apikey'
+  disabled: false
+  language: en_US
+  tokens: [ 'my-secret-token' ]
+  weigth: 1
+  display_error_messages: true
+  about:
+     website: https://example.com
+     wikidata_id: Q306656
+     official_api_documentation: https://example.com/api-doc
+     use_official_api: true
+     require_api_key: true
+     results: HTML
+  enable_http: false
+  enable_http2: false
+  retries: 1
+  retry_on_http_error: true # or 403 or [404, 429]
+  max_connections: 100
+  max_keepalive_connections: 10
+  keepalive_expiry: 5.0
+  proxies:
+    http:
+      - http://proxy1:8080
+      - http://proxy2:8080
+    https:
+      - http://proxy1:8080
+      - http://proxy2:8080
+      - socks5://user:password@proxy3:1080
+      - socks5h://user:password@proxy4:1080
+
+
+
+
name :

Name that will be used across SearXNG to define this engine. In settings, on +the result page…

+
+
engine :

Name of the python file used to handle requests and responses to and from this +search engine.

+
+
shortcut :

Code used to execute bang requests (in this case using !bi)

+
+
base_urloptional

Part of the URL that should be stable across every request. Can be useful to +use multiple sites using only one engine, or updating the site URL without +touching at the code.

+
+
send_accept_language_header :

Several engines that support languages (or regions) deal with the HTTP header +Accept-Language to build a response that fits to the locale. When this +option is activated, the language (locale) that is selected by the user is used +to build and send a Accept-Language header in the request to the origin +search engine.

+
+
categoriesoptional

Define in which categories this engine will be active. Most of the time, it is +defined in the code of the engine, but in a few cases it is useful, like when +describing multiple search engine using the same code.

+
+
timeoutoptional

Timeout of the search with the current search engine. Be careful, it will +modify the global timeout of SearXNG.

+
+
api_keyoptional

In a few cases, using an API needs the use of a secret key. How to obtain them +is described in the file.

+
+
disabledoptional

To disable by default the engine, but not deleting it. It will allow the user +to manually activate it in the settings.

+
+
languageoptional

If you want to use another language for a specific engine, you can define it +by using the full ISO code of language and country, like fr_FR, en_US, +de_DE.

+
+
tokensoptional

A list of secret tokens to make this engine private, more details see +Private Engines (tokens).

+
+
weigthdefault 1

Weighting of the results of this engine.

+
+
display_error_messagesdefault true

When an engine returns an error, the message is displayed on the user interface.

+
+
networkoptional

Use the network configuration from another engine. +In addition, there are two default networks:

+
    +

  • ipv4 set local_addresses to 0.0.0.0 (use only IPv4 local addresses)

    • +

  • ipv6 set local_addresses to :: (use only IPv6 local addresses)

    • +
+
+
+
+

Note

+

A few more options are possible, but they are pretty specific to some +engines, and so won’t be described here.

+
+ +
+
+

use_default_settings

+ +

The user defined settings.yml is loaded from the settings.yml location +and can relied on the default configuration git://searx/settings.yml using:

+
+

use_default_settings: true

+
+
+
server:

In the following example, the actual settings are the default settings defined +in git://searx/settings.yml with the exception of the secret_key and +the bind_address:

+
use_default_settings: true
+server:
+    secret_key: "ultrasecretkey"   # change this!
+    bind_address: "0.0.0.0"
+
+
+
+
engines:

With use_default_settings: true, each settings can be override in a +similar way, the engines section is merged according to the engine +name. In this example, SearXNG will load all the engine and the arch linux +wiki engine has a token:

+
use_default_settings: true
+server:
+  secret_key: "ultrasecretkey"   # change this!
+engines:
+  - name: arch linux wiki
+    tokens: ['$ecretValue']
+
+
+
+
engines: / remove:

It is possible to remove some engines from the default settings. The following +example is similar to the above one, but SearXNG doesn’t load the the google +engine:

+
use_default_settings:
+  engines:
+    remove:
+      - google
+server:
+  secret_key: "ultrasecretkey"   # change this!
+engines:
+  - name: arch linux wiki
+    tokens: ['$ecretValue']
+
+
+
+
engines: / keep_only:

As an alternative, it is possible to specify the engines to keep. In the +following example, SearXNG has only two engines:

+
use_default_settings:
+  engines:
+    keep_only:
+      - google
+      - duckduckgo
+server:
+  secret_key: "ultrasecretkey"   # change this!
+engines:
+  - name: google
+    tokens: ['$ecretValue']
+  - name: duckduckgo
+    tokens: ['$ecretValue']
+
+
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/engines/sql-engines.html b/admin/engines/sql-engines.html new file mode 100644 index 00000000..8192b2fc --- /dev/null +++ b/admin/engines/sql-engines.html @@ -0,0 +1,289 @@ + + + + + + + + + SQL Engines — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

SQL Engines

+ +

With the SQL engines you can bind SQL databases into SearXNG. The following +Relational Database Management System (RDBMS) are supported:

+ +

All of the engines above are just commented out in the settings.yml, as you have to set the required attributes for the +engines, e.g. database: …

+
- name: ...
+  engine: {sqlite|postgresql|mysql_server}
+  database: ...
+  result_template: {template_name}
+  query_str: ...
+
+
+

By default, the engines use the key-value template for displaying results / +see simple +theme. If you are not satisfied with the original result layout, you can use +your own template, set result_template attribute to {template_name} and +place the templates at:

+
searx/templates/{theme_name}/result_templates/{template_name}
+
+
+

If you do not wish to expose these engines on a public instance, you can still +add them and limit the access by setting tokens as described in section +Private Engines (tokens).

+
+

Configure the engines

+

The configuration of the new database engines are similar. You must put a valid +SQL-SELECT query in query_str. At the moment you can only bind at most one +parameter in your query. By setting the attribute limit you can define how +many results you want from the SQL server. Basically, it is the same as the +LIMIT keyword in SQL.

+

Please, do not include LIMIT or OFFSET in your SQL query as the engines +rely on these keywords during paging. If you want to configure the number of +returned results use the option limit.

+
+

SQLite

+ +

SQLite is a small, fast and reliable SQL database engine. It does not require +any extra dependency. To demonstrate the power of database engines, here is a +more complex example which reads from a MediathekView (DE) movie database. For +this example of the SQlite engine download the database:

+ +

and unpack into searx/data/filmliste-v2.db. To search the database use e.g +Query to test: !mediathekview concert

+
- name: mediathekview
+  engine: sqlite
+  disabled: False
+  categories: general
+  result_template: default.html
+  database: searx/data/filmliste-v2.db
+  query_str:  >-
+    SELECT title || ' (' || time(duration, 'unixepoch') || ')' AS title,
+           COALESCE( NULLIF(url_video_hd,''), NULLIF(url_video_sd,''), url_video) AS url,
+           description AS content
+      FROM film
+     WHERE title LIKE :wildcard OR description LIKE :wildcard
+     ORDER BY duration DESC
+
+
+
+
+

Extra Dependencies

+

For using PostgreSQL or MySQL you need to +install additional packages in Python’s Virtual Environment of your SearXNG +instance. To switch into the environment (Install SearXNG & dependencies) you can use +utils/searxng.sh:

+
$ sudo utils/searxng.sh instance cmd bash
+(searxng-pyenv)$ pip install ...
+
+
+
+
+

PostgreSQL

+ +

PostgreSQL is a powerful and robust open source database. Before configuring +the PostgreSQL engine, you must install the dependency psychopg2. You can +find an example configuration below:

+
- name: my_database
+  engine: postgresql
+  database: my_database
+  username: searxng
+  password: password
+  query_str: 'SELECT * from my_table WHERE my_column = %(query)s'
+
+
+
+
+

MySQL

+ +

MySQL is said to be the most popular open source database. Before enabling MySQL +engine, you must install the package mysql-connector-python.

+

The authentication plugin is configurable by setting auth_plugin in the +attributes. By default it is set to caching_sha2_password. This is an +example configuration for querying a MySQL server:

+
- name: my_database
+  engine: mysql_server
+  database: my_database
+  username: searxng
+  password: password
+  limit: 5
+  query_str: 'SELECT * from my_table WHERE my_column=%(query)s'
+
+
+
+
+
+

Acknowledgment

+

This development was sponsored by Search and Discovery Fund of NLnet Foundation.

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/index.html b/admin/index.html new file mode 100644 index 00000000..22da12c2 --- /dev/null +++ b/admin/index.html @@ -0,0 +1,217 @@ + + + + + + + + + Administrator documentation — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Administrator documentation

+
+

Contents

+ +
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/installation-apache.html b/admin/installation-apache.html new file mode 100644 index 00000000..0f7411c5 --- /dev/null +++ b/admin/installation-apache.html @@ -0,0 +1,478 @@ + + + + + + + + + Apache — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Apache

+

This section explains how to set up a SearXNG instance using the HTTP server Apache. +If you did use the Installation Script and do not have any special preferences +you can install the SearXNG site using +searxng.sh:

+
$ sudo -H ./utils/searxng.sh install apache
+
+
+

If you have special interests or problems with setting up Apache, the following +section might give you some guidance.

+ + +
+

The Apache HTTP server

+

If Apache is not installed, install it now. If apache is new to you, the +Getting Started, Configuration Files and Terms Used to Describe +Directives documentation gives first orientation. There is also a list of +Apache directives to keep in the pocket.

+
+
sudo -H apt-get install apache2
+
+
+
+

Now at http://localhost you should see some kind of Welcome or Test page. +How this default site is configured, depends on the linux distribution +(compare Apache directives).

+
+
less /etc/apache2/sites-enabled/000-default.conf
+
+
+

In this file, there is a line setting the DocumentRoot directive:

+
DocumentRoot /var/www/html
+
+
+

And the welcome page is the HTML file at /var/www/html/index.html.

+
+
+

Debian’s Apache layout

+

Be aware, Debian’s Apache layout is quite different from the standard Apache +configuration. For details look at the apache2.README.Debian +(/usr/share/doc/apache2/README.Debian.gz). Some commands you should know on +Debian:

+ +
+
+

Apache modules

+

To load additional modules, in most distributions you have to un-comment the +lines with the corresponding LoadModule directive, except in Debian’s Apache layout.

+
+

Debian’s Apache layout uses a2enmod and a2dismod to +activate or disable modules:

+
sudo -H a2enmod ssl
+sudo -H a2enmod headers
+sudo -H a2enmod proxy
+sudo -H a2enmod proxy_http
+sudo -H a2enmod proxy_uwsgi
+
+
+
+
+
+

Apache sites

+
+

In Debian’s Apache layout you create a searxng.conf with the +<Location /searxng > directive and save this file in the sites +available folder at /etc/apache2/sites-available. To enable the +searxng.conf use a2ensite:

+
sudo -H a2ensite searxng.conf
+
+
+
+
+
+
+

Apache’s SearXNG site

+ +

To proxy the incoming requests to the SearXNG instance Apache needs the +mod_proxy module (Apache modules).

+ +

Depending on what your SearXNG installation is listening on, you need a http +mod_proxy_http) or socket (mod_proxy_uwsgi) communication to upstream.

+

The Installation Script installs the reference setup and a uWSGI setup that listens on a socket by default. +You can install and activate your own searxng.conf like shown in +Apache sites.

+
+
# -*- coding: utf-8; mode: apache -*-
+
+LoadModule ssl_module           /usr/lib/apache2/modules/mod_ssl.so
+LoadModule headers_module       /usr/lib/apache2/modules/mod_headers.so
+LoadModule proxy_module         /usr/lib/apache2/modules/mod_proxy.so
+LoadModule proxy_uwsgi_module   /usr/lib/apache2/modules/mod_proxy_uwsgi.so
+# LoadModule setenvif_module      /usr/lib/apache2/modules/mod_setenvif.so
+#
+# SetEnvIf Request_URI /searxng dontlog
+# CustomLog /dev/null combined env=dontlog
+
+<Location /searxng>
+
+    Require all granted
+    Order deny,allow
+    Deny from all
+    # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1
+    Allow from all
+
+    # add the trailing slash
+    RedirectMatch  308 /searxng$ /searxng/
+
+    ProxyPreserveHost On
+    ProxyPass unix:/usr/local/searxng/run/socket|uwsgi://uwsgi-uds-searxng/
+
+    # see flaskfix.py
+    RequestHeader set X-Scheme %{REQUEST_SCHEME}s
+    RequestHeader set X-Script-Name /searxng
+
+    # see limiter.py
+    RequestHeader set X-Real-IP %{REMOTE_ADDR}s
+    RequestHeader append X-Forwarded-For %{REMOTE_ADDR}s
+
+</Location>
+
+# uWSGI serves the static files and in settings.yml we use::
+#
+#   ui:
+#     static_use_hash: true
+#
+# Alias /searxng/static/ /usr/local/searxng/searxng-src/searx/static/
+
+
+
+

Restart service:

+
+
sudo -H systemctl restart apache2
+sudo -H service uwsgi restart searxng
+
+
+
+
+
+

disable logs

+

For better privacy you can disable Apache logs. In the examples above activate +one of the lines and restart apache:

+
SetEnvIf Request_URI "/searxng" dontlog
+# CustomLog /dev/null combined env=dontlog
+
+
+

The CustomLog directive disables logs for the entire (virtual) server, use it +when the URL of the service does not have a path component (/searxng), so when +SearXNG is located at root (/).

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/installation-docker.html b/admin/installation-docker.html new file mode 100644 index 00000000..86c4eace --- /dev/null +++ b/admin/installation-docker.html @@ -0,0 +1,325 @@ + + + + + + + + + Docker Container — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Docker Container

+
+ +

If you intend to create a public instance using Docker, use our well maintained +docker container

+ + +

The sources are hosted at searxng-docker and the container includes:

+ +

The default SearXNG setup +of this container:

+ +
+
+

Get Docker

+

If you plan to build and maintain a docker image by yourself, make sure you have +Docker installed. On Linux don’t +forget to add your user to the docker group (log out and log back in so that +your group membership is re-evaluated):

+
$ sudo usermod -a -G docker $USER
+
+
+
+
+

searxng/searxng

+ +

The docker image is based on git://Dockerfile and available from +searxng/searxng @dockerhub. Using the docker image is quite easy, for +instance you can pull the searxng/searxng @dockerhub image and deploy a local +instance using docker run:

+
$ mkdir my-instance
+$ cd my-instance
+$ export PORT=8080
+$ docker pull searxng/searxng
+$ docker run --rm \
+             -d -p ${PORT}:8080 \
+             -v "${PWD}/searxng:/etc/searxng" \
+             -e "BASE_URL=http://localhost:$PORT/" \
+             -e "INSTANCE_NAME=my-instance" \
+             searxng/searxng
+2f998.... # container's ID
+
+
+

Open your WEB browser and visit the URL:

+
$ xdg-open "http://localhost:$PORT"
+
+
+

Inside ${PWD}/searxng, you will find settings.yml and uwsgi.ini. You +can modify these files according to your needs and restart the Docker image.

+
$ docker container restart 2f998
+
+
+

Use command container ls to list running containers, add flag -a to list +exited containers also. With container stop a running container can be +stoped. To get rid of a container use container rm:

+
$ docker container ls
+CONTAINER ID   IMAGE             COMMAND                  CREATED         ...
+2f998d725993   searxng/searxng   "/sbin/tini -- /usr/…"   7 minutes ago   ...
+
+$ docker container stop 2f998
+$ docker container rm 2f998
+
+
+ +

If you won’t use docker anymore and want to get rid of all conatiners & images +use the following prune command:

+
$ docker stop $(docker ps -aq)       # stop all containers
+$ docker system prune                # make some housekeeping
+$ docker rmi -f $(docker images -q)  # drop all images
+
+
+
+

shell inside container

+ +

Like in many other distributions, Alpine’s /bin/sh is dash. Dash is meant to be +POSIX-compliant. +Compared to debian, in the Alpine image bash is not installed. The +git://dockerfiles/docker-entrypoint.sh script is checked against dash +(make tests.shell).

+

To open a shell inside the container:

+
$ docker exec -it 2f998 sh
+
+
+
+
+
+

Build the image

+

It’s also possible to build SearXNG from the embedded git://Dockerfile:

+
$ git clone https://github.com/searxng/searxng.git
+$ cd searxng
+$ make docker.build
+...
+Successfully built 49586c016434
+Successfully tagged searxng/searxng:latest
+Successfully tagged searxng/searxng:1.0.0-209-9c823800-dirty
+
+$ docker images
+REPOSITORY        TAG                        IMAGE ID       CREATED          SIZE
+searxng/searxng   1.0.0-209-9c823800-dirty   49586c016434   13 minutes ago   308MB
+searxng/searxng   latest                     49586c016434   13 minutes ago   308MB
+alpine            3.13                       6dbb9cc54074   3 weeks ago      5.61MB
+
+
+
+
+

Command line

+ +

In the git://Dockerfile the ENTRYPOINT is defined as +git://dockerfiles/docker-entrypoint.sh

+
docker run --rm -it searxng/searxng -h
+
+
+
Command line:
+  -h  Display this help
+  -d  Dry run to update the configuration files.
+  -f  Always update on the configuration files (existing files are renamed with
+      the .old suffix).  Without this option, the new configuration files are
+      copied with the .new suffix
+Environment variables:
+  INSTANCE_NAME settings.yml : general.instance_name
+  AUTOCOMPLETE  settings.yml : search.autocomplete
+  BASE_URL      settings.yml : server.base_url
+  MORTY_URL     settings.yml : result_proxy.url
+  MORTY_KEY     settings.yml : result_proxy.key
+  BIND_ADDRESS  uwsgi bind to the specified TCP socket using HTTP protocol.
+                Default value: 0.0.0.0:8080
+Volume:
+  /etc/searxng  the docker entry point copies settings.yml and uwsgi.ini in
+                this directory (see the -f command line option)"
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/installation-nginx.html b/admin/installation-nginx.html new file mode 100644 index 00000000..988014fb --- /dev/null +++ b/admin/installation-nginx.html @@ -0,0 +1,361 @@ + + + + + + + + + NGINX — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

NGINX

+

This section explains how to set up a SearXNG instance using the HTTP server nginx. +If you have used the Installation Script and do not have any special preferences +you can install the SearXNG site using +searxng.sh:

+
$ sudo -H ./utils/searxng.sh install nginx
+
+
+

If you have special interests or problems with setting up nginx, the following +section might give you some guidance.

+ + +
+

The nginx HTTP server

+

If nginx is not installed, install it now.

+
+
sudo -H apt-get install nginx
+
+
+
+

Now at http://localhost you should see a Welcome to nginx! page, on Fedora you +see a Fedora Webserver - Test Page. The test page comes from the default +nginx server configuration. How this default site is configured, +depends on the linux distribution:

+
+
less /etc/nginx/nginx.conf
+
+
+

There is one line that includes site configurations from:

+
include /etc/nginx/sites-enabled/*;
+
+
+
+
+
+

NGINX’s SearXNG site

+

Now you have to create a configuration file (searxng.conf) for the SearXNG +site. If nginx is new to you, the nginx beginners guide is a good starting +point and the Getting Started wiki is always a good resource to keep in the +pocket.

+

Depending on what your SearXNG installation is listening on, you need a http or socket +communication to upstream.

+
+
location /searxng {
+
+    uwsgi_pass unix:///usr/local/searxng/run/socket;
+
+    include uwsgi_params;
+
+    uwsgi_param    HTTP_HOST             $host;
+    uwsgi_param    HTTP_CONNECTION       $http_connection;
+
+    # see flaskfix.py
+    uwsgi_param    HTTP_X_SCHEME         $scheme;
+    uwsgi_param    HTTP_X_SCRIPT_NAME    /searxng;
+
+    # see limiter.py
+    uwsgi_param    HTTP_X_REAL_IP        $remote_addr;
+    uwsgi_param    HTTP_X_FORWARDED_FOR  $proxy_add_x_forwarded_for;
+}
+
+# uWSGI serves the static files and in settings.yml we use::
+#
+#   ui:
+#     static_use_hash: true
+#
+# location /searxng/static/ {
+#     alias /usr/local/searxng/searxng-src/searx/static/;
+# }
+
+
+
+

The Installation Script installs the reference setup and a uWSGI setup that listens on a socket by default.

+
+

Create configuration at /etc/nginx/sites-available/ and place a +symlink to sites-enabled:

+
sudo -H ln -s /etc/nginx/sites-available/searxng.conf \
+              /etc/nginx/sites-enabled/searxng.conf
+
+
+
+

Restart services:

+
+
sudo -H systemctl restart nginx
+sudo -H service uwsgi restart searxng
+
+
+
+
+
+

Disable logs

+

For better privacy you can disable nginx logs in /etc/nginx/nginx.conf.

+
http {
+    # ...
+    access_log /dev/null;
+    error_log  /dev/null;
+    # ...
+}
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/installation-scripts.html b/admin/installation-scripts.html new file mode 100644 index 00000000..e2ae162a --- /dev/null +++ b/admin/installation-scripts.html @@ -0,0 +1,200 @@ + + + + + + + + + Installation Script — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Installation Script

+ +

The following will install a setup as shown in the reference architecture. First you need to get a clone of the repository. The clone is only needed for +the installation procedure and some maintenance tasks.

+ +

Jump to a folder that is readable by others and start to clone SearXNG, +alternatively you can create your own fork and clone from there.

+
$ cd ~/Downloads
+$ git clone https://github.com/searxng/searxng.git searxng
+$ cd searxng
+
+
+ +

To install a SearXNG reference setup +including a uWSGI setup as described in the +Step by step installation and in the uWSGI section type:

+
$ sudo -H ./utils/searxng.sh install all
+
+
+
+

Attention

+

For the installation procedure, use a sudoer login to run the scripts. If +you install from root, take into account that the scripts are creating a +searxng user. In the installation procedure this new created user does +need read access to the cloned SearXNG repository, which is not the case if you clone +it into a folder below /root!

+
+ +

When all services are installed and running fine, you can add SearXNG to your +HTTP server. We do not have any preferences for the HTTP server, you can use +whatever you prefer.

+

We use caddy in our docker image and we have +implemented installation procedures for:

+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/installation-searxng.html b/admin/installation-searxng.html new file mode 100644 index 00000000..25457478 --- /dev/null +++ b/admin/installation-searxng.html @@ -0,0 +1,597 @@ + + + + + + + + + Step by step installation — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Step by step installation

+ +

In this section we show the setup of a SearXNG instance that will be installed +by the Installation Script.

+
+

Install packages

+
+
$ sudo -H apt-get install -y \
+    python3-dev python3-babel python3-venv \
+    uwsgi uwsgi-plugin-python3 \
+    git build-essential libxslt-dev zlib1g-dev libffi-dev libssl-dev
+
+
+
+
+

Hint

+

This installs also the packages needed by uWSGI

+
+
+
+

Create user

+
+
$ sudo -H useradd --shell /bin/bash --system \
+    --home-dir "/usr/local/searxng" \
+    --comment 'Privacy-respecting metasearch engine' \
+    searxng
+
+$ sudo -H mkdir "/usr/local/searxng"
+$ sudo -H chown -R "searxng:searxng" "/usr/local/searxng"
+
+
+
+
+
+

Install SearXNG & dependencies

+

Start a interactive shell from new created user and clone SearXNG:

+
+
$ sudo -H -u searxng -i
+(searxng)$ git clone "https://github.com/searxng/searxng" \
+                   "/usr/local/searxng/searxng-src"
+
+
+
+

In the same shell create virtualenv:

+
+
(searxng)$ python3 -m venv "/usr/local/searxng/searx-pyenv"
+(searxng)$ echo ". /usr/local/searxng/searx-pyenv/bin/activate" \
+                   >>  "/usr/local/searxng/.profile"
+
+
+
+

To install SearXNG’s dependencies, exit the SearXNG bash session you opened above +and start a new one. Before installing, check if your virtualenv was sourced +from the login (~/.profile):

+
+
$ sudo -H -u searxng -i
+
+(searxng)$ command -v python && python --version
+/usr/local/searxng/searx-pyenv/bin/python
+Python 3.8.1
+
+# update pip's boilerplate ..
+pip install -U pip
+pip install -U setuptools
+pip install -U wheel
+pip install -U pyyaml
+
+# jump to SearXNG's working tree and install SearXNG into virtualenv
+(searxng)$ cd "/usr/local/searxng/searxng-src"
+(searxng)$ pip install -e .
+
+
+
+
+

Tip

+

Open a second terminal for the configuration tasks and leave the (searx)$ +terminal open for the tasks below.

+
+
+
+

Configuration

+ +

To create a initial /etc/searxng/settings.yml we recommend to start with a +copy of the file git://utils/templates/etc/searxng/settings.yml. This setup +use default settings from +git://searx/settings.yml and is shown in the tab “Use default settings” +below. This setup:

+ +

Modify the /etc/searxng/settings.yml to your needs:

+
+
# SearXNG settings
+
+use_default_settings: true
+
+general:
+  debug: false
+  instance_name: "SearXNG"
+
+search:
+  safe_search: 2
+  autocomplete: 'duckduckgo'
+
+server:
+  secret_key: "ultrasecretkey"
+  limiter: true
+  image_proxy: true
+
+redis:
+  url: unix:///usr/local/searxng-redis/run/redis.sock?db=0
+
+ui:
+  static_use_hash: true
+
+# preferences:
+#   lock:
+#     - autocomplete
+#     - method
+
+enabled_plugins:
+  - 'Hash plugin'
+  - 'Search on category select'
+  - 'Self Informations'
+  - 'Tracker URL remover'
+  - 'Ahmia blacklist'
+  # - 'Hostname replace'  # see hostname_replace configuration below
+  # - 'Infinite scroll'
+  # - 'Open Access DOI rewrite'
+  # - 'Vim-like hotkeys'
+
+# plugins:
+#   - only_show_green_results
+
+
+

To see the entire file jump to git://utils/templates/etc/searxng/settings.yml

+
+

For a minimal setup you need to set server:secret_key.

+
+
$ sudo -H mkdir -p "/etc/searxng"
+$ sudo -H cp "/usr/local/searxng/searxng-src/utils/templates/etc/searxng/settings.yml" \
+             "/etc/searxng/settings.yml"
+
+
+
+
+
+

Check

+

To check your SearXNG setup, optional enable debugging and start the webapp. +SearXNG looks at the exported environment $SEARXNG_SETTINGS_PATH for a +configuration file.

+
+
# enable debug ..
+$ sudo -H sed -i -e "s/debug : False/debug : True/g" "/etc/searxng/settings.yml"
+
+# start webapp
+$ sudo -H -u searxng -i
+(searxng)$ cd /usr/local/searxng/searxng-src
+(searxng)$ export SEARXNG_SETTINGS_PATH="/etc/searxng/settings.yml"
+(searxng)$ python searx/webapp.py
+
+# disable debug
+$ sudo -H sed -i -e "s/debug : True/debug : False/g" "/etc/searxng/settings.yml"
+
+
+
+

Open WEB browser and visit http://127.0.0.1:8888 . If you are inside a +container or in a script, test with curl:

+
+
$ xdg-open http://127.0.0.1:8888
+
+
+
+

If everything works fine, hit [CTRL-C] to stop the webapp and disable the +debug option in settings.yml. You can now exit SearXNG user bash session (enter exit +command twice). At this point SearXNG is not demonized; uwsgi allows this.

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/installation-uwsgi.html b/admin/installation-uwsgi.html new file mode 100644 index 00000000..b5e0a716 --- /dev/null +++ b/admin/installation-uwsgi.html @@ -0,0 +1,637 @@ + + + + + + + + + uWSGI — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

uWSGI

+ + +
+

Origin uWSGI

+

How uWSGI is implemented by distributors varies. The uWSGI project itself +recommends two methods:

+
    +

  1. systemd.unit template file as described here One service per app in systemd:

    2. +
+
+

There is one systemd unit template on the system installed and one uwsgi +ini file per uWSGI-app placed at dedicated locations. Take archlinux and a +searxng.ini as example:

+
systemd template unit: /usr/lib/systemd/system/uwsgi@.service
+        contains: [Service]
+                  ExecStart=/usr/bin/uwsgi --ini /etc/uwsgi/%I.ini
+
+SearXNG application:   /etc/uwsgi/searxng.ini
+        links to: /etc/uwsgi/apps-available/searxng.ini
+
+
+

The SearXNG app (template /etc/uwsgi/%I.ini) can be maintained as known +from common systemd units:

+
$ systemctl enable  uwsgi@searxng
+$ systemctl start   uwsgi@searxng
+$ systemctl restart uwsgi@searxng
+$ systemctl stop    uwsgi@searxng
+
+
+
+
    +

  1. The uWSGI Emperor which fits for maintaining a large range of uwsgi +apps and there is a Tyrant mode to secure multi-user hosting.

    2. +
+
+

The Emperor mode is a special uWSGI instance that will monitor specific +events. The Emperor mode (the service) is started by a (common, not template) +systemd unit.

+

The Emperor service will scan specific directories for uwsgi ini files +(also know as vassals). If a vassal is added, removed or the timestamp is +modified, a corresponding action takes place: a new uWSGI instance is started, +reload or stopped. Take Fedora and a searxng.ini as example:

+
to install & start SearXNG instance create --> /etc/uwsgi.d/searxng.ini
+to reload the instance edit timestamp      --> touch /etc/uwsgi.d/searxng.ini
+to stop instance remove ini                --> rm /etc/uwsgi.d/searxng.ini
+
+
+
+
+
+

Distributors

+

The uWSGI Emperor mode and systemd unit template is what the distributors +mostly offer their users, even if they differ in the way they implement both +modes and their defaults. Another point they might differ in is the packaging of +plugins (if so, compare Install packages) and what the default python +interpreter is (python2 vs. python3).

+

While archlinux does not start a uWSGI service by default, Fedora (RHEL) starts +a Emperor in Tyrant mode by default (you should have read Pitfalls of the Tyrant mode). Worth to know; debian (ubuntu) follow a complete different +approach, read see Debian’s uWSGI layout.

+
+

Debian’s uWSGI layout

+

Be aware, Debian’s uWSGI layout is quite different from the standard uWSGI +configuration. Your are familiar with Debian’s Apache layout? .. they do a +similar thing for the uWSGI infrastructure. The folders are:

+
/etc/uwsgi/apps-available/
+/etc/uwsgi/apps-enabled/
+
+
+

The uwsgi ini file is enabled by a symbolic link:

+
ln -s /etc/uwsgi/apps-available/searxng.ini /etc/uwsgi/apps-enabled/
+
+
+

More details can be found in the uwsgi.README.Debian +(/usr/share/doc/uwsgi/README.Debian.gz). Some commands you should know on +Debian:

+
Commands recognized by init.d script
+====================================
+
+You can issue to init.d script following commands:
+  * start        | starts daemon
+  * stop         | stops daemon
+  * reload       | sends to daemon SIGHUP signal
+  * force-reload | sends to daemon SIGTERM signal
+  * restart      | issues 'stop', then 'start' commands
+  * status       | shows status of daemon instance (running/not running)
+
+'status' command must be issued with exactly one argument: '<confname>'.
+
+Controlling specific instances of uWSGI
+=======================================
+
+You could control specific instance(s) by issuing:
+
+    SYSTEMCTL_SKIP_REDIRECT=1 service uwsgi <command> <confname> <confname>...
+
+where:
+  * <command> is one of 'start', 'stop' etc.
+  * <confname> is the name of configuration file (without extension)
+
+For example, this is how instance for /etc/uwsgi/apps-enabled/hello.xml is
+started:
+
+    SYSTEMCTL_SKIP_REDIRECT=1 service uwsgi start hello
+
+
+
+
+
+

uWSGI maintenance

+
+
# init.d --> /usr/share/doc/uwsgi/README.Debian.gz
+# For uWSGI debian uses the LSB init process, this might be changed
+# one day, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=833067
+
+create     /etc/uwsgi/apps-available/searxng.ini
+enable:    sudo -H ln -s /etc/uwsgi/apps-available/searxng.ini /etc/uwsgi/apps-enabled/
+start:     sudo -H service uwsgi start   searxng
+restart:   sudo -H service uwsgi restart searxng
+stop:      sudo -H service uwsgi stop    searxng
+disable:   sudo -H rm /etc/uwsgi/apps-enabled/searxng.ini
+
+
+
+
+
+

uWSGI setup

+

Create the configuration ini-file according to your distribution and restart the +uwsgi application. As shown below, the Installation Script installs by +default:

+
    +

  • a uWSGI setup that listens on a socket and

    • +

  • enables cache busting.

    • +
+
+
# -*- mode: conf; coding: utf-8  -*-
+[uwsgi]
+
+# uWSGI core
+# ----------
+#
+# https://uwsgi-docs.readthedocs.io/en/latest/Options.html#uwsgi-core
+
+# Who will run the code / Hint: in emperor-tyrant mode uid & gid setting will be
+# ignored [1].  Mode emperor-tyrant is the default on fedora (/etc/uwsgi.ini).
+#
+# [1] https://uwsgi-docs.readthedocs.io/en/latest/Emperor.html#tyrant-mode-secure-multi-user-hosting
+#
+uid = searxng
+gid = searxng
+
+# set (python) default encoding UTF-8
+env = LANG=C.UTF-8
+env = LANGUAGE=C.UTF-8
+env = LC_ALL=C.UTF-8
+
+# chdir to specified directory before apps loading
+chdir = /usr/local/searxng/searxng-src/searx
+
+# SearXNG configuration (settings.yml)
+env = SEARXNG_SETTINGS_PATH=/etc/searxng/settings.yml
+
+# disable logging for privacy
+disable-logging = true
+
+# The right granted on the created socket
+chmod-socket = 666
+
+# Plugin to use and interpreter config
+single-interpreter = true
+
+# enable master process
+master = true
+
+# load apps in each worker instead of the master
+lazy-apps = true
+
+# load uWSGI plugins
+plugin = python3,http
+
+# By default the Python plugin does not initialize the GIL.  This means your
+# app-generated threads will not run.  If you need threads, remember to enable
+# them with enable-threads.  Running uWSGI in multithreading mode (with the
+# threads options) will automatically enable threading support. This *strange*
+# default behaviour is for performance reasons.
+enable-threads = true
+
+
+# plugin: python
+# --------------
+#
+# https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-python
+
+# load a WSGI module
+module = searx.webapp
+
+# set PYTHONHOME/virtualenv
+virtualenv = /usr/local/searxng/searx-pyenv
+
+# add directory (or glob) to pythonpath
+pythonpath = /usr/local/searxng/searxng-src
+
+
+# speak to upstream
+# -----------------
+
+socket = /usr/local/searxng/run/socket
+buffer-size = 8192
+
+# uWSGI serves the static files and in settings.yml we use::
+#
+#   ui:
+#     static_use_hash: true
+#
+static-map = /static=/usr/local/searxng/searxng-src/searx/static
+# expires set to one year since there are hashes
+static-expires = /* 31557600
+static-gzip-all = True
+offload-threads = %k
+
+# Cache
+cache2 = name=searxngcache,items=2000,blocks=2000,blocksize=4096,bitmap=1
+
+
+
+
+
+

Pitfalls of the Tyrant mode

+

The implementation of the process owners and groups in the Tyrant mode is +somewhat unusual and requires special consideration. In Tyrant mode mode the +Emperor will run the vassal using the UID/GID of the vassal configuration file +(user and group of the app .ini file).

+

Without option emperor-tyrant-initgroups=true in /etc/uwsgi.ini the +process won’t get the additional groups, but this option is not available in +2.0.x branch (see #2099@uWSGI) the feature #752@uWSGI has been merged (on +Oct. 2014) to the master branch of uWSGI but had never been released; the last +major release is from Dec. 2013, since the there had been only bugfix releases +(see #2425uWSGI). To shorten up:

+
+

In Tyrant mode, there is no way to get additional groups, and the uWSGI +process misses additional permissions that may be needed.

+
+

For example on Fedora (RHEL): If you try to install a redis DB with socket +communication and you want to connect to it from the SearXNG uWSGI, you will see a +Permission denied in the log of your instance:

+
ERROR:searx.redisdb: [searxng (993)] can't connect redis DB ...
+ERROR:searx.redisdb:   Error 13 connecting to unix socket: /usr/local/searxng-redis/run/redis.sock. Permission denied.
+ERROR:searx.plugins.limiter: init limiter DB failed!!!
+
+
+

Even if your searxng user of the uWSGI process is added to additional groups +to give access to the socket from the redis DB:

+
$ groups searxng
+searxng : searxng searxng-redis
+
+
+

To see the effective groups of the uwsgi process, you have to look at the status +of the process, by example:

+
$ ps -aef | grep '/usr/sbin/uwsgi --ini searxng.ini'
+searxng       93      92  0 12:43 ?        00:00:00 /usr/sbin/uwsgi --ini searxng.ini
+searxng      186      93  0 12:44 ?        00:00:01 /usr/sbin/uwsgi --ini searxng.ini
+
+
+

Here you can see that the additional “Groups” of PID 186 are unset (missing gid +of searxng-redis):

+
$ cat /proc/186/task/186/status
+...
+Uid:      993     993     993     993
+Gid:      993     993     993     993
+FDSize:   128
+Groups:
+...
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/installation.html b/admin/installation.html new file mode 100644 index 00000000..bbcbc409 --- /dev/null +++ b/admin/installation.html @@ -0,0 +1,161 @@ + + + + + + + + + Installation — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Installation

+

You’re spoilt for choice, choose your preferred method of installation.

+ +

The Step by step installation is an excellent illustration of how a SearXNG +instance is build up (see uWSGI Setup). If you do not have any +special preferences, its recommend to use the Docker Container or the +Installation Script.

+
+

Attention

+

SearXNG is growing rapidly, you should regularly read our Migrate and stay tuned! section. If you want to upgrade an existing instance or migrate +from searx to SearXNG, you should read this section first!

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/plugins.html b/admin/plugins.html new file mode 100644 index 00000000..3b26dcf3 --- /dev/null +++ b/admin/plugins.html @@ -0,0 +1,212 @@ + + + + + + + + + Plugins builtin — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Plugins builtin

+ +

Configuration defaults (at built time):

+
+
DO:
+

Default on

+
+
+ + +++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 1 Plugins configured at built time (defaults)

Name

DO

Description

+

JS & CSS dependencies

+

Autodetect search language

Automatically detect the query search language and switch to it.

Hash plugin

y

Converts strings to different hash digests.

Hostname replace

Rewrite result hostnames or remove results based on the hostname

Open Access DOI rewrite

Avoid paywalls by redirecting to open-access versions of publications when available

Search on category select

y

Perform search immediately if a category selected. Disable to select multiple categories. (JavaScript required)

Self Information

y

Displays your IP if the query is “ip” and your user agent if the query contains “user agent”.

Tor check plugin

This plugin checks if the address of the request is a TOR exit node, and informs the user if it is, like check.torproject.org but from searxng.

Tracker URL remover

y

Remove trackers arguments from the returned URL

Vim-like hotkeys

Navigate search results with Vim-like hotkeys (JavaScript required). Press “h” key on main or result page to get help.

+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/admin/update-searxng.html b/admin/update-searxng.html new file mode 100644 index 00000000..91022462 --- /dev/null +++ b/admin/update-searxng.html @@ -0,0 +1,272 @@ + + + + + + + + + SearXNG maintenance — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

SearXNG maintenance

+ + +
+

How to update

+

How to update depends on the Installation method. If you have used the +Installation Script, use the update command from the utils/searxng.sh +script.

+
sudo -H ./utils/searxng.sh instance update
+
+
+
+
+

How to inspect & debug

+

How to debug depends on the Installation method. If you have used the +Installation Script, use the inspect command from the utils/searxng.sh +script.

+
sudo -H ./utils/searxng.sh instance inspect
+
+
+
+
+

Migrate and stay tuned!

+ +

SearXNG is a rolling release; each commit to the master branch is a release. +SearXNG is growing rapidly, the services and opportunities are change every now +and then, to name just a few:

+
    +

  • Bot protection has been switched from filtron to SearXNG’s limiter, this requires a Redis database.

    • +

  • The image proxy morty is no longer needed, it has been replaced by the +image proxy from SearXNG.

    • +

  • To save bandwith cache busting has been implemented. +To get in use, the static-expires needs to be set in the uWSGI setup.

    • +
+

To stay tuned and get in use of the new features, instance maintainers have to +update the SearXNG code regularly (see How to update). As the above +examples show, this is not always enough, sometimes services have to be set up +or reconfigured and sometimes services that are no longer needed should be +uninstalled.

+
+

Hint

+

First of all: SearXNG is installed by the script utils/searxng.sh. If you +have old filtron, morty or searx setup you should consider complete +uninstall/reinstall.

+
+

Here you will find a list of changes that affect the infrastructure. Please +check to what extent it is necessary to update your installations:

+
+
PR 1595: [fix] uWSGI: increase buffer-size

Re-install uWSGI (utils/searxng.sh) or fix your uWSGI searxng.ini +file manually.

+
+
+
+

remove obsolete services

+

If your searx instance was installed “Step by step” or by the “Installation +scripts”, you need to undo the installation procedure completely. If you have +morty & filtron installed, it is recommended to uninstall these services also. +In case of scripts, to uninstall use the scripts from the origin you installed +searx from or try:

+
$ sudo -H ./utils/filtron.sh remove all
+$ sudo -H ./utils/morty.sh   remove all
+$ sudo -H ./utils/searx.sh   remove all
+
+
+
+

Hint

+

If you are migrate from searx take into account that the .config.sh is no +longer used.

+
+

If you upgrade from searx or from before PR 1332 has been merged and you +have filtron and/or morty installed, don’t forget to remove HTTP sites.

+

Apache:

+
$ sudo -H ./utils/filtron.sh apache remove
+$ sudo -H ./utils/morty.sh apache remove
+
+
+

nginx:

+
$ sudo -H ./utils/filtron.sh nginx remove
+$ sudo -H ./utils/morty.sh nginx remove
+
+
+
+
+

Check after Installation

+

Once you have done your installation, you can run a SearXNG check procedure, +to see if there are some left overs. In this example there exists a old +/etc/searx/settings.yml:

+
$ sudo -H ./utils/searxng.sh instance check
+
+SearXNG checks
+--------------
+ERROR: settings.yml in /etc/searx/ is deprecated, move file to folder /etc/searxng/
+INFO:  [OK] (old) account 'searx' does not exists
+INFO:  [OK] (old) account 'filtron' does not exists
+INFO:  [OK] (old) account 'morty' does not exists
+...
+INFO    searx.redisdb                 : connecting to Redis db=0 path='/usr/local/searxng-redis/run/redis.sock'
+INFO    searx.redisdb                 : connected to Redis
+
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/contribution_guide.html b/dev/contribution_guide.html new file mode 100644 index 00000000..4245bf7e --- /dev/null +++ b/dev/contribution_guide.html @@ -0,0 +1,295 @@ + + + + + + + + + How to contribute — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

How to contribute

+ +
+

Prime directives: Privacy, Hackability

+

SearXNG has two prime directives, privacy-by-design and hackability . The +hackability comes in three levels:

+
    +

  • support of search engines

    • +

  • plugins to alter search behaviour

    • +

  • hacking SearXNG itself

    • +
+

Note the lack of “world domination” among the directives. SearXNG has no +intention of wide mass-adoption, rounded corners, etc. The prime directive +“privacy” deserves a separate chapter, as it’s quite uncommon unfortunately.

+
+

Privacy-by-design

+

SearXNG was born out of the need for a privacy-respecting search tool which +can be extended easily to maximize both, its search and its privacy protecting +capabilities.

+

A few widely used features work differently or turned off by default or not +implemented at all as a consequence of privacy-by-design.

+

If a feature reduces the privacy preserving aspects of searx, it should be +switched off by default or should not implemented at all. There are plenty of +search engines already providing such features. If a feature reduces the +protection of searx, users must be informed about the effect of choosing to +enable it. Features that protect privacy but differ from the expectations of +the user should also be explained.

+

Also, if you think that something works weird with searx, it’s might be because +of the tool you use is designed in a way to interfere with the privacy respect. +Submitting a bugreport to the vendor of the tool that misbehaves might be a good +feedback to reconsider the disrespect to its customers (e.g. GET vs POST +requests in various browsers).

+

Remember the other prime directive of SearXNG is to be hackable, so if the above +privacy concerns do not fancy you, simply fork it.

+
+

Happy hacking.

+
+
+
+
+

Code

+ +

In order to submit a patch, please follow the steps below:

+
    +

  • Follow coding conventions.

    +
      +

    • PEP8 standards apply, except the convention of line length

      • +

    • Maximum line length is 120 characters

      • +
    +
    • +

  • The cardinal rule for creating good commits is to ensure there is only one +logical change per commit / read Structural split of changes

    • +

  • Check if your code breaks existing tests. If so, update the tests or fix your +code.

    • +

  • If your code can be unit-tested, add unit tests.

    • +

  • Add yourself to the git://AUTHORS.rst file.

    • +

  • Choose meaningful commit messages, read Conventional Commits

    +
    <type>[optional scope]: <description>
+
+[optional body]
+
+[optional footer(s)]
+
    +
    +
    • +

  • Create a pull request.

    • +
+

For more help on getting started with SearXNG development, see Development Quickstart.

+
+
+

Translation

+

Translation currently takes place on weblate.

+
+
+

Documentation

+ +

The documentation is built using Sphinx. So in order to be able to generate +the required files, you have to install it on your system. Much easier, use +our Makefile.

+

Here is an example which makes a complete rebuild:

+
$ make docs.clean docs.html
+...
+The HTML pages are in dist/docs.
+
+
+
+

live build

+ +

Live build is like WYSIWYG. If you want to edit the documentation, its +recommended to use. The Makefile target docs.live builds the docs, opens +URL in your favorite browser and rebuilds every time a reST file has been +changed.

+
$ make docs.live
+...
+The HTML pages are in dist/docs.
+... Serving on http://0.0.0.0:8000
+... Start watching changes
+
+
+

Live builds are implemented by sphinx-autobuild. Use environment +$(SPHINXOPTS) to pass arguments to the sphinx-autobuild command. Except +option --host (which is always set to 0.0.0.0) you can pass any +argument. E.g to find and use a free port, use:

+
$ SPHINXOPTS="--port 0" make docs.live
+...
+... Serving on http://0.0.0.0:50593
+...
+
+
+
+
+

deploy on github.io

+

To deploy documentation at github.io use Makefile target make docs.gh-pages, which builds the documentation and runs all the needed git add, +commit and push:

+
$ make docs.clean docs.gh-pages
+
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/engine_overview.html b/dev/engine_overview.html new file mode 100644 index 00000000..38796a2f --- /dev/null +++ b/dev/engine_overview.html @@ -0,0 +1,777 @@ + + + + + + + + + Engine Overview — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Engine Overview

+ + +

SearXNG is a metasearch-engine, so it uses different search engines to provide +better results.

+

Because there is no general search API which could be used for every search +engine, an adapter has to be built between SearXNG and the external search +engines. Adapters are stored under the folder git://searx/engines.

+
+

General Engine Configuration

+

It is required to tell SearXNG the type of results the engine provides. The +arguments can be set in the engine file or in the settings file (normally +settings.yml). The arguments in the settings file override the ones in the +engine file.

+

It does not matter if an option is stored in the engine file or in the settings. +However, the standard way is the following:

+
+

Engine File

+ + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 2 Common options in the engine module

argument

type

information

categories

list

pages, in which the engine is working

paging

boolean

support multiple pages

time_range_support

boolean

support search time range

engine_type

str

    +

  • online [ref] by +default, other possibles values are:

    • +

  • offline [ref]

    • +

  • online_dictionary

    • +

  • online_currency

    • +
+
+
+
+

Engine settings.yml

+

For a more detailed description, see Engine settings in the settings.yml.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 3 Common options in the engine setup (settings.yml)

argument

type

information

name

string

name of search-engine

engine

string

name of searxng-engine (file name without .py)

enable_http

bool

enable HTTP (by default only HTTPS is enabled).

shortcut

string

shortcut of search-engine

timeout

string

specific timeout for search-engine

display_error_messages

boolean

display error messages on the web UI

proxies

dict

set proxies for a specific engine +(e.g. proxies : {http: socks5://proxy:port, +https: socks5://proxy:port})

+
+
+

Overrides

+

A few of the options have default values in the namespace of engine’s python +modul, but are often overwritten by the settings. If None is assigned to an +option in the engine file, it has to be redefined in the settings, otherwise +SearXNG will not start with that engine (global names with a leading underline can +be None).

+

Here is an very simple example of the global names in the namespace of engine’s +module:

+
# engine dependent config
+categories = ['general']
+paging = True
+_non_overwritten_global = 'foo'
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 4 The naming of overrides is arbitrary / recommended overrides are:

argument

type

information

base_url

string

base-url, can be overwritten to use same +engine on other URL

number_of_results

int

maximum number of results per request

language

string

ISO code of language and country like en_US

api_key

string

api-key if required by engine

+
+
+
+

Making a Request

+

To perform a search an URL have to be specified. In addition to specifying an +URL, arguments can be passed to the query.

+
+

Passed Arguments (request)

+

These arguments can be used to construct the search query. Furthermore, +parameters with default value can be redefined for special purposes.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 5 If the engine_type is online

argument

type

default-value, information

url

str

''

method

str

'GET'

headers

set

{}

data

set

{}

cookies

set

{}

verify

bool

True

headers.User-Agent

str

a random User-Agent

category

str

current category, like 'general'

safesearch

int

0, between 0 and 2 (normal, moderate, strict)

time_range

Optional[str]

None, can be day, week, month, year

pageno

int

current pagenumber

language

str

specific language code like 'en_US', or 'all' if unspecified

+ + + + + + + + + + + + + + + + + + + + + + +
Table 6 If the engine_type is online_dictionary, in addition to the + online arguments:

argument

type

default-value, information

from_lang

str

specific language code like 'en_US'

to_lang

str

specific language code like 'en_US'

query

str

the text query without the languages

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 7 If the engine_type is online_currency`, in addition to the + online arguments:

argument

type

default-value, information

amount

float

the amount to convert

from

str

ISO 4217 code

to

str

ISO 4217 code

from_name

str

currency name

to_name

str

currency name

+
+
+

Specify Request

+

The function def request(query, params): always returns the params variable, the +following parameters can be used to specify a search request:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

argument

type

information

url

str

requested url

method

str

HTTP request method

headers

set

HTTP header information

data

set

HTTP data information

cookies

set

HTTP cookies

verify

bool

Performing SSL-Validity check

allow_redirects

bool

Follow redirects

max_redirects

int

maximum redirects, hard limit

soft_max_redirects

int

maximum redirects, soft limit. Record an error but don’t stop the engine

raise_for_httperror

bool

True by default: raise an exception if the HTTP code of response is >= 300

+
+
+
+

Media Types

+

Each result item of an engine can be of different media-types. Currently the +following media-types are supported. To set another media-type as default, +the parameter template must be set to the desired type.

+ + + + + + + + + + + + + + + + + + + + + +
Table 8 Parameter of the default media type:

result-parameter

information

url

string, url of the result

title

string, title of the result

content

string, general result-text

publishedDate

datetime.datetime, time of publish

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 9 Parameter of the images media type:

result-parameter

information

template

is set to images.html

url

string, url to the result site

title

string, title of the result (partly implemented)

content

(partly implemented)

publishedDate

datetime.datetime, +time of publish (partly implemented)

img_src

string, url to the result image

thumbnail_src

string, url to a small-preview image

+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 10 Parameter of the videos media type:

result-parameter

information

template

is set to videos.html

url

string, url of the result

title

string, title of the result

content

(not implemented yet)

publishedDate

datetime.datetime, time of publish

thumbnail

string, url to a small-preview image

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 11 Parameter of the torrent media type:

result-parameter

information

template

is set to torrent.html

url

string, url of the result

title

string, title of the result

content

string, general result-text

publishedDate

datetime.datetime, +time of publish (not implemented yet)

seed

int, number of seeder

leech

int, number of leecher

filesize

int, size of file in bytes

files

int, number of files

magnetlink

string, magnetlink of the result

torrentfile

string, torrentfile of the result

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 12 Parameter of the map media type:

result-parameter

information

template

is set to map.html

url

string, url of the result

title

string, title of the result

content

string, general result-text

publishedDate

datetime.datetime, time of publish

latitude

latitude of result (in decimal format)

longitude

longitude of result (in decimal format)

boundingbox

boundingbox of result (array of 4. values +[lat-min, lat-max, lon-min, lon-max])

geojson

geojson of result (https://geojson.org/)

osm.type

type of osm-object (if OSM-Result)

osm.id

id of osm-object (if OSM-Result)

address.name

name of object

address.road

street name of object

address.house_number

house number of object

address.locality

city, place of object

address.postcode

postcode of object

address.country

country of object

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 13 Parameter of the paper media type / + see BibTeX field types and BibTeX format

result-parameter

Python type

information

template

str

is set to paper.html

title

str

title of the result

content

str

abstract

comments

str

free text display in italic below the content

tags

List[str]

free tag list

publishedDate

datetime

last publication date

type

str

short description of medium type, e.g. book, pdf or html …

authors

List[str]

list of authors of the work (authors with a “s”)

editor

str

list of editors of a book

publisher

str

name of the publisher

journal

str

name of the journal or magazine the article was +published in

volume

str

volume number

pages

str

page range where the article is

number

str

number of the report or the issue number for a journal article

doi

str

DOI number (like 10.1038/d41586-018-07848-2)

issn

List[str]

ISSN number like 1476-4687

isbn

List[str]

ISBN number like 9780201896831

pdf_url

str

URL to the full article, the PDF version

html_url

str

URL to full article, HTML version

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/index.html b/dev/index.html new file mode 100644 index 00000000..a7dacc5d --- /dev/null +++ b/dev/index.html @@ -0,0 +1,225 @@ + + + + + + + + + Developer documentation — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Developer documentation

+
+

Contents

+ +
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/lxcdev.html b/dev/lxcdev.html new file mode 100644 index 00000000..548ddb10 --- /dev/null +++ b/dev/lxcdev.html @@ -0,0 +1,447 @@ + + + + + + + + + Developing in Linux Containers — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Developing in Linux Containers

+

In this article we will show, how you can make use of Linux Containers (LXC) in +distributed and heterogeneous development cycles (TL;DR; jump to the +Summary).

+ + +
+

Motivation

+

Usually in our development cycle, we edit the sources and run some test and/or +builds by using make [ref] before we commit. This cycle +is simple and perfect but might fail in some aspects we should not overlook.

+
+

The environment in which we run all our development processes matters!

+
+

The Makefile and the Python environment (make install) encapsulate a lot for us, but +they do not have access to all prerequisites. For example, there may have +dependencies on packages that are installed on the developer’s desktop, but +usually are not preinstalled on a server or client system. Another example is; +settings have been made to the software on developer’s desktop that would never +be set on a production system.

+
+

Linux Containers are isolate environments and not to mix up all the +prerequisites from various projects on developer’s desktop is always a good +choice.

+
+

The scripts from DevOps tooling box can divide in those to install and maintain +software:

+ +

and the script utils/lxc.sh, with we can scale our installation, maintenance or +even development tasks over a stack of isolated containers / what we call the:

+
+

SearXNG LXC suite

+
+
+

Hint

+

If you see any problems with the internet connectivity of your +containers read section Internet Connectivity & Docker.

+
+
+
+

Gentlemen, start your engines!

+

Before you can start with containers, you need to install and initiate LXD +once:

+
+
$ snap install lxd
+$ lxd init --auto
+
+
+
+

And you need to clone from origin or if you have your own fork, clone from your +fork:

+
+
$ cd ~/Downloads
+$ git clone https://github.com/searxng/searxng.git searxng
+$ cd searxng
+
+
+
+

The SearXNG suite consists of several images, see export +LXC_SUITE=(... near by git://utils/lxc-searxng.env#L19. For this blog post +we exercise on a archlinux image. The container of this image is named +searxng-archlinux. Lets build the container, but be sure that this container +does not already exists, so first lets remove possible old one:

+
+
$ sudo -H ./utils/lxc.sh remove searxng-archlinux
+$ sudo -H ./utils/lxc.sh build searxng-archlinux
+
+
+
+ +

In this container we install all services including searx, morty & filtron in once:

+
+
$ sudo -H ./utils/lxc.sh install suite searxng-archlinux
+
+
+
+

To proxy HTTP from filtron and morty in the container to the outside of the +container, install nginx into the container. Once for the bot blocker filtron:

+
+
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
+  ./utils/filtron.sh nginx install
+...
+INFO:  got 429 from http://10.174.184.156/searx
+
+
+
+

and once for the content sanitizer (content proxy morty):

+
+
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
+  ./utils/morty.sh nginx install
+...
+INFO:  got 200 from http://10.174.184.156/morty/
+
+
+
+ +

On your system, the IP of your searxng-archlinux container differs from +http://10.174.184.156/searx, just open the URL reported in your installation +protocol in your WEB browser from the desktop to test the instance from outside +of the container.

+

In such a earXNG suite admins can maintain and access the debug log of the +different services quite easy.

+
+
+

In containers, work as usual

+

Usually you open a root-bash using sudo -H bash. In case of LXC containers +open the root-bash in the container using ./utils/lxc.sh cmd +searxng-archlinux:

+
+
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
+INFO:  [searxng-archlinux] bash
+[root@searxng-archlinux searx]# pwd
+/share/searxng
+
+
+
+

The prompt [root@searxng-archlinux ...] signals, that you are the root user in +the searxng-container. To debug the running SearXNG instance use:

+
+
$ ./utils/searx.sh inspect service
+...
+use [CTRL-C] to stop monitoring the log
+...
+
+
+
+

Back in the browser on your desktop open the service http://10.174.184.156/searx +and run your application tests while the debug log is shown in the terminal from +above. You can stop monitoring using CTRL-C, this also disables the “debug +option” in SearXNG’s settings file and restarts the SearXNG uwsgi application. +To debug services from filtron and morty analogous use:

+

Another point we have to notice is that the service (SearXNG +runs under dedicated system user account with the same name (compare +Create user). To get a shell from these accounts, simply call:

+
+
$ ./utils/searxng.sh instance cmd bash
+
+
+
+

To get in touch, open a shell from the service user (searxng@searxng-archlinux):

+
+
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance cmd bash
+INFO:  [searxng-archlinux] ./utils/searxng.sh instance cmd bash
+[searxng@searxng-archlinux ~]$
+
+
+
+

The prompt [searxng@searxng-archlinux] signals that you are logged in as system +user searx in the searxng-archlinux container and the python virtualenv +(searxng-pyenv) environment is activated.

+
+
(searxng-pyenv) [searxng@searxng-archlinux ~]$ pwd
+/usr/local/searxng
+
+
+
+
+
+

Wrap production into developer suite

+

In this section we will see how to change the “Fully functional SearXNG suite” +from a LXC container (which is quite ready for production) into a developer +suite. For this, we have to keep an eye on the Step by step installation:

+
    +

  • SearXNG setup in: /etc/searxng/settings.yml

    • +

  • SearXNG user’s home: /usr/local/searxng

    • +

  • virtualenv in: /usr/local/searxng/searxng-pyenv

    • +

  • SearXNG software in: /usr/local/searxng/searxng-src

    • +
+

With the use of the utils/searxng.sh the SearXNG service was installed as +uWSGI application. To maintain this service, we can use +systemctl (compare uWSGI maintenance).

+
+
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
+  systemctl stop uwsgi@searxng
+
+
+
+

With the command above, we stopped the SearXNG uWSGI-App in the archlinux +container.

+

The uWSGI-App for the archlinux dsitros is configured in +git://utils/templates/etc/uwsgi/apps-archlinux/searxng.ini, from where at +least you should attend the settings of uid, chdir, env and +http:

+
env = SEARXNG_SETTINGS_PATH=/etc/searxng/settings.yml
+http = 127.0.0.1:8888
+
+chdir = /usr/local/searxng/searxng-src/searx
+virtualenv = /usr/local/searxng/searxng-pyenv
+pythonpath = /usr/local/searxng/searxng-src
+
+
+

If you have read the “Good to know section” you remember, that +each container shares the root folder of the repository and the command +utils/lxc.sh cmd handles relative path names transparent. To wrap the +SearXNG installation into a developer one, we simple have to create a smylink to +the transparent reposetory from the desktop. Now lets replace the +repository at searxng-src in the container with the working tree from outside +of the container:

+
+
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
+  mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
+
+$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
+  ln -s /share/searx/ /usr/local/searxng/searxng-src
+
+
+
+

Now we can develop as usual in the working tree of our desktop system. Every +time the software was changed, you have to restart the SearXNG service (in the +container):

+
+
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
+  systemctl restart uwsgi@searx
+
+
+
+

Remember: In containers, work as usual .. here are just some examples from my +daily usage:

+
+

To inspect the SearXNG instance (already described above):

+
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
+  ./utils/searx.sh inspect service
+
+
+

Run Makefile, e.g. to test inside the container:

+
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
+  make test
+
+
+

To install all prerequisites needed for a Buildhosts:

+
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
+  ./utils/searxng.sh install buildhost
+
+
+

To build the docs on a buildhost Buildhosts:

+
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
+  make docs.html
+
+
+
+
+
+

Summary

+

We build up a fully functional SearXNG suite in a archlinux container:

+
$ sudo -H ./utils/lxc.sh install suite searxng-archlinux
+
+
+

To access HTTP from the desktop we installed nginx for the services inside the +container:

+
+
$ ./utils/filtron.sh nginx install
+$ ./utils/morty.sh nginx install
+
+
+
+

To wrap the suite into a developer one, we created a symbolic link to the +repository which is shared transparent from the desktop’s file system into +the container :

+
+
$ mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
+$ ln -s /share/searx/ /usr/local/searxng/searxng-src
+$ systemctl restart uwsgi@searx
+
+
+
+

To get information about the searxNG suite in the archlinux container we can +use:

+
+
$ sudo -H ./utils/lxc.sh show suite searxng-archlinux
+...
+[searxng-archlinux]  INFO:  (eth0) filtron:    http://10.174.184.156:4004/ http://10.174.184.156/searx
+[searxng-archlinux]  INFO:  (eth0) morty:      http://10.174.184.156:3000/
+[searxng-archlinux]  INFO:  (eth0) docs.live:  http://10.174.184.156:8080/
+[searxng-archlinux]  INFO:  (eth0) IPv6:       http://[fd42:573b:e0b3:e97e:216:3eff:fea5:9b65]
+...
+
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/makefile.html b/dev/makefile.html new file mode 100644 index 00000000..e0d397f9 --- /dev/null +++ b/dev/makefile.html @@ -0,0 +1,471 @@ + + + + + + + + + Makefile — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Makefile

+ +

All relevant build tasks are implemented in git://manage and for CI or +IDE integration a small Makefile wrapper is available. If you are not +familiar with Makefiles, we recommend to read gnu-make introduction.

+

The usage is simple, just type make {target-name} to build a target. +Calling the help target gives a first overview (make help):

+
INFO:  sourced NVM environment from /home/runner/.nvm
+nvm.: use nvm (without dot) to execute nvm commands directly
+  install   : install NVM locally at /home/runner/work/searxng/searxng/.nvm
+  clean     : remove NVM installation
+  status    : prompt some status informations about nvm & node
+  nodejs    : install Node.js latest LTS
+  cmd ...   : run command ... in NVM environment
+  bash      : start bash interpreter with NVM environment sourced
+buildenv:
+  rebuild ./utils/brand.env
+webapp.:
+  run       : run developer instance
+weblate.:
+  push.translations: push translation changes from SearXNG to Weblate's counterpart
+  to.translations: Update 'translations' branch with last additions from Weblate.
+data.:
+  all       : update searx/languages.py and ./data/*
+  languages : update searx/data/engines_languages.json & searx/languages.py
+  useragents: update searx/data/useragents.json with the most recent versions of Firefox.
+docs.:
+  html      : build HTML documentation
+  live      : autobuild HTML documentation while editing
+  gh-pages  : deploy on gh-pages branch
+  prebuild  : build reST include files (./build/docs/includes)
+  clean     : clean documentation build
+docker.:
+  build     : build docker image
+  push      : build and push docker image
+gecko.driver:
+  download & install geckodriver if not already installed (required for
+  robot_tests)
+redis:
+  build     : build redis binaries at /home/runner/work/searxng/searxng/dist/redis/6.2.6/amd64
+  install   : create user (searxng-redis) and install systemd service (searxng-redis)
+  help      : show more redis commands
+node.:
+  env       : download & install npm dependencies locally
+  clean     : drop locally npm installations
+py.:
+  build     : Build python packages at ./dist
+  clean     : delete virtualenv and intermediate py files
+pyenv.:
+  install   : developer install of SearXNG into virtualenv
+  uninstall : uninstall developer installation
+  cmd ...   : run command ... in virtualenv
+  OK        : test if virtualenv is OK
+pypi.upload:
+  Upload python packages to PyPi (to test use pypi.upload.test)
+format.:
+  python    : format Python code source using black
+test.:
+  yamllint  : lint YAML files (YAMLLINT_FILES)
+  pylint    : lint PYLINT_FILES, searx/engines, searx & tests
+  pyright   : static type check of python sources
+  black     : check black code format
+  unit      : run unit tests
+  coverage  : run unit tests with coverage
+  robot     : run robot test
+  rst       : test .rst files incl. README.rst
+  clean     : clean intermediate test stuff
+themes.:
+  all       : build all themes
+  simple    : build simple theme
+pygments.:
+  less      : build LESS files for pygments
+go.:
+  ls        : list golang binary archives (stable)
+  golang    : (re-) install golang binary in user's $HOME/local folder
+  install   : install go package in user's $HOME/go-apps folder
+  bash      : start bash interpreter with golang environment sourced
+static.build.:  [build] /static
+  commit    : build & commit /static folder
+  drop      : drop last commit if it was previously done by static.build.commit
+  restore   : git restore of the /static folder (after themes.all)
+environment ...
+  SEARXNG_REDIS_URL : 
+----
+run            - run developer instance
+install        - developer install of SearxNG into virtualenv
+uninstall      - uninstall developer installation
+clean          - clean up working tree
+search.checker - check search engines
+test           - run shell & CI tests
+test.shell     - test shell scripts
+ci.test        - run CI tests
+
+
+ +
+

Python environment (make install)

+ +

We do no longer need to build up the virtualenv manually. Jump into your git +working tree and release a make install to get a virtualenv with a +developer install of SearXNG (git://setup.py).

+
$ cd ~/searxng-clone
+$ make install
+PYENV     [virtualenv] installing ./requirements*.txt into local/py3
+...
+PYENV     OK
+PYENV     [install] pip install -e 'searx[test]'
+...
+Successfully installed argparse-1.4.0 searx
+BUILDENV  INFO:searx:load the default settings from ./searx/settings.yml
+BUILDENV  INFO:searx:Initialisation done
+BUILDENV  build utils/brand.env
+
+
+

If you release make install multiple times the installation will only +rebuild if the sha256 sum of the requirement files fails. With other words: +the check fails if you edit the requirements listed in +git://requirements-dev.txt and git://requirements.txt).

+
$ make install
+PYENV     OK
+PYENV     [virtualenv] requirements.sha256 failed
+          [virtualenv] - 6cea6eb6def9e14a18bf32f8a3e...  ./requirements-dev.txt
+          [virtualenv] - 471efef6c73558e391c3adb35f4...  ./requirements.txt
+...
+PYENV     [virtualenv] installing ./requirements*.txt into local/py3
+...
+PYENV     OK
+PYENV     [install] pip install -e 'searx[test]'
+...
+Successfully installed argparse-1.4.0 searx
+BUILDENV  INFO:searx:load the default settings from ./searx/settings.yml
+BUILDENV  INFO:searx:Initialisation done
+BUILDENV  build utils/brand.env
+
+
+ +

If you think, something goes wrong with your ./local environment or you change +the git://setup.py file, you have to call make clean.

+
+
+

make buildenv

+

Rebuild instance’s environment with the modified settings from the +brand: and server: section of your +settings.yml.

+

We have all SearXNG setups are centralized in the settings.yml file. +This setup is available as long we are in a installed instance. E.g. the +installed instance on the server or the installed developer instance at +./local (the later one is created by a make install or make run).

+

Tasks running outside of an installed instance, especially those tasks and +scripts running at (pre-) installation time do not have access to the SearXNG +setup (from a installed instance). Those tasks need a build environment.

+

The make buildenv target will update the build environment in:

+ +

Tasks running outside of an installed instance, need the following settings +from the YAML configuration:

+ +
+
+

Node.js environment (make node.env)

+

Node.js version 16.13.0 or higher is required to build the themes. +If the requirement is not met, the build chain uses nvm (Node Version +Manager) to install latest LTS of Node.js locally: there is no need to +install nvm or npm on your system.

+

Use make nvm.status to get the current status of you Node.js and nvm setup.

+

Here is the output you will typically get on a Ubuntu 20.04 system which serves +only a no longer active Release +Node.js v10.19.0.

+
$ make nvm.status
+INFO:  Node.js is installed at /usr/bin/node
+INFO:  Node.js is version v10.19.0
+WARN:  minimal Node.js version is 16.13.0
+INFO:  npm is installed at /usr/bin/npm
+INFO:  npm is version 6.14.4
+WARN:  NVM is not installed
+INFO:  to install NVM and Node.js (LTS) use: manage nvm install --lts
+
+
+

To install you can also use make nvm.nodejs

+
+
+

make nvm.nodejs

+

Install latest Node.js LTS locally (uses nvm):

+
$ make nvm.nodejs
+INFO:  install (update) NVM at /share/searxng/.nvm
+INFO:  clone: https://github.com/nvm-sh/nvm.git
+...
+Downloading and installing node v16.13.0...
+...
+INFO:  Node.js is installed at searxng/.nvm/versions/node/v16.13.0/bin/node
+INFO:  Node.js is version v16.13.0
+INFO:  npm is installed at searxng/.nvm/versions/node/v16.13.0/bin/npm
+INFO:  npm is version 8.1.0
+INFO:  NVM is installed at searxng/.nvm
+
+
+
+
+

make run

+

To get up a running a developer instance simply call make run. This enables +debug option in git://searx/settings.yml, starts a ./searx/webapp.py +instance and opens the URL in your favorite WEB browser (xdg-open):

+
$ make run
+
+
+

Changes to theme’s HTML templates (jinja2) are instant. Changes to the CSS & JS +sources of the theme need to be rebuild. You can do that by running:

+
$ make themes.all
+
+
+

Alternatively to themes.all you can run live builds of the theme you are +modify:

+
$ LIVE_THEME=simple make run
+
+
+
+
+

make clean

+

Drops all intermediate files, all builds, but keep sources untouched. Before +calling make clean stop all processes using the Python environment (make install) or +Node.js environment (make node.env).

+
$ make clean
+CLEAN     pyenv
+PYENV     [virtualenv] drop local/py3
+CLEAN     docs -- build/docs dist/docs
+CLEAN     themes -- locally installed npm dependencies
+...
+CLEAN     test stuff
+CLEAN     common files
+
+
+
+
+

make docs docs.autobuild docs.clean

+

We describe the usage of the doc.* targets in the How to contribute / +Documentation section. If you want to edit the documentation +read our live build section. If you are working in your own brand, +adjust your Global Settings.

+
+
+

make docs.gh-pages

+

To deploy on github.io first adjust your Global Settings. For any +further read deploy on github.io.

+
+
+

make test

+

Runs a series of tests: make test.pylint, test.pep8, test.unit +and test.robot. You can run tests selective, e.g.:

+
$ make test.pep8 test.unit test.sh
+TEST      test.pep8 OK
+...
+TEST      test.unit OK
+...
+TEST      test.sh OK
+
+
+
+
+

make test.shell

+

Lint shell scripts / if you have changed some bash scripting run this test before +commit.

+
+
+

make test.pylint

+

Pylint is known as one of the best source-code, bug and quality checker for the +Python programming language. The pylint profile used in the SearXNG project is +found in project’s root folder git://.pylintrc.

+
+
+

search.checker.{engine name}

+

To check all engines:

+
make search.checker
+
+
+

To check a engine with whitespace in the name like google news replace space +by underline:

+
make search.checker.google_news
+
+
+

To see HTTP requests and more use SEARXNG_DEBUG:

+
make SEARXNG_DEBUG=1 search.checker.google_news
+
+
+

To filter out HTTP redirects (3xx):

+
make SEARXNG_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
+...
+Engine google news                   Checking
+https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
+https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
+--
+https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
+https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
+--
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/offline_engines.html b/dev/offline_engines.html new file mode 100644 index 00000000..a191008d --- /dev/null +++ b/dev/offline_engines.html @@ -0,0 +1,207 @@ + + + + + + + + + Offline Engines — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Offline Engines

+ +

To extend the functionality of SearXNG, offline engines are going to be +introduced. An offline engine is an engine which does not need Internet +connection to perform a search and does not use HTTP to communicate.

+

Offline engines can be configured, by adding those to the engines list of +settings.yml. An example skeleton for offline +engines can be found in Demo Offline Engine (demo_offline.py).

+
+

Programming Interface

+
+
init(engine_settings=None)

All offline engines can have their own init function to setup the engine before +accepting requests. The function gets the settings from settings.yml as a +parameter. This function can be omitted, if there is no need to setup anything +in advance.

+
+
+

search(query, params)

+
+

Each offline engine has a function named search. This function is +responsible to perform a search and return the results in a presentable +format. (Where presentable means presentable by the selected result +template.)

+

The return value is a list of results retrieved by the engine.

+
+
+
Engine representation in /config

If an engine is offline, the attribute offline is set to True.

+
+
+
+
+

Extra Dependencies

+

If an offline engine depends on an external tool, SearXNG does not install it by +default. When an administrator configures such engine and starts the instance, +the process returns an error with the list of missing dependencies. Also, +required dependencies will be added to the comment/description of the engine, so +admins can install packages in advance.

+

If there is a need to install additional packages in Python’s Virtual +Environment of your SearXNG instance you need to switch into the environment +(Install SearXNG & dependencies) first, for this you can use utils/searxng.sh:

+
$ sudo utils/searxng.sh instance cmd bash
+(searxng-pyenv)$ pip install ...
+
+
+
+
+

Private engines (Security)

+

To limit the access to offline engines, if an instance is available publicly, +administrators can set token(s) for each of the Private Engines (tokens). If a +query contains a valid token, then SearXNG performs the requested private +search. If not, requests from an offline engines return errors.

+
+
+

Acknowledgement

+

This development was sponsored by Search and Discovery Fund of NLnet Foundation .

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/plugins.html b/dev/plugins.html new file mode 100644 index 00000000..37184d82 --- /dev/null +++ b/dev/plugins.html @@ -0,0 +1,266 @@ + + + + + + + + + Plugins — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Plugins

+ +

Plugins can extend or replace functionality of various components of searx.

+
+

Example plugin

+
name = 'Example plugin'
+description = 'This plugin extends the suggestions with the word "example"'
+default_on = False  # disabled by default
+
+js_dependencies = tuple()  # optional, list of static js files
+css_dependencies = tuple()  # optional, list of static css files
+
+
+# attach callback to the post search hook
+#  request: flask request object
+#  ctx: the whole local context of the post search hook
+def post_search(request, search):
+    search.result_container.suggestions.add('example')
+    return True
+
+
+
+
+

External plugins

+

SearXNG supports external plugins / there is no need to install one, SearXNG +runs out of the box. But to demonstrate; in the example below we install the +SearXNG plugins from The Green Web Foundation [ref]:

+
$ sudo utils/searxng.sh instance cmd bash
+(searxng-pyenv)$ pip install git+https://github.com/return42/tgwf-searx-plugins
+
+
+

In the settings.yml activate the plugins: section and add module +only_show_green_results from tgwf-searx-plugins.

+
plugins:
+  ...
+  - only_show_green_results
+  ...
+
+
+
+
+

Plugin entry points

+

Entry points (hooks) define when a plugin runs. Right now only three hooks are +implemented. So feel free to implement a hook if it fits the behaviour of your +plugin. A plugin doesn’t need to implement all the hooks.

+
+ +

Runs BEFORE the search request.

+

search.result_container can be changed.

+

Return a boolean:

+
    +

  • True to continue the search

    • +

  • False to stop the search

    • +
+
+
Parameters:
+
+
+
Returns:
+

False to stop the search

+
+
Return type:
+

bool

+
+
+
+ +
+
+post_search(request, search) → None
+

Runs AFTER the search request.

+
+
Parameters:
+
+
+
+
+ +
+
+on_result(request, search, result) → bool
+

Runs for each result of each engine.

+

result can be changed.

+

If result[“url”] is defined, then result[“parsed_url”] = urlparse(result[‘url’])

+
+

Warning

+

result[“url”] can be changed, but result[“parsed_url”] must be updated too.

+
+

Return a boolean:

+
    +

  • True to keep the result

    • +

  • False to remove the result

    • +
+
+
Parameters:
+
+
+
Returns:
+

True to keep the result

+
+
Return type:
+

bool

+
+
+
+ +
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/quickstart.html b/dev/quickstart.html new file mode 100644 index 00000000..f00e0d1f --- /dev/null +++ b/dev/quickstart.html @@ -0,0 +1,193 @@ + + + + + + + + + Development Quickstart — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Development Quickstart

+

SearXNG loves developers, just clone and start hacking. All the rest is done for +you simply by using make.

+
git clone https://github.com/searxng/searxng.git searxng
+
+
+

Here is how a minimal workflow looks like:

+
    +

  1. start hacking

    2. +

  2. run your code: make run

    3. +

  3. test your code: make test

    4. +
+

If you think at some point something fails, go back to start. Otherwise, +choose a meaningful commit message and we are happy to receive your pull +request. To not end in wild west we have some directives, please pay attention +to our “How to contribute” guideline.

+

If you implement themes, you will need to setup a Node.js environment (make node.env) once:

+
make node.env
+
+
+

Before you call make run (2.), you need to compile the modified styles and +JavaScript:

+
make themes.all
+
+
+

Alternatively you can also compile selective the theme you have modified, +e.g. the simple theme.

+
make themes.simple
+
+
+
+

Tip

+

To get live builds while modifying CSS & JS use: LIVE_THEME=simple make run

+
+

If you finished your tests you can start to commit your changes. To separate +the modified source code from the build products first run:

+
make static.build.restore
+
+
+

This will restore the old build products and only your changes of the code +remain in the working tree which can now be added & commited. When all sources +are commited, you can commit the build products simply by:

+
make static.build.commit
+
+
+

Commiting the build products should be the last step, just before you send us +your PR. There is also a make target to rewind this last build commit:

+
make static.build.drop
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/reST.html b/dev/reST.html new file mode 100644 index 00000000..d43057c8 --- /dev/null +++ b/dev/reST.html @@ -0,0 +1,1688 @@ + + + + + + + + + reST primer — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

reST primer

+ +

We at SearXNG are using reStructuredText (aka reST) markup for all kind of +documentation, with the builders from the Sphinx project a HTML output is +generated and deployed at github.io. For build prerequisites read +Build docs.

+

The source files of Searx’s documentation are located at git://docs. Sphinx +assumes source files to be encoded in UTF-8 by default. Run make docs.live to build HTML while editing.

+ + +

Sphinx and reST have their place in the python ecosystem. Over that reST is +used in popular projects, e.g the Linux kernel documentation [kernel doc].

+ +

reST is a plaintext markup language, its markup is mostly intuitive and +you will not need to learn much to produce well formed articles with. I use the +word mostly: like everything in live, reST has its advantages and +disadvantages, some markups feel a bit grumpy (especially if you are used to +other plaintext markups).

+
+

Soft skills

+

Before going any deeper into the markup let’s face on some soft skills a +trained author brings with, to reach a well feedback from readers:

+
    +

  • Documentation is dedicated to an audience and answers questions from the +audience point of view.

    • +

  • Don’t detail things which are general knowledge from the audience point of +view.

    • +

  • Limit the subject, use cross links for any further reading.

    • +
+

To be more concrete what a point of view means. In the (git://docs) +folder we have three sections (and the blog folder), each dedicate to a +different group of audience.

+
+
User’s POV: git://docs/user

A typical user knows about search engines and might have heard about +meta crawlers and privacy.

+
+
Admin’s POV: git://docs/admin

A typical Admin knows about setting up services on a linux system, but he does +not know all the pros and cons of a SearXNG setup.

+
+
Developer’s POV: git://docs/dev

Depending on the readability of code, a typical developer is able to read and +understand source code. Describe what a item aims to do (e.g. a function). +If the chronological order matters, describe it. Name the out-of-limits +conditions and all the side effects a external developer will not know.

+
+
+
+
+

Basic inline markup

+ +

Basic inline markup is done with asterisks and backquotes. If asterisks or +backquotes appear in running text and could be confused with inline markup +delimiters, they have to be escaped with a backslash (\*pointer).

+ + +++++ + + + + + + + + + + + + + + + + + + + + + + + + +
Table 14 basic inline markup

description

rendered

markup

one asterisk for emphasis

italics

*italics*

two asterisks for strong emphasis

boldface

**boldface**

backquotes for code samples and literals

foo()

``foo()``

quote asterisks or backquotes

*foo is a pointer

\*foo is a pointer

+
+
+

Basic article structure

+

The basic structure of an article makes use of heading adornments to markup +chapter, sections and subsections.

+
+

reST template

+

reST template for an simple article:

+
.. _doc refname:
+
+==============
+Document title
+==============
+
+Lorem ipsum dolor sit amet, consectetur adipisici elit ..  Further read
+:ref:`chapter refname`.
+
+.. _chapter refname:
+
+Chapter
+=======
+
+Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut
+aliquid ex ea commodi consequat ...
+
+.. _section refname:
+
+Section
+-------
+
+lorem ..
+
+.. _subsection refname:
+
+Subsection
+~~~~~~~~~~
+
+lorem ..
+
+
+
+
+

Headings

+
    +

  1. title - with overline for document title:

    2. +
+
+
==============
+Document title
+==============
+
+
+
+
    +

  1. chapter - with anchor named anchor name:

    +
    .. _anchor name:
+
+Chapter
+=======
+
    +
    +
    2. +

  2. section

    +
    Section
+-------
+
    +
    +
    3. +

  3. subsection

    +
    Subsection
+~~~~~~~~~~
+
    +
    +
    4. +
+
+
+ +
+

Literal blocks

+

The simplest form of literal-blocks is a indented block introduced by +two colons (::). For highlighting use highlight or code-block directive. To include literals from external files use +literalinclude or kernel-include +directive (latter one expands environment variables in the path name).

+
+

::

+
::
+
+  Literal block
+
+Lorem ipsum dolor::
+
+  Literal block
+
+Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy
+eirmod tempor invidunt ut labore ::
+
+  Literal block
+
+
+
+

Literal block

+
Literal block
+
+
+

Lorem ipsum dolor:

+
Literal block
+
+
+

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy +eirmod tempor invidunt ut labore

+
Literal block
+
+
+
+
+
+

code-block

+ +

The code-block directive is a variant of the code directive +with additional options. To learn more about code literals visit +Showing code examples.

+
The URL ``/stats`` handle is shown in :ref:`stats-handle`
+
+.. code-block:: Python
+   :caption: python code block
+   :name: stats-handle
+
+   @app.route('/stats', methods=['GET'])
+   def stats():
+       """Render engine statistics page."""
+       stats = get_engines_stats()
+       return render(
+           'stats.html'
+           , stats = stats )
+
+
+

+
+
+
+

Code block

+

The URL /stats handle is shown in python code block

+
+
Listing 1 python code block
+
@app.route('/stats', methods=['GET'])
+def stats():
+    """Render engine statistics page."""
+    stats = get_engines_stats()
+    return render(
+        'stats.html'
+        , stats = stats )
+
+
+
+
+
+
+
+

Unicode substitution

+

The unicode directive converts Unicode +character codes (numerical values) to characters. This directive can only be +used within a substitution definition.

+
.. |copy| unicode:: 0xA9 .. copyright sign
+.. |(TM)| unicode:: U+2122
+
+Trademark |(TM)| and copyright |copy| glyphs.
+
+
+
+

Unicode

+

Trademark ™ and copyright © glyphs.

+
+
+
+

Roles

+ +

A custom interpreted text role (ref) is an inline piece of +explicit markup. It signifies that that the enclosed text should be interpreted +in a specific way.

+

The general markup is one of:

+
:rolename:`ref-name`
+:rolename:`ref text <ref-name>`
+
+
+ + +++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 16 smart refs with sphinx.ext.extlinks and intersphinx

role

rendered example

markup

guilabel

Cancel

:guilabel:`&Cancel`

kbd

C-x C-f

:kbd:`C-x C-f`

menuselection

Open ‣ File

:menuselection:`Open --> File`

download

this file

:download:`this file <reST.rst>`

math

a^2 + b^2 = c^2

:math:`a^2 + b^2 = c^2`

ref

Simple SVG image.

:ref:`svg image example`

command

ls -la

:command:`ls -la`

emphasis

italic

:emphasis:`italic`

strong

bold

:strong:`bold`

literal

foo()

:literal:`foo()`

subscript

H2O

H\ :sub:`2`\ O

superscript

E = mc2

E = mc\ :sup:`2`

title-reference

Time

:title:`Time`

+
+
+

Figures & Images

+ +

Searx’s sphinx setup includes: Scalable figure and image handling. Scalable here means; +scalable in sense of the build process. Normally in absence of a converter +tool, the build process will break. From the authors POV it’s annoying to care +about the build process when handling with images, especially since he has no +access to the build process. With Scalable figure and image handling the build process +continues and scales output quality in dependence of installed image processors.

+

If you want to add an image, you should use the kernel-figure (inheritance +of figure) and kernel-image (inheritance of image) +directives. E.g. to insert a figure with a scalable image format use SVG +(Simple SVG image.):

+
.. _svg image example:
+
+.. kernel-figure:: svg_image.svg
+   :alt: SVG image example
+
+   Simple SVG image
+
+ To refer the figure, a caption block is needed: :ref:`svg image example`.
+
+
+
+SVG image example
+

Fig. 3 Simple SVG image.

+
+
+

To refer the figure, a caption block is needed: Simple SVG image..

+
+

DOT files (aka Graphviz)

+

With kernel-figure & kernel-image reST support for DOT formatted files is +given.

+ +

A simple example is shown in DOT’s hello world example:

+
.. _dot file example:
+
+.. kernel-figure:: hello.dot
+   :alt: hello world
+
+   DOT's hello world example
+
+
+
+

hello.dot

+
+hello world
+

Fig. 4 DOT’s hello world example

+
+
+
+
+
+

kernel-render DOT

+

Embed render markups (or languages) like Graphviz’s DOT is provided by the +kernel-render directive. A simple example of embedded DOT is +shown in figure Embedded DOT (Graphviz) code:

+
.. _dot render example:
+
+.. kernel-render:: DOT
+   :alt: digraph
+   :caption: Embedded  DOT (Graphviz) code
+
+   digraph foo {
+     "bar" -> "baz";
+   }
+
+Attribute ``caption`` is needed, if you want to refer the figure: :ref:`dot
+render example`.
+
+
+

Please note build tools. If Graphviz is +installed, you will see an vector image. If not, the raw markup is inserted as +literal-block.

+
+

kernel-render DOT

+
+digraph
+

Fig. 5 Embedded DOT (Graphviz) code

+
+
+

Attribute caption is needed, if you want to refer the figure: Embedded DOT (Graphviz) code.

+
+
+
+

kernel-render SVG

+

A simple example of embedded SVG is shown in figure Embedded SVG markup:

+
.. _svg render example:
+
+.. kernel-render:: SVG
+   :caption: Embedded **SVG** markup
+   :alt: so-nw-arrow
+
+
+
+
<?xml version="1.0" encoding="UTF-8"?>
+<svg xmlns="http://www.w3.org/2000/svg" version="1.1"
+     baseProfile="full" width="70px" height="40px"
+     viewBox="0 0 700 400"
+     >
+  <line x1="180" y1="370"
+        x2="500" y2="50"
+        stroke="black" stroke-width="15px"
+        />
+  <polygon points="585 0 525 25 585 50"
+           transform="rotate(135 525 25)"
+           />
+</svg>
+
+
+
+
+

kernel-render SVG

+
+so-nw-arrow
+

Fig. 6 Embedded SVG markup

+
+
+
+
+
+
+

List markups

+
+

Bullet list

+

List markup (ref) is simple:

+
- This is a bulleted list.
+
+  1. Nested lists are possible, but be aware that they must be separated from
+     the parent list items by blank line
+  2. Second item of nested list
+
+- It has two items, the second
+  item uses two lines.
+
+#. This is a numbered list.
+#. It has two items too.
+
+
+
+

bullet list

+
    +

  • This is a bulleted list.

    +
      +

    1. Nested lists are possible, but be aware that they must be separated from +the parent list items by blank line

      2. +

    2. Second item of nested list

      3. +
    +
    • +

  • It has two items, the second +item uses two lines.

    • +
+
    +

  1. This is a numbered list.

    2. +

  2. It has two items too.

    3. +
+
+
+
+

Horizontal list

+

The .. hlist:: transforms a bullet list into a more compact +list.

+
.. hlist::
+
+   - first list item
+   - second list item
+   - third list item
+   ...
+
+
+
+

hlist

+
    +

  • first list item

    • +

  • second list item

    • +

  • third list item

    • +

  • next list item

    • +
+
    +

  • next list item xxxx

    • +

  • next list item yyyy

    • +

  • next list item zzzz

    • +
+
+
+
+
+

Definition list

+ +

Each definition list (ref) item contains a term, +optional classifiers and a definition. A term is a simple one-line word or +phrase. Optional classifiers may follow the term on the same line, each after +an inline ‘ : ‘ (space, colon, space). A definition is a block indented +relative to the term, and may contain multiple paragraphs and other body +elements. There may be no blank line between a term line and a definition block +(this distinguishes definition lists from block quotes). Blank lines are +required before the first and after the last definition list item, but are +optional in-between.

+

Definition lists are created as follows:

+
term 1 (up to a line of text)
+    Definition 1.
+
+See the typo : this line is not a term!
+
+  And this is not term's definition.  **There is a blank line** in between
+  the line above and this paragraph.  That's why this paragraph is taken as
+  **block quote** (:duref:`ref <block-quotes>`) and not as term's definition!
+
+term 2
+    Definition 2, paragraph 1.
+
+    Definition 2, paragraph 2.
+
+term 3 : classifier
+    Definition 3.
+
+term 4 : classifier one : classifier two
+    Definition 4.
+
+
+
+

definition list

+
+
term 1 (up to a line of text)

Definition 1.

+
+
+

See the typo : this line is not a term!

+
+

And this is not term’s definition. There is a blank line in between +the line above and this paragraph. That’s why this paragraph is taken as +block quote (ref) and not as term’s definition!

+
+
+
term 2

Definition 2, paragraph 1.

+

Definition 2, paragraph 2.

+
+
term 3classifier

Definition 3.

+
+
+

term 4 : classifier one : classifier two

+
+
+
+

Quoted paragraphs

+

Quoted paragraphs (ref) are created by just indenting +them more than the surrounding paragraphs. Line blocks (ref) are a way of preserving line breaks:

+
normal paragraph ...
+lorem ipsum.
+
+   Quoted paragraph ...
+   lorem ipsum.
+
+| These lines are
+| broken exactly like in
+| the source file.
+
+
+
+

Quoted paragraph and line block

+

normal paragraph … +lorem ipsum.

+
+

Quoted paragraph … +lorem ipsum.

+
+
+
These lines are
+
broken exactly like in
+
the source file.
+
+
+
+
+

Field Lists

+ +

Field lists are used as part of an extension syntax, such as options for +directives, or database-like records meant for further processing. Field lists +are mappings from field names to field bodies. They marked up like this:

+
:fieldname: Field content
+:foo:       first paragraph in field foo
+
+            second paragraph in field foo
+
+:bar:       Field content
+
+
+
+

Field List

+
+
fieldname:
+

Field content

+
+
foo:
+

first paragraph in field foo

+

second paragraph in field foo

+
+
bar:
+

Field content

+
+
+
+

They are commonly used in Python documentation:

+
def my_function(my_arg, my_other_arg):
+    """A function just for me.
+
+    :param my_arg: The first of my arguments.
+    :param my_other_arg: The second of my arguments.
+
+    :returns: A message (just for me, of course).
+    """
+
+
+
+
+

Further list blocks

+
    +

  • field lists (ref, with caveats noted in +Field Lists)

    • +

  • option lists (ref)

    • +

  • quoted literal blocks (ref)

    • +

  • doctest blocks (ref)

    • +
+
+
+
+

Admonitions

+ +
+

Generic admonition

+

The generic admonition needs a title:

+
.. admonition:: generic admonition title
+
+   lorem ipsum ..
+
+
+
+

generic admonition title

+

lorem ipsum ..

+
+
+
+

Specific admonitions

+

Specific admonitions: hint, note, tip attention, +caution, danger, error, , important, and +warning .

+
.. hint::
+
+   lorem ipsum ..
+
+.. note::
+
+   lorem ipsum ..
+
+.. warning::
+
+   lorem ipsum ..
+
+
+
+

Hint

+

lorem ipsum ..

+
+
+

Note

+

lorem ipsum ..

+
+
+

Tip

+

lorem ipsum ..

+
+
+

Attention

+

lorem ipsum ..

+
+
+

Caution

+

lorem ipsum ..

+
+
+

Danger

+

lorem ipsum ..

+
+
+

Important

+

lorem ipsum ..

+
+
+

Error

+

lorem ipsum ..

+
+
+

Warning

+

lorem ipsum ..

+
+
+
+
+

Tables

+ +

ASCII-art tables like Simple tables and Grid tables might +be comfortable for readers of the text-files, but they have huge disadvantages +in the creation and modifying. First, they are hard to edit. Think about +adding a row or a column to a ASCII-art table or adding a paragraph in a cell, +it is a nightmare on big tables.

+ +

Second the diff of modifying ASCII-art tables is not meaningful, e.g. widening a +cell generates a diff in which also changes are included, which are only +ascribable to the ASCII-art. Anyway, if you prefer ASCII-art for any reason, +here are some helpers:

+ +
+

Simple tables

+

Simple tables allow colspan but not rowspan. If +your table need some metadata (e.g. a title) you need to add the .. table:: +directive (ref) in front and place the table in its body:

+
.. table:: foo gate truth table
+   :widths: grid
+   :align: left
+
+   ====== ====== ======
+       Inputs    Output
+   ------------- ------
+   A      B      A or B
+   ====== ====== ======
+   False
+   --------------------
+   True
+   --------------------
+   True   False  True
+          (foo)
+   ------ ------ ------
+   False  True
+          (foo)
+   ====== =============
+
+
+
+

Simple ASCII table

+ + +++++ + + + + + + + + + + + + + + + + + + + + + + +
Table 17 foo gate truth table

Inputs

Output

A

B

A or B

False

True

True

False +(foo)

True

False

True +(foo)

+
+
+
+

Grid tables

+

Grid tables allow colspan colspan and rowspan:

+
.. table:: grid table example
+   :widths: 1 1 5
+
+   +------------+------------+-----------+
+   | Header 1   | Header 2   | Header 3  |
+   +============+============+===========+
+   | body row 1 | column 2   | column 3  |
+   +------------+------------+-----------+
+   | body row 2 | Cells may span columns.|
+   +------------+------------+-----------+
+   | body row 3 | Cells may  | - Cells   |
+   +------------+ span rows. | - contain |
+   | body row 4 |            | - blocks. |
+   +------------+------------+-----------+
+
+
+
+

ASCII grid table

+ + +++++ + + + + + + + + + + + + + + + + + + + + + +
Table 18 grid table example

Header 1

Header 2

Header 3

body row 1

column 2

column 3

body row 2

Cells may span columns.

body row 3

Cells may +span rows.

    +

  • Cells

    • +

  • contain

    • +

  • blocks.

    • +
+

body row 4

+
+
+
+

flat-table

+

The flat-table is a further developed variant of the list tables. It is a double-stage list similar to the +list-table with some additional features:

+
+
column-span: cspan

with the role cspan a cell can be extended through additional columns

+
+
row-span: rspan

with the role rspan a cell can be extended through additional rows

+
+
auto-span:

spans rightmost cell of a table row over the missing cells on the right side +of that table-row. With Option :fill-cells: this behavior can changed +from auto span to auto fill, which automatically inserts (empty) cells +instead of spanning the last cell.

+
+
options:
+
header-rows:
+

[int] count of header rows

+
+
stub-columns:
+

[int] count of stub columns

+
+
widths:
+

[[int] [int] … ] widths of columns

+
+
fill-cells:
+

instead of auto-span missing cells, insert missing cells

+
+
+
+
roles:
+
cspan:
+

[int] additional columns (morecols)

+
+
rspan:
+

[int] additional rows (morerows)

+
+
+
+
+

The example below shows how to use this markup. The first level of the staged +list is the table-row. In the table-row there is only one markup allowed, +the list of the cells in this table-row. Exception are comments ( .. ) +and targets (e.g. a ref to row 2 of table’s body).

+
.. flat-table:: ``flat-table`` example
+   :header-rows: 2
+   :stub-columns: 1
+   :widths: 1 1 1 1 2
+
+   * - :rspan:`1` head / stub
+     - :cspan:`3` head 1.1-4
+
+   * - head 2.1
+     - head 2.2
+     - head 2.3
+     - head 2.4
+
+   * .. row body 1 / this is a comment
+
+     - row 1
+     - :rspan:`2` cell 1-3.1
+     - cell 1.2
+     - cell 1.3
+     - cell 1.4
+
+   * .. Comments and targets are allowed on *table-row* stage.
+     .. _`row body 2`:
+
+     - row 2
+     - cell 2.2
+     - :rspan:`1` :cspan:`1`
+       cell 2.3 with a span over
+
+       * col 3-4 &
+       * row 2-3
+
+   * - row 3
+     - cell 3.2
+
+   * - row 4
+     - cell 4.1
+     - cell 4.2
+     - cell 4.3
+     - cell 4.4
+
+   * - row 5
+     - cell 5.1 with automatic span to right end
+
+   * - row 6
+     - cell 6.1
+     - ..
+
+
+
+

List table

+ + +++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Table 19 flat-table example

head / stub

head 1.1-4

head 2.1

head 2.2

head 2.3

head 2.4

row 1

cell 1-3.1

cell 1.2

cell 1.3

cell 1.4

row 2

+

cell 2.2

+

+cell 2.3 with a span over

+
    +

  • col 3-4 &

    • +

  • row 2-3

    • +
+

row 3

cell 3.2

row 4

cell 4.1

cell 4.2

cell 4.3

cell 4.4

row 5

cell 5.1 with automatic span to right end

row 6

cell 6.1

+
+
+
+

CSV table

+

CSV table might be the choice if you want to include CSV-data from a outstanding +(build) process into your documentation.

+
.. csv-table:: CSV table example
+   :header: .. , Column 1, Column 2
+   :widths: 2 5 5
+   :stub-columns: 1
+   :file: csv_table.txt
+
+
+

Content of file csv_table.txt:

+
stub col row 1, column, "loremLorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy
+eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam
+voluptua."
+stub col row 1, "At vero eos et accusam et justo duo dolores et ea rebum. Stet clita
+kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.", column
+stub col row 1, column, column
+
+
+
+

CSV table

+ + +++++ + + + + + + + + + + + + + + + + + + + + +
Table 20 CSV table example

Column 1

Column 2

stub col row 1

column

loremLorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy +eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam +voluptua.

stub col row 1

At vero eos et accusam et justo duo dolores et ea rebum. Stet clita +kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

column

stub col row 1

column

column

+
+
+
+
+

Templating

+ +

Templating is suitable for documentation which is created generic at the build +time. The sphinx-jinja extension evaluates jinja templates in the Python environment (make install) (with SearXNG modules installed). We use this e.g. to build chapter: +Configured Engines. Below the jinja directive from the +git://docs/admin/engines.rst is shown:

+
==================
+Configured Engines
+==================
+
+.. sidebar:: Further reading ..
+
+   - :ref:`engines-dev`
+   - :ref:`settings engine`
+
+Explanation of the :ref:`general engine configuration` shown in the table
+:ref:`configured engines`.
+
+.. jinja:: searx
+
+   SearXNG supports {{engines | length}} search engines (of which {{enabled_engine_count}} are enabled by default).
+
+   {% for category, engines in categories_as_tabs.items() %}
+
+   {{category}} search engines
+   ---------------------------------------
+
+   {% for group, engines in engines | group_engines_in_tab %}
+
+   {% if loop.length > 1 %}
+   {{group}}
+   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+   {% endif %}
+
+   .. flat-table::
+      :header-rows: 2
+      :stub-columns: 1
+
+      * - :cspan:`5` Engines configured by default (in :ref:`settings.yml <engine settings>`)
+        - :cspan:`3` :ref:`Supported features <engine file>`
+
+      * - Name
+        - Shortcut
+        - Module
+        - Disabled
+        - Timeout
+        - Weight
+        - Paging
+        - Language
+        - Safe search
+        - Time range
+
+      {% for mod in engines %}
+
+      * - `{{mod.name}} <{{mod.about and mod.about.website}}>`_
+        - ``!{{mod.shortcut}}``
+        - {%- if 'searx.engines.' + mod.__name__ in documented_modules %}
+          :py:mod:`~searx.engines.{{mod.__name__}}`
+          {%- else %}
+          :origin:`{{mod.__name__}} <searx/engines/{{mod.__name__}}.py>`
+          {%- endif %}
+        - {{(mod.disabled and "y") or ""}}
+          {%- if mod.about and  mod.about.language %}
+          ({{mod.about.language | upper}})
+          {%- endif %}
+        - {{mod.timeout}}
+        - {{mod.weight or 1 }}
+        {% if mod.engine_type == 'online' %}
+        - {{(mod.paging and "y") or ""}}
+        - {{(mod.language_support and "y") or ""}}
+        - {{(mod.safesearch and "y") or ""}}
+        - {{(mod.time_range_support and "y") or ""}}
+        {% else %}
+        - :cspan:`3` not applicable ({{mod.engine_type}})
+        {% endif %}
+
+     {% endfor %}
+     {% endfor %}
+     {% endfor %}
+
+
+

The context for the template is selected in the line .. jinja:: searx. In +sphinx’s build configuration (git://docs/conf.py) the searx context +contains the engines and plugins.

+
import searx.search
+import searx.engines
+import searx.plugins
+searx.search.initialize()
+jinja_contexts = {
+   'searx': {
+      'engines': searx.engines.engines,
+      'plugins': searx.plugins.plugins
+   },
+}
+
+
+
+
+

Tabbed views

+

With sphinx-tabs extension we have tabbed views. To provide installation +instructions with one tab per distribution we use the group-tabs directive, +others are basic-tabs and code-tabs. Below a group-tab example from +Build docs is shown:

+
.. tabs::
+
+   .. group-tab:: Ubuntu / debian
+
+      .. code-block:: sh
+
+         $ sudo apt install shellcheck
+
+   .. group-tab:: Arch Linux
+
+      .. code-block:: sh
+
+         $ sudo pacman -S shellcheck
+
+   .. group-tab::  Fedora / RHEL
+
+      .. code-block:: sh
+
+         $ sudo dnf install ShellCheck
+
+
+
+
+

Math equations

+ +

The input language for mathematics is LaTeX markup using the CTAN: amsmath +package.

+

To embed LaTeX markup in reST documents, use role :math: for +inline and directive .. math:: for block markup.

+
In :math:numref:`schroedinger general` the time-dependent Schrödinger equation
+is shown.
+
+.. math::
+   :label: schroedinger general
+
+    \mathrm{i}\hbar\dfrac{\partial}{\partial t} |\,\psi (t) \rangle =
+          \hat{H} |\,\psi (t) \rangle.
+
+
+
+

LaTeX math equation

+

In (1) the time-dependent Schrödinger equation +is shown.

+
+

(1)\mathrm{i}\hbar\dfrac{\partial}{\partial t} |\,\psi (t) \rangle = + \hat{H} |\,\psi (t) \rangle.

+
+

The next example shows the difference of \tfrac (textstyle) and \dfrac +(displaystyle) used in a inline markup or another fraction.

+
``\tfrac`` **inline example** :math:`\tfrac{\tfrac{1}{x}+\tfrac{1}{y}}{y-z}`
+``\dfrac`` **inline example** :math:`\dfrac{\dfrac{1}{x}+\dfrac{1}{y}}{y-z}`
+
+
+
+

Line spacing

+

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy +eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam +voluptua. … +\tfrac inline example \tfrac{\tfrac{1}{x}+\tfrac{1}{y}}{y-z} +At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd +gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

+

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy +eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam +voluptua. … +\tfrac inline example \dfrac{\dfrac{1}{x}+\dfrac{1}{y}}{y-z} +At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd +gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/search_api.html b/dev/search_api.html new file mode 100644 index 00000000..8ff83660 --- /dev/null +++ b/dev/search_api.html @@ -0,0 +1,236 @@ + + + + + + + + + Search API — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Search API

+

The search supports both GET and POST.

+

Furthermore, two endpoints / and /search are available for querying.

+

GET /

+

GET /search

+
+

Parameters

+ +
+
qrequired

The search query. This string is passed to external search services. Thus, +SearXNG supports syntax of each search service. For example, site:github.com +SearXNG is a valid query for Google. However, if simply the query above is +passed to any search engine which does not filter its results based on this +syntax, you might not get the results you wanted.

+

See more at Search syntax

+
+
categoriesoptional

Comma separated list, specifies the active search categories (see +Configured Engines)

+
+
enginesoptional

Comma separated list, specifies the active search engines (see +Configured Engines).

+
+
languagedefault from search:

Code of the language.

+
+
pagenodefault 1

Search page number.

+
+
time_rangeoptional

[ day, month, year ]

+

Time range of search for engines which support it. See if an engine supports +time range search in the preferences page of an instance.

+
+
formatoptional

[ json, csv, rss ]

+

Output format of results. Format needs to be activated in search:.

+
+
results_on_new_tabdefault 0

[ 0, 1 ]

+

Open search results on new tab.

+
+
image_proxydefault from server:

[ True, False ]

+

Proxy image results through SearXNG.

+
+
autocompletedefault from search:

[ google, dbpedia, duckduckgo, startpage, wikipedia, +swisscows, qwant ]

+

Service which completes words as you type.

+
+
safesearchdefault from search:

[ 0, 1, 2 ]

+

Filter search results of engines which support safe search. See if an engine +supports safe search in the preferences page of an instance.

+
+
themedefault simple

[ simple ]

+

Theme of instance.

+

Please note, available themes depend on an instance. It is possible that an +instance administrator deleted, created or renamed themes on their instance. +See the available options in the preferences page of the instance.

+
+
enabled_pluginsoptional

List of enabled plugins.

+
+
default:
+

Hash_plugin, Search_on_category_select, +Self_Information, Tracker_URL_remover, +Ahmia_blacklist

+
+
values:
+

Hash_plugin, Search_on_category_select, +Self_Information, Tracker_URL_remover, +Ahmia_blacklist,

+

Hostname_replace, Open_Access_DOI_rewrite, +Vim-like_hotkeys, Tor_check_plugin

+
+
+
+
disabled_plugins: optional

List of disabled plugins.

+
+
default:
+

Hostname_replace, Open_Access_DOI_rewrite, +Vim-like_hotkeys, Tor_check_plugin

+
+
values:
+

see values from enabled_plugins

+
+
+
+
enabled_enginesoptionalall engines

List of enabled engines.

+
+
disabled_enginesoptionalall engines

List of disabled engines.

+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/searxng_extra/index.html b/dev/searxng_extra/index.html new file mode 100644 index 00000000..93f3ddb8 --- /dev/null +++ b/dev/searxng_extra/index.html @@ -0,0 +1,171 @@ + + + + + + + + + Tooling box searxng_extra — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Tooling box searxng_extra

+

In the folder git://searxng_extra/ we maintain some tools useful for CI and +developers.

+
+

Contents

+ +
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/searxng_extra/standalone_searx.py.html b/dev/searxng_extra/standalone_searx.py.html new file mode 100644 index 00000000..a1012286 --- /dev/null +++ b/dev/searxng_extra/standalone_searx.py.html @@ -0,0 +1,265 @@ + + + + + + + + + searxng_extra/standalone_searx.py — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

searxng_extra/standalone_searx.py

+

Script to run SearXNG from terminal.

+

Getting categories without initiate the engine will only return [‘general’]

+
>>> import searx.engines
+... list(searx.engines.categories.keys())
+['general']
+>>> import searx.search
+... searx.search.initialize()
+... list(searx.engines.categories.keys())
+['general', 'it', 'science', 'images', 'news', 'videos', 'music', 'files', 'social media', 'map']
+
+
+

Example to use this script:

+
$ python3 searxng_extra/standalone_searx.py rain
+
+
+
+

Danger

+

Be warned, using the standalone_searx.py won’t give you privacy!

+

On the contrary, this script behaves like a SearXNG server: your IP is +exposed and tracked by all active engines (google, bing, qwant, … ), with +every query!

+
+

Example to run it from python:

+
>>> import importlib
+... import json
+... import sys
+... import searx.engines
+... import searx.search
+... search_query = 'rain'
+... # initialize engines
+... searx.search.initialize()
+... # load engines categories once instead of each time the function called
+... engine_cs = list(searx.engines.categories.keys())
+... # load module
+... spec = importlib.util.spec_from_file_location(
+...     'utils.standalone_searx', 'searxng_extra/standalone_searx.py')
+... sas = importlib.util.module_from_spec(spec)
+... spec.loader.exec_module(sas)
+... # use function from module
+... prog_args = sas.parse_argument([search_query], category_choices=engine_cs)
+... search_q = sas.get_search_query(prog_args, engine_categories=engine_cs)
+... res_dict = sas.to_dict(search_q)
+... sys.stdout.write(json.dumps(
+...     res_dict, sort_keys=True, indent=4, ensure_ascii=False,
+...     default=sas.json_serial))
+{
+    "answers": [],
+    "infoboxes": [ {...} ],
+    "paging": true,
+    "results": [... ],
+    "results_number": 820000000.0,
+    "search": {
+        "lang": "all",
+        "pageno": 1,
+        "q": "rain",
+        "safesearch": 0,
+        "timerange": null
+    },
+    "suggestions": [...]
+}
+
+
+
+
+searxng_extra.standalone_searx.get_search_query(args: Namespace, engine_categories: Optional[List[str]] = None) → SearchQuery[source]
+

Get search results for the query

+
+ +
+
+searxng_extra.standalone_searx.json_serial(obj: Any) → Any[source]
+

JSON serializer for objects not serializable by default json code.

+
+
Raises:
+

TypeError – raised when obj is not serializable

+
+
+
+ +
+
+searxng_extra.standalone_searx.no_parsed_url(results: List[Dict[str, Any]]) → List[Dict[str, Any]][source]
+

Remove parsed url from dict.

+
+ +
+
+searxng_extra.standalone_searx.parse_argument(args: Optional[List[str]] = None, category_choices: Optional[List[str]] = None) → Namespace[source]
+

Parse command line.

+
+
Raises:
+

SystemExit – Query argument required on args

+
+
+

Examples:

+
>>> import importlib
+... # load module
+... spec = importlib.util.spec_from_file_location(
+...     'utils.standalone_searx', 'utils/standalone_searx.py')
+... sas = importlib.util.module_from_spec(spec)
+... spec.loader.exec_module(sas)
+... sas.parse_argument()
+usage: ptipython [-h] [--category [{general}]] [--lang [LANG]] [--pageno [PAGENO]] [--safesearch [{0,1,2}]] [--timerange [{day,week,month,year}]]
+                 query
+SystemExit: 2
+>>> sas.parse_argument(['rain'])
+Namespace(category='general', lang='all', pageno=1, query='rain', safesearch='0', timerange=None)
+
+
+
+ +
+
+searxng_extra.standalone_searx.to_dict(search_query: SearchQuery) → Dict[str, Any][source]
+

Get result from parsed arguments.

+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/searxng_extra/update.html b/dev/searxng_extra/update.html new file mode 100644 index 00000000..8fd653bf --- /dev/null +++ b/dev/searxng_extra/update.html @@ -0,0 +1,302 @@ + + + + + + + + + searxng_extra/update/ — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

searxng_extra/update/

+

[source]

+

Scripts to update static data in git://searx/data/

+
+

update_ahmia_blacklist.py

+

[source]

+

This script saves Ahmia’s blacklist for onion sites.

+

Output file: git://searx/data/ahmia_blacklist.txt (CI Update data +…).

+
+
+

update_currencies.py

+

[source]

+

Fetch currencies from git://searx/engines/wikidata.py engine.

+

Output file: git://searx/data/currencies.json (CI Update data …).

+
+
+

update_engine_descriptions.py

+

[source]

+

Fetch website description from websites and from +git://searx/engines/wikidata.py engine.

+

Output file: git://searx/data/engine_descriptions.json.

+
+
+searxng_extra.update.update_engine_descriptions.get_output()[source]
+

From descriptions[engine][language] = [description, source] +To

+
    +

  • output[language][engine] = description_and_source

    • +
  • +
    description_and_source can be:
      +

    • [description, source]

      • +

    • description (if source = “wikipedia”)

      • +

    • [f”engine:lang”, “ref”] (reference to another existing description)

      • +
    +
    +
    +
    • +
+
+ +
+
+

update_external_bangs.py

+

[source]

+

Update git://searx/data/external_bangs.json using the duckduckgo bangs +(CI Update data …).

+

https://duckduckgo.com/newbang loads:

+ +

This script loads the javascript, then the bangs.

+

The javascript URL may change in the future ( for example +https://duckduckgo.com/bv2.js ), but most probably it will requires to update +RE_BANG_VERSION

+
+
+searxng_extra.update.update_external_bangs.merge_when_no_leaf(node)[source]
+

Minimize the number of nodes

+

A -> B -> C

+
    +

  • B is child of A

    • +

  • C is child of B

    • +
+

If there are no C equals to <LEAF_KEY>, then each C are merged +into A. For example (5 nodes):

+
d -> d -> g -> <LEAF_KEY> (ddg)
+  -> i -> g -> <LEAF_KEY> (dig)
+
+
+

becomes (3 noodes):

+
d -> dg -> <LEAF_KEY>
+  -> ig -> <LEAF_KEY>
+
+
+
+ +
+
+

update_firefox_version.py

+

[source]

+

Fetch firefox useragent signatures

+

Output file: git://searx/data/useragents.json (CI Update data …).

+
+
+

update_languages.py

+

[source]

+

This script generates languages.py from intersecting each engine’s supported +languages.

+

Output files: git://searx/data/engines_languages.json and +git://searx/languages.py (CI Update data …).

+
+
+class searxng_extra.update.update_languages.UnicodeEscape[source]
+

Escape unicode string in pprint.pformat

+
+ +
+
+searxng_extra.update.update_languages.get_unicode_flag(lang_code)[source]
+

Determine a unicode flag (emoji) that fits to the lang_code

+
+ +
+
+

update_osm_keys_tags.py

+

[source]

+

Fetch OSM keys and tags.

+

To get the i18n names, the scripts uses Wikidata Query Service instead of for +example OSM tags API (sidenote: the actual change log from +map.atownsend.org.uk might be useful to normalize OSM tags).

+

Output file: git://searx/data/osm_keys_tags (CI Update data …).

+
+
SPARQL_TAGS_REQUEST :

Wikidata SPARQL query that returns type-categories and types. The +returned tag is Tag:{category}={type} (see get_tags()). +Example:

+ +
+
SPARQL_KEYS_REQUEST :

Wikidata SPARQL query that returns keys. Example with “payment”:

+ +

rdfs:label get all the labels without language selection +(as opposed to SERVICE wikibase:label).

+
+
+
+
+

update_pygments.py

+

[source]

+

Update pygments style

+

Call this script after each upgrade of pygments

+
+
+

update_wikidata_units.py

+

[source]

+

Fetch units from git://searx/engines/wikidata.py engine.

+

Output file: git://searx/data/wikidata_units.json (CI Update data +…).

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/dev/translation.html b/dev/translation.html new file mode 100644 index 00000000..65644b88 --- /dev/null +++ b/dev/translation.html @@ -0,0 +1,201 @@ + + + + + + + + + Translation — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Translation

+ +

Translation takes place on translate.codeberg.org.

+

Translations which has been added by translators on the translate.codeberg.org UI are +committed to Weblate’s counterpart of the SearXNG origin repository which is +located at https://translate.codeberg.org/git/searxng/searxng.

+

There is no need to clone this repository, SearXNG’s PR workflow to be in sync with Weblate take +care of the synchronization with the origin. To avoid merging commits from +the counterpart directly on the master branch of SearXNG origin, a pull +request (PR) is created by this workflow.

+

Weblate monitors the translations branch, not the master branch. This +branch is an orphan branch, decoupled from the master branch (we already know +orphan branches from the gh-pages). The translations branch contains +only the

+
    +

  • translation/messages.pot and the

    • +

  • translation/*/messages.po files, nothing else.

    • +
+
+../_images/translation.svg
+

Fig. 2 SearXNG’s PR workflow to be in sync with Weblate

+
+
+
+
Sync from origin to weblate: using make weblate.push.translations

For each commit on the master branch of SearXNG origin the GitHub job +babel / Update translations branch checks for updated translations.

+
+
Sync from weblate to origin: using make weblate.translations.commit

Every Friday, the GitHub workflow babel / create PR for additons from +weblate creates a PR with the +updated translation files:

+
    +

  • translation/messages.pot,

    • +

  • translation/*/messages.po and

    • +

  • translation/*/messages.mo

    • +
+
+
+
+

wlc

+

All weblate integration is done by GitHub workflows, but if you want to use wlc, +copy this content into wlc configuration in your HOME ~/.config/weblate

+
[keys]
+https://translate.codeberg.org/api/ = APIKEY
+
+
+

Replace APIKEY by your API key.

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/donate.html b/donate.html new file mode 100644 index 00000000..a4e2699f --- /dev/null +++ b/donate.html @@ -0,0 +1,159 @@ + + + + + + + + + Donate to searxng.org — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ + + + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/genindex.html b/genindex.html new file mode 100644 index 00000000..b901961c --- /dev/null +++ b/genindex.html @@ -0,0 +1,871 @@ + + + + + + + + + Index — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ + +

Index

+ +
+ A + | B + | C + | D + | E + | F + | G + | H + | I + | J + | L + | M + | N + | O + | P + | Q + | R + | S + | T + | U + +
+

A

+ + +
+ +

B

+ + +
+ +

C

+ + + +
+ +

D

+ + + +
+ +

E

+ + + +
+ +

F

+ + + +
+ +

G

+ + + +
+ +

H

+ + + +
+ +

I

+ + + +
+ +

J

+ + +
+ +

L

+ + + +
+ +

M

+ + +
+ +

N

+ + + +
+ +

O

+ + + +
+ +

P

+ + + +
+ +

Q

+ + + +
+ +

R

+ + + +
+ +

S

+ + + +
+ +

T

+ + + +
+ +

U

+ + + +
+ + + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/index.html b/index.html new file mode 100644 index 00000000..b5a57dfc --- /dev/null +++ b/index.html @@ -0,0 +1,222 @@ + + + + + + + + + Welcome to SearXNG — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Welcome to SearXNG

+
+

Search without being tracked.

+
+

SearXNG is a free internet metasearch engine which aggregates results from more +than 70 search services. Users are neither tracked nor profiled. Additionally, +SearXNG can be used over Tor for online anonymity.

+

Get started with SearXNG by using one of the instances listed at searx.space. +If you don’t trust anyone, you can set up your own, see Installation.

+ + +
+

Contents

+ +
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/objects.inv b/objects.inv new file mode 100644 index 00000000..cce09baa Binary files /dev/null and b/objects.inv differ diff --git a/own-instance.html b/own-instance.html new file mode 100644 index 00000000..b2a60513 --- /dev/null +++ b/own-instance.html @@ -0,0 +1,197 @@ + + + + + + + + + Why use a private instance? — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Why use a private instance?

+
+

“Is it worth to run my own instance?”

+
+

.. is a common question among SearXNG users. Before answering this question, +see what options a SearXNG user has.

+

Public instances are open to everyone who has access to its URL. Usually, these +are operated by unknown parties (from the users’ point of view). Private +instances can be used by a select group of people. It is for example a SearXNG of +group of friends or a company which can be accessed through VPN. Also it can be +single user one which runs on the user’s laptop.

+

To gain more insight on how these instances work let’s dive into how SearXNG +protects its users.

+
+

How does SearXNG protect privacy?

+

SearXNG protects the privacy of its users in multiple ways regardless of the type +of the instance (private, public). Removal of private data from search requests +comes in three forms:

+
+
    +

  1. removal of private data from requests going to search services

    2. +

  2. not forwarding anything from a third party services through search services +(e.g. advertisement)

    3. +

  3. removal of private data from requests going to the result pages

    4. +
+
+

Removing private data means not sending cookies to external search engines and +generating a random browser profile for every request. Thus, it does not matter +if a public or private instance handles the request, because it is anonymized in +both cases. IP addresses will be the IP of the instance. But SearXNG can be +configured to use proxy or Tor. Result proxy is supported, too.

+

SearXNG does not serve ads or tracking content unlike most search services. So +private data is not forwarded to third parties who might monetize it. Besides +protecting users from search services, both referring page and search query are +hidden from visited result pages.

+
+

What are the consequences of using public instances?

+

If someone uses a public instance, they have to trust the administrator of that +instance. This means that the user of the public instance does not know whether +their requests are logged, aggregated and sent or sold to a third party.

+

Also, public instances without proper protection are more vulnerable to abusing +the search service, In this case the external service in exchange returns +CAPTCHAs or bans the IP of the instance. Thus, search requests return less +results.

+
+
+

I see. What about private instances?

+

If users run their own instances, everything is in their +control: the source code, logging settings and private data. Unknown instance +administrators do not have to be trusted.

+

Furthermore, as the default settings of their instance is editable, there is no +need to use cookies to tailor SearXNG to their needs. So preferences will not be +reset to defaults when clearing browser cookies. As settings are stored on +their computer, it will not be accessible to others as long as their computer is +not compromised.

+
+
+
+

Conclusion

+

Always use an instance which is operated by people you trust. The privacy +features of SearXNG are available to users no matter what kind of instance they +use.

+

If someone is on the go or just wants to try SearXNG for the first time public +instances are the best choices. Additionally, public instance are making a +world a better place, because those who cannot or do not want to run an +instance, have access to a privacy respecting search service.

+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/py-modindex.html b/py-modindex.html new file mode 100644 index 00000000..f769fe53 --- /dev/null +++ b/py-modindex.html @@ -0,0 +1,279 @@ + + + + + + + + + Python Module Index — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + + + +
+
+
+
+ + +

Python Module Index

+ +
+ s +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
 
+ s
+ searx +
    + searx.babel_extract +
    + searx.engines +
    + searx.engines.demo_offline +
    + searx.engines.demo_online +
    + searx.engines.google +
    + searx.engines.google_images +
    + searx.engines.google_news +
    + searx.engines.google_videos +
    + searx.engines.tineye +
    + searx.engines.xpath +
    + searx.engines.yahoo +
    + searx.infopage +
    + searx.locales +
    + searx.plugins.autodetect_search_language +
    + searx.plugins.limiter +
    + searx.plugins.tor_check +
    + searx.redisdb +
    + searx.redislib +
    + searx.utils +
+ searxng_extra +
    + searxng_extra.standalone_searx +
    + searxng_extra.update.update_ahmia_blacklist +
    + searxng_extra.update.update_currencies +
    + searxng_extra.update.update_engine_descriptions +
    + searxng_extra.update.update_external_bangs +
    + searxng_extra.update.update_firefox_version +
    + searxng_extra.update.update_languages +
    + searxng_extra.update.update_osm_keys_tags +
    + searxng_extra.update.update_pygments +
    + searxng_extra.update.update_wikidata_units +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/search.html b/search.html new file mode 100644 index 00000000..46ef7eb6 --- /dev/null +++ b/search.html @@ -0,0 +1,132 @@ + + + + + + + + + Search — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+ +

Search

+ + + + +

+ Searching for multiple words only shows matches that contain + all words. +

+ + +
+ + + +
+ + + +
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/searchindex.js b/searchindex.js new file mode 100644 index 00000000..cbea635e --- /dev/null +++ b/searchindex.js @@ -0,0 +1 @@ +Search.setIndex({"docnames": ["admin/api", "admin/architecture", "admin/buildhosts", "admin/engines/command-line-engines", "admin/engines/configured_engines", "admin/engines/index", "admin/engines/nosql-engines", "admin/engines/private-engines", "admin/engines/recoll", "admin/engines/search-indexer-engines", "admin/engines/searx.engines.xpath", "admin/engines/settings", "admin/engines/sql-engines", "admin/index", "admin/installation", "admin/installation-apache", "admin/installation-docker", "admin/installation-nginx", "admin/installation-scripts", "admin/installation-searxng", "admin/installation-uwsgi", "admin/plugins", "admin/update-searxng", "dev/contribution_guide", "dev/engine_overview", "dev/index", "dev/lxcdev", "dev/makefile", "dev/offline_engines", "dev/plugins", "dev/quickstart", "dev/reST", "dev/search_api", "dev/searxng_extra/index", "dev/searxng_extra/standalone_searx.py", "dev/searxng_extra/update", "dev/translation", "donate", "index", "own-instance", "src/index", "src/searx.babel_extract", "src/searx.engines", "src/searx.engines.demo_offline", "src/searx.engines.demo_online", "src/searx.engines.google", "src/searx.engines.tineye", "src/searx.engines.yahoo", "src/searx.infopage", "src/searx.locales", "src/searx.plugins.autodetect_search_language", "src/searx.plugins.limiter", "src/searx.plugins.tor_check", "src/searx.redisdb", "src/searx.redislib", "src/searx.search", "src/searx.utils", "user/index", "utils/index", "utils/lxc.sh", "utils/searxng.sh"], "filenames": ["admin/api.rst", "admin/architecture.rst", "admin/buildhosts.rst", "admin/engines/command-line-engines.rst", "admin/engines/configured_engines.rst", "admin/engines/index.rst", "admin/engines/nosql-engines.rst", "admin/engines/private-engines.rst", "admin/engines/recoll.rst", "admin/engines/search-indexer-engines.rst", "admin/engines/searx.engines.xpath.rst", "admin/engines/settings.rst", "admin/engines/sql-engines.rst", "admin/index.rst", "admin/installation.rst", "admin/installation-apache.rst", "admin/installation-docker.rst", "admin/installation-nginx.rst", "admin/installation-scripts.rst", "admin/installation-searxng.rst", "admin/installation-uwsgi.rst", "admin/plugins.rst", "admin/update-searxng.rst", "dev/contribution_guide.rst", "dev/engine_overview.rst", "dev/index.rst", "dev/lxcdev.rst", "dev/makefile.rst", "dev/offline_engines.rst", "dev/plugins.rst", "dev/quickstart.rst", "dev/reST.rst", "dev/search_api.rst", "dev/searxng_extra/index.rst", "dev/searxng_extra/standalone_searx.py.rst", "dev/searxng_extra/update.rst", "dev/translation.rst", "donate.rst", "index.rst", "own-instance.rst", "src/index.rst", "src/searx.babel_extract.rst", "src/searx.engines.rst", "src/searx.engines.demo_offline.rst", "src/searx.engines.demo_online.rst", "src/searx.engines.google.rst", "src/searx.engines.tineye.rst", "src/searx.engines.yahoo.rst", "src/searx.infopage.rst", "src/searx.locales.rst", "src/searx.plugins.autodetect_search_language.rst", "src/searx.plugins.limiter.rst", "src/searx.plugins.tor_check.rst", "src/searx.redisdb.rst", "src/searx.redislib.rst", "src/searx.search.rst", "src/searx.utils.rst", "user/index.rst", "utils/index.rst", "utils/lxc.sh.rst", "utils/searxng.sh.rst"], "titles": ["Administration API", "Architecture", "Buildhosts", "Command Line Engines", "Configured Engines", "Engines & Settings", "NoSQL databases", "Private Engines (tokens)", "Recoll Engine", "Local Search Engines", "XPath Engine", "settings.yml", "SQL Engines", "Administrator documentation", "Installation", "Apache", "Docker Container", "NGINX", "Installation Script", "Step by step installation", "uWSGI", "Plugins builtin", "SearXNG maintenance", "How to contribute", "Engine Overview", "Developer documentation", "Developing in Linux Containers", "Makefile", "Offline Engines", "Plugins", "Development Quickstart", "reST primer", "Search API", "Tooling box searxng_extra", "searxng_extra/standalone_searx.py", "searxng_extra/update/", "Translation", "Donate to searxng.org", "Welcome to SearXNG", "Why use a private instance?", "Source-Code", "Custom message extractor (i18n)", "Load Engines", "Demo Offline Engine", "Demo Online Engine", "Google Engines", "Tineye", "Yahoo Engine", "Online /info", "Locales", "Search language plugin", "Limiter Plugin", "Tor check plugin", "Redis DB", "Redis Library", "Search", "Utility functions for the engines", "User information", "DevOps tooling box", "utils/lxc.sh", "utils/searxng.sh"], "terms": {"config": [0, 7, 19, 20, 22, 24, 28, 31, 36, 59], "http": [0, 2, 8, 9, 10, 11, 12, 13, 16, 18, 19, 20, 22, 23, 24, 26, 27, 28, 29, 30, 31, 35, 36, 38, 45, 46, 47, 50, 52, 56, 59, 60], "1": [0, 2, 4, 6, 10, 11, 15, 16, 17, 19, 20, 26, 27, 31, 32, 34, 49, 54, 55, 56, 60], "autocomplet": [0, 1, 11, 16, 19, 32], "categori": [0, 11, 12, 19, 21, 24, 31, 32, 34, 35, 55], "map": [0, 3, 10, 11, 20, 31, 34, 35, 42, 47, 49, 50, 57], "imag": [0, 2, 11, 13, 18, 19, 22, 25, 26, 27, 32, 34, 40, 44, 46, 57, 59], "default_local": [0, 11, 19], "default_them": [0, 11, 19], "simpl": [0, 6, 9, 10, 11, 12, 19, 24, 26, 27, 30, 32, 42, 43, 44, 46, 54], "engin": [0, 13, 19, 23, 25, 29, 31, 32, 34, 35, 38, 39, 40, 46, 49, 50, 55], "enabl": [0, 3, 4, 7, 9, 11, 12, 15, 16, 17, 19, 20, 23, 24, 27, 31, 32, 45, 51, 52], "true": [0, 1, 6, 9, 10, 15, 17, 20, 24, 28, 29, 31, 32, 34, 42, 45, 51, 53, 56], "name": [0, 3, 4, 6, 7, 9, 10, 11, 12, 15, 17, 19, 20, 21, 22, 25, 26, 28, 29, 35, 37, 40, 42, 43, 44, 45, 48, 49, 52, 54, 55, 56, 57, 59], "openstreetmap": [0, 4, 35], "shortcut": [0, 3, 4, 6, 7, 9, 11, 24, 31, 42, 43, 44], "osm": [0, 4, 24, 35], "arch": [0, 2, 4, 7, 11, 15, 17, 19, 20, 31], "linux": [0, 2, 4, 7, 11, 15, 16, 17, 19, 20, 25, 31, 38, 59], "wiki": [0, 7, 11, 16, 17, 19, 31, 35, 59], "al": [0, 4, 7, 50], "googl": [0, 4, 11, 19, 27, 32, 34, 38, 40], "goi": [0, 4], "fals": [0, 1, 6, 10, 11, 12, 19, 29, 31, 32, 34, 43, 44, 45, 48, 56], "bitbucket": [0, 4, 10], "bb": [0, 4], "instance_nam": [0, 1, 11, 16, 19], "searx": [0, 2, 6, 8, 10, 11, 12, 14, 15, 17, 19, 20, 22, 23, 24, 26, 27, 29, 31, 34, 35, 37, 38, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 53, 54, 55, 56, 59, 60], "local": [0, 1, 5, 8, 11, 13, 15, 16, 17, 19, 20, 22, 24, 26, 27, 29, 38, 40, 43, 48, 50, 53, 59, 60], "de": [0, 4, 10, 11, 12, 19, 20, 45, 50, 56, 60], "deutsch": 0, "german": [0, 11, 50], "en": [0, 10, 11, 19, 20, 27, 31, 47, 48, 50, 56], "english": [0, 11, 50, 56], "eo": [0, 31, 50], "esperanto": 0, "plugin": [0, 2, 11, 12, 13, 19, 20, 23, 25, 31, 32, 38, 40, 55], "rewrit": [0, 19, 21], "vim": [0, 19, 21, 32], "like": [0, 2, 6, 7, 9, 11, 12, 15, 16, 19, 21, 23, 24, 27, 30, 31, 34, 50, 52], "hotkei": [0, 19, 21], "safe_search": [0, 1, 10, 11, 19], "0": [0, 1, 4, 6, 10, 11, 15, 16, 17, 19, 20, 22, 23, 24, 26, 27, 31, 32, 34, 46, 53, 54, 55, 56, 58, 59, 60], "The": [0, 1, 2, 3, 6, 7, 9, 10, 11, 12, 13, 14, 16, 18, 19, 20, 22, 27, 28, 29, 31, 32, 35, 36, 39, 41, 43, 44, 45, 46, 49, 50, 51, 52, 54, 58, 59, 60], "can": [0, 1, 2, 3, 6, 7, 8, 9, 11, 12, 15, 16, 17, 18, 19, 20, 22, 23, 24, 26, 27, 28, 29, 30, 31, 35, 37, 38, 39, 41, 45, 46, 49, 50, 53, 54, 56, 57, 59, 60], "embed": [0, 16, 31], "websit": [0, 11, 31, 35, 46], "just": [0, 6, 9, 12, 22, 26, 27, 30, 31, 39, 43, 44, 59], "past": 0, "exampl": [0, 3, 6, 7, 9, 10, 12, 15, 19, 20, 22, 23, 24, 25, 26, 28, 32, 34, 35, 39, 43, 44, 49, 50, 52, 56, 57, 58, 59], "html": [0, 2, 11, 12, 15, 19, 20, 23, 24, 26, 27, 31, 45, 48, 56], "site": [0, 10, 11, 13, 22, 24, 31, 32, 35, 60], "url": [0, 1, 10, 11, 12, 15, 16, 19, 21, 23, 24, 26, 27, 29, 34, 35, 39, 44, 45, 46, 47, 51, 53, 56, 59], "searxng": [0, 1, 2, 4, 6, 11, 12, 13, 14, 18, 20, 21, 23, 24, 27, 28, 29, 30, 31, 32, 34, 36, 41, 46, 48, 49, 50, 52, 53, 54, 57, 58], "instanc": [0, 1, 3, 6, 7, 9, 11, 12, 14, 15, 16, 17, 19, 20, 22, 26, 27, 28, 29, 32, 38, 46, 48, 49, 54, 59, 60], "valu": [0, 3, 6, 10, 11, 12, 16, 19, 24, 28, 31, 32, 35, 42, 45, 48, 49, 51, 54, 56, 60], "ar": [0, 2, 3, 4, 6, 9, 10, 11, 12, 16, 18, 19, 20, 22, 23, 26, 27, 28, 29, 30, 31, 32, 35, 36, 38, 40, 41, 45, 46, 47, 49, 50, 51, 54, 57], "customiz": 0, "form": [0, 11, 19, 31, 39, 50], "method": [0, 9, 14, 19, 20, 22, 24, 31, 38, 41, 48, 56], "post": [0, 19, 23, 26, 29, 32], "action": [0, 20], "org": [0, 2, 8, 10, 11, 19, 20, 21, 24, 31, 35, 36, 38, 52, 56, 59], "input": [0, 7, 31], "type": [0, 3, 9, 10, 11, 18, 19, 23, 25, 27, 29, 32, 35, 39, 42, 45, 48, 56, 59], "text": [0, 8, 9, 24, 31, 50, 56], "q": [0, 11, 16, 19, 27, 32, 34], "hidden": [0, 7, 39], "gener": [0, 1, 10, 12, 15, 16, 19, 20, 23, 25, 34, 35, 39, 50, 57], "social": [0, 11, 34], "media": [0, 10, 11, 25, 29, 34], "languag": [0, 4, 10, 11, 19, 20, 21, 24, 27, 31, 32, 35, 38, 40, 45, 47, 48, 49, 55, 56], "lang": [0, 10, 20, 34, 35, 47, 55, 56], "all": [0, 6, 9, 10, 11, 12, 15, 16, 18, 19, 20, 22, 23, 24, 26, 27, 28, 29, 30, 31, 32, 34, 35, 36, 44, 45, 48, 50, 52, 54, 55, 56, 58, 59, 60], "date": [0, 24, 46], "filter": [0, 10, 11, 19, 27, 32, 43, 44, 57], "time_rang": [0, 10, 24, 32, 55], "month": [0, 10, 24, 32, 34], "revers": [1, 16, 46, 59], "proxi": [1, 11, 15, 16, 19, 22, 24, 26, 32, 39, 58, 59], "apach": [1, 11, 13, 18, 20, 22, 38, 59, 60], "nginx": [1, 11, 13, 18, 22, 26, 38, 58, 59, 60], "step": [1, 13, 14, 18, 22, 23, 26, 30, 38, 60], "instal": [1, 2, 6, 9, 11, 12, 13, 15, 16, 17, 20, 23, 25, 26, 28, 29, 31, 38, 58], "herein": 1, "you": [1, 2, 3, 6, 8, 9, 11, 12, 14, 15, 16, 17, 18, 19, 20, 22, 23, 26, 27, 28, 30, 31, 32, 34, 36, 37, 38, 39, 43, 46, 57, 58, 59, 60], "find": [1, 3, 6, 7, 9, 11, 12, 16, 19, 22, 23, 46, 57, 59], "some": [1, 7, 10, 11, 15, 16, 17, 18, 19, 20, 22, 23, 26, 27, 30, 31, 33, 45, 50, 51, 58, 59, 60], "hint": [1, 20, 31, 42], "suggest": [1, 10, 29, 34], "about": [1, 7, 11, 22, 23, 26, 27, 38, 59], "typic": [1, 27, 31], "infrastructur": [1, 20, 22], "we": [1, 2, 7, 11, 15, 17, 18, 19, 20, 26, 27, 29, 30, 31, 33, 36, 37, 40, 43, 44, 45, 46, 49, 58, 59], "start": [1, 15, 16, 17, 18, 19, 20, 23, 24, 25, 27, 28, 30, 38, 42, 46, 59], "refer": [1, 8, 15, 17, 18, 31, 35, 39, 50, 55, 56], "public": [1, 3, 6, 9, 12, 16, 21, 24, 31], "which": [1, 3, 4, 10, 11, 12, 18, 19, 20, 23, 24, 26, 27, 28, 30, 31, 32, 35, 36, 38, 39, 43, 44, 49, 54], "build": [1, 10, 11, 13, 14, 19, 26, 30, 35, 44, 46, 47, 59, 60], "up": [1, 7, 11, 14, 15, 16, 17, 19, 20, 22, 26, 27, 31, 38, 44, 45, 59], "maintain": [1, 16, 19, 20, 22, 26, 31, 33, 50, 58], "script": [1, 13, 14, 15, 16, 17, 19, 20, 22, 26, 27, 34, 35, 38, 45, 54, 58, 59, 60], "from": [1, 2, 7, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 28, 29, 30, 31, 32, 34, 35, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 50, 52, 53, 54, 55, 56, 57, 59, 60], "our": [1, 14, 16, 18, 23, 26, 27, 30, 31, 40, 58, 59], "devop": [1, 18, 22, 26, 38, 59], "tool": [1, 2, 8, 18, 19, 22, 23, 25, 26, 28, 31, 38, 59], "box": [1, 18, 22, 25, 26, 29, 38, 59], "activ": [1, 11, 15, 19, 23, 26, 29, 32, 34, 42, 45, 50, 57, 60], "server": [1, 12, 13, 16, 18, 19, 26, 27, 32, 34, 37, 46, 51, 54, 58, 59, 60], "limit": [1, 3, 6, 7, 9, 10, 11, 12, 15, 16, 17, 19, 20, 22, 24, 28, 31, 38, 40, 45, 54], "image_proxi": [1, 11, 19, 32], "ui": [1, 15, 17, 19, 20, 24, 36, 45], "static_use_hash": [1, 11, 15, 17, 19, 20], "etc": [1, 11, 15, 16, 17, 19, 20, 22, 23, 26, 55, 59, 60], "set": [1, 2, 3, 4, 6, 8, 9, 10, 12, 13, 15, 16, 17, 19, 20, 22, 23, 26, 27, 28, 29, 31, 32, 38, 39, 42, 43, 44, 45, 49, 51, 52, 53, 54, 56, 57, 58, 59, 60], "yml": [1, 4, 5, 6, 9, 12, 13, 15, 16, 17, 19, 20, 22, 26, 27, 28, 29, 31, 32, 43, 44, 51, 52, 53, 56, 60], "use_default_set": 1, "debug": [1, 11, 13, 18, 19, 26, 27], "search": [1, 3, 5, 6, 7, 8, 10, 12, 13, 16, 19, 21, 23, 24, 25, 26, 28, 29, 31, 34, 38, 39, 40, 43, 45, 46, 47, 52], "2": [1, 4, 10, 11, 19, 20, 24, 27, 30, 31, 32, 34, 46, 50, 54, 56, 57], "duckduckgo": [1, 4, 11, 19, 32, 35, 57], "secret_kei": [1, 11, 19, 54], "ultrasecretkei": [1, 11, 19], "redi": [1, 16, 19, 20, 22, 27, 28, 38, 40, 51, 60], "unix": [1, 11, 15, 17, 19, 20, 53], "usr": [1, 11, 15, 16, 17, 19, 20, 22, 26, 27, 53, 60], "run": [1, 2, 3, 9, 11, 15, 17, 18, 19, 20, 22, 23, 25, 26, 29, 30, 31, 34, 39, 53, 58, 60], "sock": [1, 11, 19, 20, 22, 53, 60], "db": [1, 6, 11, 12, 16, 19, 20, 22, 38, 40, 54], "If": [2, 3, 6, 7, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 22, 23, 26, 27, 28, 29, 30, 31, 35, 37, 38, 39, 42, 44, 47, 49, 54, 56, 58, 59], "have": [2, 6, 9, 11, 12, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 26, 27, 28, 30, 31, 39, 49, 59], "ani": [2, 11, 12, 14, 15, 17, 18, 23, 26, 27, 31, 32, 34, 38, 48, 49, 56, 59], "contribut": [2, 25, 27, 30, 38], "send": [2, 11, 20, 30, 39], "u": [2, 4, 11, 19, 26, 27, 30, 31, 37, 45, 56], "your": [2, 3, 6, 8, 9, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 27, 28, 29, 30, 31, 34, 36, 38, 43, 44, 46, 57, 59], "pr": [2, 22, 23, 30, 31, 36], "see": [2, 6, 7, 10, 11, 12, 14, 15, 16, 17, 19, 20, 22, 23, 26, 27, 29, 31, 32, 35, 38, 41, 42, 45, 49, 51, 54, 56, 57, 59], "how": [2, 11, 12, 13, 14, 15, 16, 17, 18, 20, 25, 26, 27, 30, 31, 38, 45, 46], "To": [2, 6, 7, 10, 11, 12, 15, 16, 18, 19, 20, 22, 23, 24, 26, 27, 28, 30, 31, 35, 36, 39, 43, 44, 49, 50, 51, 57, 59, 60], "get": [2, 9, 11, 13, 15, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 28, 30, 31, 32, 34, 35, 38, 43, 44, 45, 48, 49, 50, 51, 53, 54, 56, 59, 60], "best": [2, 27, 31, 39, 49, 56], "result": [2, 3, 6, 9, 10, 11, 12, 19, 21, 24, 28, 29, 31, 32, 34, 38, 39, 43, 44, 45, 46, 49, 55, 56], "its": [2, 6, 14, 23, 31, 32, 39, 42, 43, 44, 46, 56, 59], "recommend": [2, 14, 19, 20, 22, 23, 27, 31, 50], "addit": [2, 6, 11, 12, 15, 20, 27, 28, 31, 41, 45, 49], "packag": [2, 6, 12, 13, 16, 20, 26, 27, 28, 31, 41, 59, 60], "host": [2, 6, 11, 15, 16, 17, 19, 20, 23, 38, 59], "util": [2, 6, 11, 12, 15, 17, 18, 19, 22, 26, 27, 28, 29, 34, 38, 40, 45, 58], "sh": [2, 6, 12, 15, 16, 17, 18, 22, 26, 27, 28, 29, 31, 38, 50, 58], "sudo": [2, 6, 11, 12, 15, 16, 17, 18, 19, 20, 22, 26, 28, 29, 31, 58, 59, 60], "h": [2, 3, 11, 15, 16, 17, 18, 19, 20, 21, 22, 26, 31, 34, 58, 59, 60], "ubuntu": [2, 4, 15, 17, 19, 20, 27, 31, 59], "debian": [2, 16, 17, 19, 31], "fedora": [2, 15, 17, 19, 20, 31, 59], "rhel": [2, 15, 17, 19, 20, 31], "apt": [2, 15, 16, 17, 19, 31], "y": [2, 4, 19, 21, 31], "python3": [2, 19, 20, 34], "dev": [2, 4, 15, 17, 19, 27, 31], "babel": [2, 19, 36, 41, 49], "venv": [2, 19], "uwsgi": [2, 13, 14, 16, 17, 18, 19, 22, 26, 38, 60], "git": [2, 8, 11, 16, 18, 19, 23, 24, 26, 27, 29, 30, 31, 33, 35, 36, 41, 58, 60], "essenti": [2, 19], "libxslt": [2, 19], "zlib1g": [2, 19], "libffi": [2, 19], "libssl": [2, 19], "pacman": [2, 15, 17, 19, 31], "": [2, 6, 7, 10, 11, 12, 13, 16, 19, 22, 23, 24, 26, 27, 28, 31, 35, 36, 39, 41, 45, 49, 50, 52, 54, 56, 59, 60], "noconfirm": [2, 19], "python": [2, 6, 11, 12, 19, 20, 24, 25, 26, 28, 31, 34, 41, 49, 50, 56, 60], "pip": [2, 6, 12, 19, 27, 28, 29], "lxml": [2, 19, 56], "base": [2, 3, 6, 8, 9, 10, 11, 16, 19, 21, 24, 26, 32, 56, 59], "devel": [2, 19], "libxml2": [2, 19], "dnf": [2, 15, 17, 19, 31], "develop": [2, 3, 6, 7, 9, 12, 19, 23, 27, 28, 31, 33, 38, 56, 58], "openssl": [2, 11, 19], "docuemt": 2, "test": [2, 11, 12, 15, 16, 17, 19, 23, 25, 26, 30, 53, 56, 59, 60], "firefox": [2, 19, 27, 35], "graphviz": 2, "imagemagick": [2, 31], "texliv": 2, "xetex": 2, "librsvg2": 2, "bin": [2, 4, 16, 19, 20, 27, 59], "latex": 2, "extra": [2, 11, 15, 19, 25], "font": 2, "dejavu": 2, "latexmk": 2, "shellcheck": [2, 31, 59], "librsvg": 2, "core": [2, 20], "latexextra": 2, "ttf": 2, "gd": [2, 50], "collect": [2, 6, 9, 46, 54], "fontsrecommend": 2, "san": 2, "serif": 2, "mono": 2, "dvisvgm": 2, "most": [2, 9, 11, 12, 15, 27, 35, 39, 50, 60], "requir": [2, 6, 9, 10, 11, 12, 15, 20, 21, 22, 23, 24, 27, 28, 31, 32, 34, 35, 42, 46, 51, 60], "setup": [2, 11, 13, 14, 15, 16, 17, 18, 19, 22, 26, 27, 28, 30, 31, 45, 53, 58], "py": [2, 3, 6, 8, 9, 11, 12, 15, 17, 19, 24, 25, 27, 28, 31, 33, 45, 48, 53, 60], "scratch": 2, "make": [2, 7, 10, 11, 16, 19, 23, 25, 26, 30, 31, 36, 37, 39, 59], "For": [2, 6, 12, 15, 17, 18, 19, 20, 23, 24, 26, 27, 31, 32, 35, 36, 45, 50, 59], "better": [2, 15, 16, 17, 19, 24, 39], "math": [2, 25], "process": [2, 7, 16, 20, 26, 27, 28], "onli": [2, 3, 7, 10, 11, 12, 16, 18, 19, 20, 23, 24, 27, 29, 30, 31, 34, 36, 42, 46, 49, 50, 54, 59], "pdf": [2, 24], "creation": [2, 31], "also": [2, 6, 11, 15, 16, 19, 20, 22, 23, 26, 27, 28, 30, 31, 39, 42, 46, 50, 56, 57], "equat": [2, 25], "when": [2, 3, 11, 15, 16, 18, 19, 21, 28, 29, 30, 31, 34, 39, 40, 42, 45, 49, 56], "output": [2, 7, 27, 31, 32, 35, 59, 60], "i": [2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 26, 27, 28, 29, 30, 31, 32, 34, 35, 36, 37, 38, 40, 41, 42, 43, 44, 45, 46, 47, 49, 50, 51, 52, 54, 56, 59, 60], "abl": [2, 11, 23, 31, 57], "do": [2, 6, 7, 9, 12, 14, 15, 17, 18, 20, 21, 23, 26, 27, 31, 39, 43, 44, 45, 46, 49, 50, 59], "support": [2, 4, 6, 9, 10, 11, 12, 17, 19, 20, 23, 24, 29, 31, 32, 35, 37, 38, 39, 45, 46, 47, 48, 49, 50, 57, 59], "without": [2, 6, 11, 16, 20, 24, 27, 34, 35, 38, 39, 48, 49], "cdn": 2, "render": [2, 48], "ext": 2, "imgmath": 2, "extens": [2, 20, 31], "here": [2, 9, 10, 11, 12, 20, 22, 23, 24, 26, 27, 30, 31], "extract": [2, 40, 41, 56], "conf": [2, 15, 17, 20, 31, 60], "file": [2, 3, 5, 8, 11, 15, 16, 17, 19, 20, 22, 23, 26, 27, 29, 34, 35, 36, 41, 46, 48, 59], "html_math_render": 2, "imgmath_image_format": 2, "svg": 2, "imgmath_font_s": 2, "14": [2, 27, 56], "show": [2, 15, 19, 20, 22, 26, 27, 31, 57, 59], "warn": [2, 27, 31, 34, 59], "dot": [2, 27], "found": [2, 20, 27, 28, 46, 47, 56], "qualiti": [2, 27, 31], "www": [2, 11, 15, 19, 31, 35, 56, 59], "command": [2, 5, 11, 13, 15, 19, 20, 22, 23, 26, 27, 28, 31, 34, 36, 38, 54, 60], "cannot": [2, 31, 39, 57], "displai": [2, 6, 11, 12, 16, 19, 21, 24], "check": [2, 3, 11, 13, 16, 21, 23, 24, 27, 36, 38, 40, 49, 51, 56, 59, 60], "imgmath_latex": 2, "us": [2, 3, 6, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 26, 27, 28, 30, 31, 33, 34, 35, 36, 38, 42, 43, 44, 45, 46, 47, 48, 49, 50, 53, 54, 56, 57, 58, 59, 60], "A": [2, 3, 7, 11, 16, 22, 23, 24, 29, 31, 35, 41, 48, 51, 52, 53, 54, 56, 59], "static": [2, 11, 15, 17, 19, 20, 22, 27, 29, 30, 35], "analysi": 2, "offlin": [3, 7, 24, 25, 31, 38, 40], "With": [3, 11, 12, 15, 16, 26, 27, 31, 59], "administr": [3, 7, 9, 11, 19, 28, 32, 38, 39, 58], "integr": [3, 9, 27, 36, 38], "arbitrari": [3, 6, 19], "shell": [3, 13, 19, 25, 26, 59], "creat": [3, 11, 13, 15, 16, 17, 18, 20, 26, 27, 31, 32, 36, 42, 54, 59], "must": [3, 6, 8, 11, 12, 20, 23, 24, 29, 31, 42, 56], "care": [3, 11, 31, 36, 59], "avoid": [3, 11, 18, 21, 36], "leak": 3, "privat": [3, 5, 6, 9, 11, 12, 13, 25, 38], "data": [3, 6, 9, 12, 13, 24, 27, 31, 35, 39, 45, 56], "easiest": 3, "solut": [3, 59], "access": [3, 6, 7, 8, 9, 11, 12, 15, 18, 19, 20, 21, 26, 27, 28, 31, 39], "token": [3, 5, 6, 9, 11, 12, 13, 28], "describ": [3, 6, 9, 11, 12, 15, 18, 20, 26, 27, 31, 56, 60], "section": [3, 6, 9, 10, 11, 12, 14, 15, 17, 18, 19, 26, 27, 29, 31, 52, 59], "flexibl": [3, 31], "imagin": 3, "power": [3, 9, 12, 31], "thi": [3, 6, 7, 8, 9, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 26, 27, 28, 29, 30, 31, 32, 34, 35, 36, 37, 39, 40, 41, 42, 43, 44, 45, 46, 49, 50, 52, 53, 54, 57, 59], "mayb": 3, "secur": [3, 19, 20, 25, 37, 38], "concern": [3, 23], "follow": [3, 6, 8, 9, 11, 12, 15, 16, 17, 18, 19, 20, 23, 24, 27, 31, 42, 43, 44, 59], "option": [3, 6, 7, 11, 12, 15, 16, 19, 20, 23, 26, 27, 29, 31, 32, 34, 39, 41, 42, 45, 48, 55, 56], "avail": [3, 11, 15, 16, 17, 19, 20, 21, 27, 28, 32, 39, 60], "comma": [3, 7, 32], "separ": [3, 7, 23, 30, 31, 32], "list": [3, 7, 10, 11, 12, 15, 16, 22, 24, 25, 27, 28, 29, 32, 34, 38, 43, 44, 45, 46, 48, 49, 52, 55, 56, 57], "element": [3, 31, 56], "special": [3, 14, 15, 17, 20, 24, 45], "queri": [3, 6, 9, 10, 11, 12, 19, 21, 24, 28, 32, 34, 35, 39, 43, 44, 45, 46, 47, 50, 52, 54, 55], "tell": [3, 24], "where": [3, 8, 11, 12, 20, 24, 26, 28, 31, 52, 54], "put": [3, 12], "term": [3, 9, 10, 15, 26, 31, 43, 50], "user": [3, 7, 10, 11, 13, 16, 18, 20, 21, 23, 24, 26, 27, 31, 37, 38, 39, 49, 50, 52, 56, 60], "l": [3, 11, 16, 27, 31, 59], "delimit": [3, 31, 49], "contain": [3, 11, 13, 14, 19, 20, 21, 25, 28, 31, 35, 36, 38, 42, 55, 58, 59], "char": 3, "titl": [3, 10, 11, 12, 19, 24, 48], "each": [3, 9, 10, 11, 19, 20, 22, 24, 26, 28, 29, 31, 32, 34, 35, 36, 54, 56, 59], "kei": [3, 6, 11, 12, 16, 19, 21, 24, 34, 35, 36, 45, 54], "parse_regex": 3, "dict": [3, 24, 29, 34, 42, 45, 46, 49, 55, 56], "regular": 3, "express": 3, "query_typ": [3, 9], "expect": [3, 7, 19, 23, 50], "possibl": [3, 10, 11, 16, 19, 24, 26, 31, 32], "path": [3, 11, 15, 19, 22, 26, 31, 56, 59], "enum": 3, "provid": [3, 7, 9, 23, 24, 31, 35], "insid": [3, 19, 26, 59], "work": [3, 11, 16, 19, 23, 24, 25, 27, 30, 39, 59], "directori": [3, 4, 11, 15, 16, 20, 48, 49], "execut": [3, 11, 27, 41, 59], "allow": [3, 11, 15, 19, 31, 45], "submit": [3, 9, 23], "someth": [3, 23, 27, 30], "includ": [3, 7, 12, 15, 16, 17, 18, 26, 27, 31, 59], "return": [3, 10, 11, 12, 21, 24, 28, 29, 31, 34, 35, 39, 42, 43, 45, 48, 49, 54, 56], "an": [3, 6, 7, 9, 10, 11, 12, 14, 19, 23, 24, 26, 27, 28, 31, 32, 36, 37, 39, 42, 46, 49, 50, 55, 56, 57], "error": [3, 10, 11, 19, 20, 22, 24, 28, 31, 46, 56], "query_enum": 3, "working_dir": 3, "ha": [3, 7, 11, 20, 22, 23, 24, 28, 31, 36, 38, 39, 45, 48, 49, 54, 59], "default": [3, 4, 6, 11, 12, 15, 16, 17, 19, 20, 23, 27, 28, 29, 31, 32, 34, 38, 39, 42, 45, 48, 49, 50, 53, 54, 56, 58, 59, 60], "result_separ": 3, "charact": [3, 19, 23, 31], "n": [3, 59], "below": [3, 6, 11, 12, 18, 19, 20, 23, 24, 29, 31], "specif": [3, 11, 20, 24, 47], "configur": [3, 5, 7, 9, 10, 11, 13, 15, 16, 17, 20, 25, 26, 27, 28, 31, 32, 36, 39, 41, 42, 56], "fnd": 3, "wa": [3, 6, 7, 9, 12, 19, 22, 23, 24, 26, 27, 28, 46, 50, 53, 54], "sponsor": [3, 6, 7, 9, 12, 28], "discoveri": [3, 6, 7, 9, 12, 28], "fund": [3, 6, 7, 9, 12, 28, 37], "nlnet": [3, 6, 7, 9, 12, 28], "foundat": [3, 6, 7, 9, 12, 28, 29], "overview": [4, 10, 11, 16, 25, 27, 32, 35, 38, 58], "explan": [4, 31], "shown": [4, 11, 15, 18, 19, 20, 26, 31, 52], "tabl": [4, 25], "139": 4, "61": 4, "featur": [4, 7, 20, 22, 23, 31, 39], "modul": [4, 20, 29, 31, 34, 41, 42, 43, 44], "disabl": [4, 11, 12, 13, 19, 20, 21, 26, 29, 31, 32, 43, 44, 59], "timeout": [4, 11, 19, 24, 31, 58], "weight": [4, 11, 31], "page": [4, 7, 9, 10, 11, 12, 15, 17, 19, 21, 23, 24, 25, 31, 32, 34, 35, 36, 39, 47, 48, 57], "safe": [4, 10, 31, 32], "time": [4, 10, 11, 12, 19, 23, 24, 26, 27, 31, 32, 34, 37, 39, 54, 56, 59], "rang": [4, 10, 20, 24, 31, 32, 54], "bing": [4, 34], "bi": [4, 11], "3": [4, 11, 16, 19, 27, 31, 35, 50, 54, 56, 59], "brave": 4, "xpath": [4, 5, 13, 56], "ddg": [4, 35, 57], "gigablast": 4, "gb": [4, 56], "4": [4, 19, 24, 27, 31, 34, 54], "go": [4, 6, 7, 27, 28, 30, 31, 39, 45], "mojeek": 4, "mjk": 4, "neeva": 4, "nv": 4, "5": [4, 11, 12, 16, 19, 31, 35, 54, 56], "qwant": [4, 11, 19, 32, 34], "qw": 4, "startpag": [4, 11, 19, 32], "sp": 4, "6": [4, 27, 31, 54, 56], "wibi": 4, "wib": 4, "json_engin": 4, "yahoo": [4, 38, 40], "yh": 4, "seznam": [4, 19], "szn": 4, "cz": 4, "goo": 4, "ja": [4, 47, 50], "naver": 4, "nvr": 4, "ko": [4, 47, 50], "alexandria": 4, "alx": 4, "archiv": [4, 11, 19, 27], "ai": 4, "7": [4, 10, 16, 19], "curli": 4, "cl": 4, "currenc": [4, 24, 35], "cc": [4, 56], "currency_convert": 4, "100": [4, 11, 19, 46], "applic": [4, 11, 20, 26, 31], "online_curr": 4, "definit": [4, 45], "ddd": 4, "duckduckgo_definit": 4, "dictzon": 4, "dc": 4, "online_dictionari": 4, "lingva": 4, "lv": [4, 50], "marginalia": 4, "mar": 4, "petalsearch": 4, "pt": [4, 49, 50], "tiney": [4, 38, 40], "tin": 4, "9": [4, 27], "online_url_search": [4, 46], "wikibook": 4, "wb": 4, "mediawiki": 4, "wikidata": [4, 35], "wd": 4, "wikipedia": [4, 11, 19, 32, 35, 57], "wp": [4, 57], "wikiquot": 4, "wq": 4, "wikisourc": 4, "w": 4, "wikivers": 4, "wv": 4, "wikivoyag": 4, "wy": 4, "yep": 4, "wikimini": 4, "wkmn": 4, "fr": [4, 10, 11, 19, 45, 49, 50, 56, 57], "bii": 4, "bing_imag": 4, "ddi": 4, "duckduckgo_imag": 4, "google_imag": [4, 45], "qwi": 4, "1x": 4, "www1x": 4, "artic": [4, 44], "arc": 4, "deviantart": 4, "da": [4, 47, 50], "flickr": 4, "fl": 4, "flickr_noapi": 4, "frinkiac": 4, "frk": 4, "librari": [4, 38, 40, 50], "congress": 4, "loc": 4, "openvers": 4, "opv": 4, "ptsi": 4, "petal_imag": 4, "unsplash": 4, "biv": 4, "bing_video": 4, "gov": 4, "google_video": [4, 45], "qwv": 4, "ccc": 4, "tv": 4, "c3tv": 4, "dailymot": 4, "dm": 4, "plai": 4, "movi": [4, 12], "gpm": 4, "invidi": 4, "iv": 4, "peertub": 4, "ptb": 4, "rumbl": 4, "ru": [4, 50], "sepiasearch": 4, "sep": 4, "vimeo": 4, "vm": 4, "youtub": 4, "yt": 4, "youtube_noapi": 4, "mediathekviewweb": 4, "mvw": 4, "ina": 4, "bing_new": 4, "gon": 4, "google_new": [4, 27, 45], "ptsn": 4, "qwn": 4, "wikinew": 4, "wn": 4, "yhn": 4, "yahoo_new": 4, "appl": 4, "apm": 4, "apple_map": 4, "photon": 4, "ph": 4, "azlyr": 4, "geniu": 4, "gen": 4, "bandcamp": 4, "bc": 4, "deezer": 4, "dz": 4, "gpodder": 4, "gpod": 4, "mixcloud": 4, "mc": [4, 31], "soundcloud": 4, "sc": [4, 50], "docker": [4, 13, 14, 18, 26, 27, 38, 58], "hub": 4, "dh": 4, "docker_hub": 4, "hoogl": 4, "ho": 4, "lib": [4, 15, 20], "r": [4, 19], "lr": [4, 27], "metacpan": 4, "cpan": 4, "npm": [4, 27], "packagist": 4, "pack": 4, "pub": 4, "pd": 4, "pypi": [4, 27, 31], "rubygem": 4, "rbg": 4, "askubuntu": 4, "stackexchang": 4, "stackoverflow": 4, "st": 4, "superus": 4, "su": [4, 50], "codeberg": [4, 36], "cb": 4, "github": [4, 11, 16, 18, 19, 26, 27, 29, 30, 31, 32, 36, 59, 60], "gh": [4, 23, 25, 36], "gitlab": 4, "gl": [4, 27, 50], "10": [4, 9, 11, 15, 19, 24, 26, 59], "sourcehut": 4, "srht": 4, "archlinux": [4, 7, 20, 59], "free": [4, 23, 24, 29, 38], "fsd": 4, "gentoo": 4, "ge": 4, "framalibr": 4, "frl": 4, "habrahabr": 4, "habr": 4, "lobst": 4, "lo": [4, 50], "mankier": 4, "man": [4, 31], "searchcod": 4, "code": [4, 10, 11, 15, 19, 20, 22, 24, 25, 27, 30, 32, 34, 38, 39, 45, 50, 54, 56, 59], "scc": 4, "searchcode_cod": 4, "arxiv": 4, "arx": 4, "crossref": 4, "cr": 4, "30": [4, 10, 11], "scholar": [4, 45], "google_scholar": 4, "pubm": 4, "semant": [4, 23], "se": 4, "semantic_scholar": 4, "openairedataset": 4, "oad": 4, "openairepubl": 4, "oap": 4, "pdbe": 4, "pdb": 4, "apk": 4, "mirror": 4, "apkm": 4, "apkmirror": 4, "store": [4, 6, 9, 24, 39, 51], "ap": 4, "apple_app_stor": 4, "fdroid": 4, "fd": 4, "gpa": 4, "google_play_app": 4, "1337x": 4, "btdigg": 4, "bt": 4, "kickass": 4, "kc": 4, "genesi": 4, "lg": 4, "nyaa": 4, "nt": 4, "openrepo": 4, "piratebai": 4, "tpb": 4, "solidtorr": 4, "solid": 4, "tokyotoshokan": 4, "tt": [4, 50], "9gag": 4, "9g": 4, "reddit": 4, "re": [4, 14, 16, 22, 27], "twitter": 4, "tw": [4, 47, 49, 50], "recol": [5, 13], "sql": [5, 13, 28], "nosql": [5, 13], "databas": [5, 11, 12, 13, 22, 31, 51, 54], "line": [5, 13, 15, 17, 23, 28, 34, 36, 59], "io": [6, 19, 20, 27, 31, 50], "abov": [6, 9, 11, 12, 15, 19, 22, 23, 26, 31, 32, 59], "comment": [6, 9, 12, 15, 19, 22, 24, 28, 31], "out": [6, 9, 12, 16, 23, 27, 29, 31, 43, 44, 45], "variou": [6, 11, 23, 26, 29, 45], "befor": [6, 11, 12, 18, 19, 20, 22, 23, 26, 27, 28, 29, 30, 31, 39, 59], "them": [6, 9, 11, 12, 20, 31], "By": [6, 8, 11, 12, 20, 45, 49, 58, 59], "templat": [6, 12, 19, 20, 24, 25, 26, 27, 28], "theme": [6, 11, 12, 19, 27, 30, 32], "satisfi": [6, 12], "origin": [6, 11, 12, 13, 22, 26, 31, 36, 43, 45, 46], "layout": [6, 11, 12], "own": [6, 11, 12, 15, 16, 18, 19, 26, 27, 28, 38, 39], "result_templ": [6, 12], "attribut": [6, 12, 28, 31, 42, 46], "template_nam": [6, 12], "place": [6, 12, 15, 17, 20, 23, 24, 31, 36, 39], "theme_nam": [6, 12], "furthermor": [6, 9, 24, 32, 39], "wish": [6, 11, 12], "expos": [6, 7, 9, 12, 34], "still": [6, 9, 11, 12, 45], "add": [6, 7, 9, 11, 12, 15, 16, 17, 18, 19, 20, 23, 29, 31, 40, 43, 44, 46, 55, 56], "first": [6, 10, 11, 14, 15, 22, 26, 27, 28, 30, 31, 39, 49, 56], "defin": [6, 11, 12, 16, 29, 31], "structur": [6, 23, 25], "need": [6, 10, 11, 12, 15, 16, 17, 18, 19, 20, 22, 23, 26, 27, 28, 29, 30, 31, 32, 36, 39, 44, 45, 51, 57, 59], "virtual": [6, 12, 15, 28], "environ": [6, 11, 12, 16, 19, 23, 25, 26, 28, 30, 38, 49, 59, 60], "switch": [6, 12, 15, 21, 22, 23, 28], "cmd": [6, 12, 26, 27, 28, 29, 58, 59, 60], "bash": [6, 12, 16, 19, 26, 27, 28, 29, 31, 59, 60], "pyenv": [6, 12, 19, 20, 26, 27, 28, 29, 60], "redis_serv": 6, "open": [6, 11, 12, 15, 16, 19, 21, 23, 26, 27, 31, 32, 39, 59], "sourc": [6, 10, 11, 12, 16, 19, 26, 27, 30, 31, 34, 35, 38, 39, 41, 42, 43, 44, 45, 46, 47, 48, 49, 54, 55, 56, 59], "bsd": 6, "licens": [6, 59], "memori": [6, 11], "select": [6, 9, 10, 11, 12, 19, 21, 27, 28, 30, 31, 35, 39, 41, 49, 50, 59], "index": [6, 8, 9, 15, 46, 56], "either": [6, 56], "look": [6, 15, 19, 20, 27, 30, 43, 44], "exact": 6, "match": [6, 9, 46, 56], "partial": [6, 31, 40, 54], "keyword": [6, 12, 41, 52, 57], "what": [6, 15, 17, 20, 22, 26, 31, 46, 54, 59], "exact_match_onli": 6, "myredi": 6, "rd": 6, "127": [6, 11, 15, 17, 19, 26, 60], "port": [6, 11, 16, 19, 23, 24, 27, 59], "6379": [6, 11], "enable_http": [6, 9, 11, 24], "password": [6, 9, 11, 12], "pymongo": 6, "document": [6, 7, 8, 9, 11, 15, 27, 31, 38, 40, 48, 59], "program": [6, 25, 27], "handl": [6, 11, 26, 31, 39, 50, 59], "json": [6, 9, 11, 19, 27, 32, 34, 35, 43, 45, 46, 56], "In": [6, 11, 15, 16, 17, 18, 19, 20, 22, 23, 24, 25, 29, 31, 33, 39, 43, 44, 57, 58, 59, 60], "order": [6, 9, 11, 12, 15, 19, 23, 31], "so": [6, 9, 11, 15, 16, 20, 23, 24, 26, 28, 29, 31, 39, 50, 51], "mymongo": 6, "md": [6, 11, 19], "27017": 6, "results_per_pag": 6, "20": [6, 19, 27, 56, 59], "busi": 6, "review": 6, "might": [7, 9, 10, 15, 16, 17, 20, 23, 26, 31, 32, 35, 39, 50, 59], "themselv": [7, 9], "want": [7, 9, 11, 12, 14, 16, 19, 20, 23, 27, 31, 32, 36, 37, 39, 58, 59], "It": [7, 9, 11, 12, 16, 23, 24, 31, 32, 39, 50, 52, 59], "becaus": [7, 23, 24, 39], "thei": [7, 11, 19, 20, 26, 31, 39], "inform": [7, 11, 19, 21, 23, 24, 26, 27, 38, 48, 52, 55, 59], "through": [7, 11, 19, 31, 32, 39], "Or": [7, 31], "would": [7, 26, 43], "rather": 7, "share": [7, 15, 20, 26, 27, 45, 59], "trust": [7, 38, 39], "friend": [7, 39], "colleagu": 7, "solv": 7, "issu": [7, 11, 19, 20, 24], "concept": 7, "exist": [7, 11, 14, 16, 19, 22, 23, 26, 27, 35, 54, 56, 59], "new": [7, 11, 12, 15, 16, 17, 18, 19, 20, 22, 27, 32, 34, 40, 50, 54], "ad": [7, 9, 11, 20, 28, 30, 31, 36, 39, 41, 45], "string": [7, 10, 19, 21, 24, 32, 35, 43, 49, 54, 56, 57], "request": [7, 9, 10, 11, 15, 19, 21, 23, 25, 27, 28, 29, 30, 31, 36, 39, 40, 44, 45, 46, 47, 48, 52, 55], "present": [7, 28], "one": [7, 11, 12, 15, 17, 19, 20, 23, 26, 27, 29, 31, 35, 37, 38, 39, 41, 42, 49, 56, 59], "restrict": 7, "my": [7, 9, 11, 16, 26, 31, 39, 43, 44, 50], "secret": [7, 11, 54], "unless": 7, "right": [7, 11, 20, 29, 31, 35, 49], "him": 7, "her": 7, "prefer": [7, 11, 14, 15, 17, 18, 19, 31, 32, 39, 47, 48, 50, 52, 57], "rest": [7, 16, 25, 27, 30, 38, 45], "api": [7, 8, 11, 13, 24, 25, 35, 36, 38, 40, 44, 46], "call": [7, 26, 27, 30, 34, 35, 45, 46, 54, 55, 56, 59], "under": [7, 24, 26, 50], "distribut": [7, 11, 15, 16, 17, 20, 26, 31, 50], "carv": 7, "stone": 7, "As": [7, 11, 20, 22, 39], "impli": 7, "admin": [7, 26, 28, 31], "know": [7, 15, 20, 26, 31, 36, 39, 51, 59], "necessari": [7, 22], "come": [7, 17, 23, 39, 50, 57, 59], "strict": [7, 10, 11, 19, 24], "instead": [7, 9, 10, 11, 20, 31, 34, 35, 54], "guidelin": [7, 30], "webui": 8, "desktop": [8, 11, 26], "full": [8, 9, 11, 24, 31, 45], "xapian": 8, "itself": [8, 20, 23, 45], "doe": [8, 11, 12, 15, 18, 20, 22, 24, 26, 28, 31, 32, 38, 46, 50, 54, 56], "offer": [8, 10, 20, 45], "web": [8, 9, 11, 16, 19, 24, 26, 27, 29, 40, 46, 47, 56, 59], "achiev": 8, "base_url": [8, 9, 11, 16, 19, 24, 27, 56], "locat": [8, 15, 17, 19, 20, 31, 36, 48], "reach": [8, 31, 54], "mount_prefix": 8, "hierarchi": 8, "mount": [8, 16], "filesystem": 8, "dl_prefix": 8, "search_dir": 8, "part": [8, 11, 31, 59], "empti": [8, 10, 11, 15, 31, 56], "domain": [8, 31, 37, 46, 47], "scenario": [8, 56], "export": [8, 16, 19, 26, 59], "interfac": [8, 11, 15, 19, 25], "content": [8, 9, 10, 12, 36, 39, 48], "though": 8, "download": [8, 11, 12, 18, 19, 26, 27, 31, 46], "comparison": 9, "altern": [9, 11, 18, 27, 30, 31, 59], "ones": [9, 24], "now": [9, 15, 17, 19, 22, 26, 29, 30], "capabl": [9, 23], "pleas": [9, 12, 22, 23, 30, 31, 32, 50, 57], "note": [9, 19, 23, 32, 57], "aim": [9, 31, 40], "individu": 9, "small": [9, 12, 24, 27, 60], "compani": [9, 39], "design": [9, 31, 37], "scale": [9, 26, 31, 59], "less": [9, 15, 17, 27, 31, 39], "than": [9, 11, 19, 31, 38, 56], "million": 9, "e": [9, 12, 16, 19, 23, 24, 26, 27, 30, 31, 39, 45, 48, 49, 50, 56, 57, 59, 60], "g": [9, 12, 16, 19, 23, 24, 26, 27, 30, 31, 35, 39, 45, 48, 49, 50, 57, 59, 60], "great": 9, "visit": [9, 16, 19, 31, 39], "later": [9, 27, 59], "facet": 9, "subset": [9, 56], "authent": [9, 12], "auth_token": 9, "me": [9, 31, 37, 59], "localhost": [9, 11, 15, 16, 17], "7700": 9, "guid": [9, 17, 31], "numer": [9, 31], "wai": [9, 11, 20, 23, 24, 31, 39], "At": [9, 11, 12, 19, 31], "moment": [9, 12], "popular": [9, 12, 31], "simple_query_str": 9, "none": [9, 10, 11, 15, 19, 24, 27, 28, 29, 31, 34, 42, 43, 48, 49, 50, 54, 55, 56], "fit": [9, 11, 20, 29, 35, 45, 49], "case": [9, 10, 11, 18, 19, 22, 26, 39, 56, 60], "custom": [9, 11, 19, 23, 31, 36, 38, 40, 45, 57], "payload": 9, "custom_query_json": 9, "9200": 9, "usernam": [9, 11, 12], "elast": 9, "changem": 9, "resourc": [9, 17], "lucen": 9, "But": [9, 11, 29, 39, 59], "indic": [9, 46], "ascend": 9, "slr": 9, "8983": 9, "sort": [9, 54], "asc": 9, "further": [10, 27, 50], "read": [10, 14, 19, 23, 26, 27, 46, 47], "search_url": [10, 46, 56], "repo": 10, "pageno": [10, 24, 32, 34, 55], "url_xpath": 10, "articl": [10, 16, 24, 25, 26, 48, 54], "class": [10, 35, 42, 46, 48, 55], "summari": [10, 25], "link": [10, 11, 19, 20, 25, 26, 35, 46], "href": 10, "title_xpath": 10, "content_xpath": 10, "p": [10, 15, 16, 17, 19, 20, 50, 59], "selector": 10, "first_page_num": 10, "number": [10, 11, 12, 19, 24, 31, 32, 35, 45, 46, 54, 56, 59], "usual": [10, 25, 39, 43], "header": [10, 11, 24, 31, 45, 50], "differ": [10, 11, 15, 20, 21, 23, 24, 26, 31, 57, 59], "cooki": [10, 24, 38, 39], "safesearch": [10, 24, 31, 32, 34, 45, 55], "moder": [10, 11, 19, 24], "lang_al": 10, "replac": [10, 19, 21, 22, 26, 27, 29, 36, 54], "no_result_for_http_statu": 10, "statu": [10, 11, 20, 27, 49, 60], "throw": 10, "page_s": 10, "offset": [10, 12], "param": [10, 24, 28, 31, 44, 45, 46, 47], "paramet": [10, 11, 12, 25, 28, 29, 42, 45, 48, 49, 54, 55, 56], "respons": [10, 11, 24, 28, 40, 44, 45, 46, 47], "resp": [10, 44, 45, 46, 47], "scrap": [10, 45], "results_xpath": 10, "item": [10, 16, 20, 24, 31, 40, 44, 46, 54], "safe_search_map": 10, "safes_search_map": 10, "safe_search_support": 10, "pag": 10, "iso": [10, 11, 24, 50], "639": [10, 50], "taken": [10, 11, 19, 31], "time_range_map": 10, "soft_max_redirect": [10, 24], "maximum": [10, 11, 19, 23, 24], "redirect": [10, 11, 21, 24, 27], "soft": [10, 24, 25], "record": [10, 11, 19, 24, 31], "don": [10, 11, 15, 16, 19, 22, 24, 31, 38, 59], "t": [10, 11, 15, 16, 19, 20, 22, 24, 29, 31, 34, 38, 42, 56, 59], "stop": [10, 16, 19, 20, 24, 26, 27, 29, 54, 59], "suggestion_xpath": 10, "thumbnail_xpath": 10, "img_src": [10, 24], "dai": [10, 20, 24, 32, 34], "24": [10, 57], "720": 10, "week": [10, 16, 24, 34], "168": [10, 11, 15], "year": [10, 20, 24, 32, 34], "8760": 10, "time_range_v": 10, "time_range_url": 10, "365": 10, "time_range_support": [10, 24, 31], "hour": 10, "initi": [11, 19, 20, 26, 31, 34, 42, 43, 44, 49, 53, 54, 59], "load": [11, 15, 19, 20, 27, 34, 35, 38, 40, 43, 59], "specifi": [11, 16, 20, 32, 45], "searxng_settings_path": [11, 19, 20, 26], "variabl": [11, 16, 24, 31], "simplifi": [11, 60], "issue_url": [11, 19], "com": [11, 16, 18, 19, 26, 27, 29, 30, 31, 32, 35, 46, 47, 56, 59, 60], "docs_url": [11, 19], "doc": [11, 13, 15, 19, 20, 25, 26, 31, 56, 59], "public_inst": [11, 19], "space": [11, 19, 27, 37, 38], "wiki_url": [11, 19], "tracker": [11, 19, 21], "chang": [11, 19, 20, 22, 23, 26, 27, 29, 30, 31, 35], "privacypolicy_url": [11, 19], "donation_url": [11, 19], "donat": [11, 19, 38], "contact_url": [11, 19], "enable_metr": [11, 19], "searxng_debug": [11, 27], "more": [11, 12, 19, 20, 23, 24, 27, 31, 32, 38, 39, 45, 50, 54, 56, 57, 59], "detail": [11, 15, 19, 20, 24, 31, 45, 46], "log": [11, 13, 16, 20, 26, 35, 39, 60], "directli": [11, 27, 36, 57, 59], "messag": [11, 23, 24, 30, 31, 36, 38, 40, 46], "browser": [11, 16, 19, 23, 26, 27, 39, 56, 57, 59], "too": [11, 19, 29, 31, 39, 46], "deactiv": [11, 19], "product": [11, 25, 30], "point": [11, 16, 17, 19, 20, 25, 26, 30, 31, 39, 46], "project": [11, 20, 26, 27, 31, 38], "written": [11, 19, 26], "info": [11, 19, 26, 27, 40, 59], "altogeth": 11, "privaci": [11, 15, 16, 17, 19, 20, 25, 26, 31, 34, 38, 51, 57], "polici": [11, 19, 59], "contact": [11, 19], "mailto": [11, 19], "address": [11, 19, 21, 24, 39, 51, 52], "anonym": [11, 38, 39], "metric": 11, "stat": [11, 19, 31], "default_lang": [11, 19], "ban_time_on_fail": [11, 19], "max_ban_time_on_fail": [11, 19], "120": [11, 19, 23], "format": [11, 19, 27, 28, 31, 32, 46], "backend": [11, 19], "leav": [11, 19], "blank": [11, 19, 31], "turn": [11, 19, 23, 46, 54], "off": [11, 15, 17, 19, 23, 59], "dbpedia": [11, 19, 32], "swisscow": [11, 19, 32], "detect": [11, 19, 21, 50], "unset": [11, 20], "otherwis": [11, 24, 30, 42], "meant": [11, 16, 31], "mean": [11, 20, 26, 28, 31, 39, 54], "IT": [11, 19], "BE": [11, 19, 49], "ban": [11, 19, 39], "second": [11, 19, 31, 49, 54], "after": [11, 19, 27, 29, 31, 35, 59], "max": [11, 19, 24, 54], "remov": [11, 16, 19, 20, 21, 26, 27, 29, 34, 39, 47, 54, 59, 60], "deni": [11, 15, 19, 20], "lower": [11, 19, 54], "csv": [11, 19, 32], "rss": [11, 19, 32], "8888": [11, 15, 17, 19, 26, 60], "bind_address": [11, 16, 19, 27], "listen": [11, 15, 17, 20, 59], "default_http_head": [11, 19], "x": [11, 15, 17, 19, 20, 31, 59], "nosniff": [11, 19], "xss": [11, 19], "protect": [11, 16, 19, 22, 23, 26, 38, 51, 57], "mode": [11, 13, 15, 19, 31, 59], "block": [11, 19, 20, 25, 45, 51, 54, 59], "noopen": [11, 19], "robot": [11, 19, 27], "tag": [11, 16, 19, 24, 35, 45, 46], "noindex": [11, 19], "nofollow": [11, 19], "referr": [11, 19], "rebuild": [11, 19, 23, 27], "env": [11, 15, 20, 25, 26, 30, 59, 60], "searxng_url": [11, 27, 59, 60], "deploi": [11, 16, 27, 31], "correct": 11, "inbound": 11, "forget": [11, 16, 19, 22, 59], "searxng_port": [11, 27, 60], "searxng_bind_address": [11, 27, 60], "bind": [11, 12, 16], "webapp": [11, 19, 20, 27], "doesn": [11, 29, 42, 56], "appli": [11, 23, 56], "searxng_secret": 11, "cryptographi": 11, "purpos": [11, 24], "rate": [11, 19, 45, 51], "bot": [11, 16, 19, 22, 26, 51], "being": [11, 38], "755": 11, "query_in_titl": [11, 19], "infinite_scrol": [11, 19], "center_align": [11, 19], "cache_url": [11, 19], "theme_arg": [11, 19], "simple_styl": [11, 19], "auto": [11, 19, 26, 31, 50, 59], "cach": [11, 16, 19, 20, 22, 48, 54, 56], "bust": [11, 16, 19, 20, 22], "decreas": [11, 19], "sinc": [11, 19, 20, 31], "automat": [11, 16, 19, 20, 21, 31], "next": [11, 19, 31], "scroll": [11, 19], "bottom": [11, 19, 35], "current": [11, 19, 23, 24, 27, 42, 54], "center": [11, 19], "left": [11, 22, 31, 45, 49], "rtl": 11, "side": [11, 18, 31, 35], "screen": 11, "affect": [11, 22, 45], "min": [11, 24, 50], "width": [11, 31, 46], "tablet": 11, "prefix": [11, 19, 45, 54, 57], "internet": [11, 19, 26, 28, 38, 58], "forgett": [11, 19], "trail": [11, 15, 19, 56], "slash": [11, 15, 19, 56], "webcach": [11, 19], "googleusercont": [11, 19], "todai": [11, 46], "style": [11, 19, 30, 35, 56], "light": [11, 19], "dark": [11, 19], "results_on_new_tab": [11, 19, 32], "tab": [11, 19, 25, 32, 59], "connect": [11, 17, 19, 20, 22, 26, 28, 38, 53, 58], "redisdb": [11, 20, 22, 53], "descript": [11, 12, 21, 23, 24, 28, 29, 31, 35, 40, 45, 49, 52, 56], "serxng": 11, "socket": [11, 15, 16, 17, 20, 60], "la": [11, 31, 50, 59], "srwxrwx": 11, "write": [11, 34], "given": [11, 31, 54, 56], "group": [11, 16, 20, 31, 39], "even": [11, 20, 26, 59], "account": [11, 18, 19, 22, 26, 37, 50], "from_url": [11, 19], "rediss": 11, "yaml": [11, 19, 27, 42], "Then": 11, "manag": [11, 12, 27, 60], "addgrp": 11, "logout": 11, "login": [11, 18, 19], "member": 11, "commun": [11, 15, 17, 19, 20, 28], "request_timeout": [11, 19], "overrid": [11, 19], "max_request_timeout": [11, 19], "useragent_suffix": [11, 19], "email": [11, 19], "pool_connect": [11, 19], "null": [11, 15, 17, 34], "pool_maxs": [11, 19], "keep": [11, 15, 17, 19, 26, 27, 29], "aliv": [11, 19], "alwai": [11, 16, 17, 22, 23, 24, 26, 39, 45, 50], "enable_http2": [11, 19], "httpx": [11, 19], "http2": [11, 19], "uncom": [11, 19], "certif": [11, 19], "advanc": [11, 19, 28], "verif": [11, 19], "compat": [11, 19, 37], "ssl": [11, 15, 19, 24, 38], "verifi": [11, 19, 24], "mitmproxi": [11, 19], "ca": [11, 19, 45, 49, 50], "cert": [11, 19], "cer": [11, 19], "proxyq": [11, 19], "latest": [11, 16, 19, 20, 27], "proxy1": [11, 19], "8080": [11, 16, 19, 26, 59], "proxy2": [11, 19], "using_tor_proxi": [11, 19, 40, 42], "extra_proxy_timeout": [11, 19], "made": [11, 26, 35], "other": [11, 16, 18, 23, 24, 27, 31, 39, 45, 56], "bigger": 11, "wait": [11, 54], "answer": [11, 31, 34, 39], "slow": 11, "consequ": [11, 23], "reactiv": 11, "mai": [11, 19, 20, 26, 31, 35, 46, 50, 59], "take": [11, 18, 20, 22, 23, 36, 46, 50, 54, 59], "suffix": [11, 16, 19], "agent": [11, 19, 21, 24, 56, 57], "keepalive_expiri": 11, "pool": [11, 19], "protocol": [11, 16, 26, 56], "round": [11, 23], "robin": 11, "fashion": 11, "source_ip": [11, 19], "multipl": [11, 21, 24, 27, 31, 39], "network": [11, 19, 59], "ip": [11, 15, 17, 21, 26, 34, 39, 51, 52, 59], "ipv4": [11, 59], "ipv6": [11, 26, 59], "192": [11, 15], "two": [11, 15, 16, 17, 20, 23, 31, 32, 37], "fe80": [11, 15, 19], "60a2": 11, "1691": 11, "e5a2": 11, "ee1f": 11, "126": [11, 19], "retri": 11, "On": [11, 15, 16, 26, 34, 45], "retry_on_http_error": 11, "between": [11, 24, 31, 50], "400": [11, 31], "599": 11, "403": [11, 19], "429": [11, 19, 26], "ssl_cert_fil": 11, "ssl_cert_dir": 11, "max_redirect": [11, 24], "syntax": [11, 32, 38, 50, 56], "video": [11, 34, 40], "music": [11, 34], "scienc": [11, 34], "fledg": 11, "dummi": 11, "demo": [11, 28, 38, 40, 54], "send_accept_language_head": 11, "api_kei": [11, 24], "apikei": [11, 36], "en_u": [11, 24], "weigth": 11, "display_error_messag": [11, 24], "wikidata_id": 11, "q306656": 11, "official_api_document": 11, "use_official_api": 11, "require_api_kei": 11, "404": 11, "max_connect": 11, "max_keepalive_connect": 11, "socks5": [11, 24], "proxy3": 11, "1080": 11, "socks5h": 11, "proxy4": 11, "across": 11, "bang": [11, 35], "should": [11, 14, 15, 17, 20, 22, 23, 26, 30, 31, 45, 49, 50], "stabl": [11, 19, 27], "everi": [11, 22, 23, 24, 26, 34, 36, 39, 40, 59], "updat": [11, 13, 16, 19, 23, 25, 27, 29, 33, 36, 42, 60], "touch": [11, 15, 17, 20, 26], "sever": [11, 26], "region": [11, 50], "deal": 11, "accept": [11, 19, 28, 50, 57, 59], "few": [11, 22, 23, 24, 57], "same": [11, 12, 19, 24, 26, 31, 56], "Be": [11, 15, 20, 34], "modifi": [11, 16, 19, 20, 27, 30, 31, 57], "obtain": 11, "delet": [11, 27, 32, 54, 59], "manual": [11, 22, 27, 31], "anoth": [11, 20, 24, 26, 31, 35, 59], "countri": [11, 24, 45], "fr_fr": [11, 49], "de_d": 11, "local_address": 11, "pretti": 11, "won": [11, 16, 20, 34], "workaround": 11, "speaker": 11, "reli": [11, 12], "actual": [11, 35], "except": [11, 15, 23, 24, 31, 56], "similar": [11, 12, 20, 31], "merg": [11, 20, 22, 26, 35, 36], "accord": [11, 16, 20, 49], "ecretvalu": 11, "keep_onli": 11, "relat": 12, "system": [12, 16, 19, 20, 23, 26, 27, 31, 59], "rdbm": 12, "mysql_serv": 12, "query_str": 12, "valid": [12, 24, 28, 32], "mani": [12, 16, 19], "basic": [12, 25, 46, 59], "dure": 12, "fast": 12, "reliabl": 12, "demonstr": [12, 29], "complex": 12, "mediathekview": 12, "filmlist": 12, "v2": 12, "bz2": 12, "unpack": 12, "concert": 12, "durat": [12, 54], "unixepoch": 12, "AS": 12, "coalesc": 12, "nullif": 12, "url_video_hd": 12, "url_video_sd": 12, "url_video": 12, "film": 12, "wildcard": 12, "OR": 12, "BY": 12, "desc": 12, "psycopg2": 12, "robust": 12, "psychopg2": 12, "my_databas": 12, "my_tabl": 12, "my_column": 12, "connector": 12, "said": 12, "auth_plugin": 12, "caching_sha2_password": 12, "depend": [13, 15, 17, 21, 22, 24, 25, 26, 27, 31, 32], "distributor": 13, "mainten": [13, 18, 26, 38, 60], "pitfal": 13, "tyrant": 13, "inspect": [13, 18, 26, 60], "migrat": [13, 14], "stai": [13, 14], "tune": [13, 14], "emb": [13, 31], "bar": [13, 31, 50, 53], "architectur": [13, 18, 38, 60], "builtin": [13, 29, 38], "buildhost": [13, 26, 27, 38, 58, 60], "lint": [13, 27], "spoilt": 14, "choic": [14, 26, 31, 39, 58], "choos": [14, 23, 30], "excel": 14, "illustr": 14, "grow": [14, 22], "rapidli": [14, 22], "regularli": [14, 22], "upgrad": [14, 22, 35], "explain": [15, 17, 23], "did": 15, "interest": [15, 16, 17], "problem": [15, 17, 19, 26, 59], "give": [15, 17, 20, 27, 34, 57], "guidanc": [15, 17], "apache2": [15, 60], "readm": [15, 20, 27], "direct": [15, 17, 25, 30, 31, 49], "orient": 15, "There": [15, 17, 20, 23, 30, 31, 36, 50, 56, 59], "pocket": [15, 17], "systemctl": [15, 17, 20, 26], "httpd": 15, "kind": [15, 31, 39], "welcom": [15, 17], "compar": [15, 16, 20, 26, 59], "000": 15, "documentroot": 15, "var": 15, "And": [15, 26, 31], "srv": 15, "followsymlink": 15, "allowoverrid": 15, "grant": [15, 20], "mod_autoindex": 15, "loadmodul": 15, "autoindex_modul": 15, "autoindex": 15, "fresh": 15, "d": [15, 16, 17, 20, 35, 56, 59], "awar": [15, 20, 31], "quit": [15, 16, 20, 23, 26], "standard": [15, 20, 23, 24, 45, 59], "gz": [15, 20], "apache2ctl": 15, "control": [15, 20, 39], "a2enmod": 15, "a2dismod": 15, "a2enconf": 15, "a2disconf": 15, "a2ensit": 15, "a2dissit": 15, "un": [15, 19], "correspond": [15, 20, 49], "proxy_http": 15, "proxy_uwsgi": 15, "ssl_modul": 15, "mod_ssl": 15, "headers_modul": 15, "mod_head": 15, "proxy_modul": 15, "mod_proxi": 15, "proxy_http_modul": 15, "mod_proxy_http": 15, "proxy_uwsgi_modul": 15, "mod_proxy_uwsgi": 15, "save": [15, 16, 19, 22, 35, 45, 59], "folder": [15, 17, 18, 20, 22, 24, 26, 27, 31, 33, 48, 58, 59], "includeopt": 15, "mkdir": [15, 16, 17, 19], "symlink": [15, 17], "ln": [15, 17, 20, 26], "old": [15, 16, 22, 26, 30], "mod_uwsgi": 15, "anymor": [15, 16], "incom": 15, "proxypreservehost": 15, "pass": [15, 23, 32, 43, 45, 50], "upstream": [15, 17, 20], "utf": [15, 20, 31, 59], "8": [15, 19, 20, 27, 31, 59], "setenvif_modul": 15, "mod_setenvif": 15, "setenvif": 15, "request_uri": 15, "dontlog": 15, "customlog": 15, "combin": [15, 54], "fd00": 15, "16": [15, 19, 27], "redirectmatch": 15, "308": 15, "proxypass": 15, "ud": 15, "flaskfix": [15, 17], "requesthead": 15, "scheme": [15, 17, 45], "request_schem": 15, "real": [15, 17], "remote_addr": [15, 17], "append": 15, "forward": [15, 17, 39, 59], "serv": [15, 17, 20, 23, 27, 39, 59], "alia": [15, 17], "src": [15, 17, 19, 20, 26, 45, 60], "restart": [15, 16, 17, 20, 26], "servic": [15, 17, 18, 19, 20, 26, 27, 31, 32, 35, 38, 39, 59, 60], "ini": [15, 16, 17, 20, 22, 26, 60], "entir": [15, 19, 54], "compon": [15, 29], "root": [15, 18, 26, 27, 59], "dockerhub": 16, "dockerfil": 16, "cheat": 16, "sheet": 16, "alpin": 16, "dash": 16, "intend": 16, "well": [16, 31], "those": [16, 26, 27, 28, 39], "who": [16, 20, 39, 51], "caddi": [16, 18], "against": [16, 19], "bandwith": [16, 19, 22], "plan": 16, "yourself": [16, 23], "sure": [16, 26], "back": [16, 26, 30, 54], "membership": 16, "evalu": [16, 31, 56, 59], "usermod": 16, "rm": [16, 20, 50], "clean": [16, 25, 59], "exit": [16, 19, 21, 52], "detach": 16, "v": [16, 19, 20, 23], "volum": [16, 24], "easi": [16, 26, 38], "pull": [16, 23, 30, 31, 36], "cd": [16, 18, 19, 26, 27], "pwd": [16, 26], "2f998": 16, "id": [16, 19, 24, 27, 50, 56], "xdg": [16, 19, 27], "flag": [16, 35], "stope": 16, "rid": [16, 27, 59], "2f998d725993": 16, "sbin": [16, 20], "tini": 16, "minut": 16, "ago": 16, "conatin": 16, "prune": 16, "aq": 16, "housekeep": 16, "rmi": 16, "f": [16, 31, 35, 56, 59], "drop": [16, 30, 44, 46, 54, 59], "tale": 16, "posix": 16, "compliant": 16, "entrypoint": 16, "exec": 16, "clone": [16, 18, 19, 26, 27, 30, 36, 60], "successfulli": [16, 27, 46], "built": [16, 23, 24], "49586c016434": 16, "209": 16, "9c823800": 16, "dirti": 16, "repositori": [16, 18, 26, 36, 59], "size": [16, 20, 22, 24, 46, 56], "13": [16, 20, 27], "308mb": 16, "6dbb9cc54074": 16, "61mb": 16, "interact": [16, 19, 58], "help": [16, 21, 23, 27, 37, 59, 60], "dry": 16, "renam": [16, 32], "copi": [16, 19, 31, 36, 59], "morty_url": 16, "result_proxi": [16, 19], "morty_kei": 16, "tcp": 16, "entri": [16, 25, 43, 44], "beginn": 17, "mainlin": 17, "webserv": 17, "80": 17, "server_nam": 17, "good": [17, 26, 59], "uwsgi_pass": 17, "uwsgi_param": 17, "http_host": 17, "http_connect": 17, "http_x_scheme": 17, "http_x_script_nam": 17, "http_x_real_ip": 17, "http_x_forwarded_for": 17, "proxy_add_x_forwarded_for": 17, "proxy_pass": 17, "proxy_set_head": 17, "proxy_buff": 17, "proxy_request_buff": 17, "proxy_buffer_s": 17, "8k": 17, "app": [17, 20, 26, 27, 31, 45, 48, 60], "access_log": 17, "error_log": 17, "unwant": 18, "effect": [18, 20, 23, 31], "procedur": [18, 22, 59], "task": [18, 19, 20, 26, 27, 31, 59], "jump": [18, 19, 26, 27, 57], "readabl": 18, "fork": [18, 23, 26, 38], "sudoer": 18, "fine": [18, 19], "whatev": [18, 59], "implement": [18, 20, 22, 23, 24, 27, 29, 30, 41, 42, 43, 44, 45, 46, 53, 54, 56, 59], "useradd": 19, "home": [19, 26, 27, 36], "dir": 19, "respect": [19, 23, 39], "metasearch": [19, 24, 38], "chown": 19, "virtualenv": [19, 20, 26, 27, 60], "m": [19, 31, 50], "echo": [19, 59], "profil": [19, 27, 38, 39], "session": [19, 49], "version": [19, 21, 24, 27, 31, 35, 49], "boilerpl": 19, "setuptool": 19, "wheel": 19, "pyyaml": 19, "tree": [19, 26, 27, 30], "termin": [19, 26, 34], "global": [19, 24, 27, 49, 54], "lock": [19, 56], "enabled_plugin": [19, 32, 52], "hash": [19, 20, 21, 51, 54, 57], "self": [19, 21, 38, 45, 59, 60], "ahmia": [19, 35], "blacklist": [19, 35], "hostnam": [19, 21, 59], "hostname_replac": [19, 32], "infinit": [19, 54], "doi": [19, 21, 24], "only_show_green_result": [19, 29], "brand": [19, 27, 60], "new_issue_url": 19, "yandex": 19, "minimun": 19, "autocomplete_min": 19, "suspend_tim": 19, "suspens": 19, "402": 19, "searxengineaccessdeni": 19, "86400": 19, "captcha": [19, 39], "searxenginecaptcha": 19, "searxenginetoomanyrequest": 19, "3600": 19, "cloudflar": 19, "cf_searxenginecaptcha": 19, "1296000": 19, "cf_searxengineaccessdeni": 19, "recaptcha": 19, "recaptcha_searxenginecaptcha": 19, "604800": 19, "buildenv": [19, 25], "http_protocol_vers": 19, "histori": 19, "caus": 19, "readthedoc": [19, 20], "client": [19, 26, 36, 50, 53, 54], "didn": 19, "static_path": 19, "templates_path": 19, "proxif": 19, "extern": [19, 24, 25, 28, 31, 32, 39], "asciimoo": 19, "morti": [19, 22, 26, 59], "base64": 19, "encod": [19, 20, 31, 45], "binari": [19, 27], "notat": 19, "commit": [19, 22, 26, 27, 30, 36], "af77ec3": 19, "3000": [19, 26], "your_morty_proxy_kei": 19, "button": [19, 45], "proxify_result": 19, "outgo": 19, "searx_userag": [19, 40, 56], "could": [19, 20, 24, 31, 46, 50, 54], "concurr": 19, "establish": 19, "plugin1": 19, "plugin2": 19, "noth": [19, 36], "tor": [19, 21, 38, 39, 40, 42], "detriment": [19, 50], "autodetect": [19, 21], "minim": [19, 27, 30, 31, 35], "cp": 19, "sed": [19, 31], "rand": 19, "hex": 19, "curl": 19, "verbos": 19, "head": 19, "insecur": 19, "try": [19, 20, 22, 39, 50], "tcp_nodelai": 19, "68": 19, "mark": [19, 31], "bundl": 19, "multius": 19, "assum": [19, 31], "close": [19, 43, 44, 46], "bodi": [19, 23, 31], "200": [19, 26, 27], "ok": [19, 22, 27], "everyth": [19, 31, 39], "hit": 19, "ctrl": [19, 26], "c": [19, 20, 26, 31, 35, 47, 50, 56], "enter": 19, "twice": 19, "demon": 19, "systemd": [20, 27], "unit": [20, 23, 27, 35], "emperor": 20, "vari": 20, "One": 20, "per": [20, 23, 24, 31, 35], "dedic": [20, 26, 31], "execstart": 20, "known": [20, 27, 49], "common": [20, 27, 38, 39, 45], "larg": [20, 59, 60], "multi": 20, "monitor": [20, 26, 36, 51], "event": 20, "scan": 20, "vassal": 20, "timestamp": 20, "reload": 20, "edit": [20, 23, 26, 27, 31, 39], "mostli": [20, 31], "both": [20, 23, 26, 32, 39], "interpret": [20, 27, 31, 45], "python2": 20, "while": [20, 26, 27, 30, 31, 43, 45, 54], "worth": [20, 39], "complet": [20, 22, 23, 32, 59, 60], "approach": 20, "familiar": [20, 27], "thing": [20, 31], "symbol": [20, 26], "recogn": 20, "init": [20, 26, 28, 40, 43, 44, 49, 59], "daemon": 20, "sighup": 20, "signal": [20, 26], "forc": 20, "sigterm": 20, "exactli": [20, 31], "argument": [20, 21, 23, 31, 34, 45, 49, 54], "confnam": 20, "systemctl_skip_redirect": 20, "hello": 20, "xml": [20, 31, 56], "lsb": 20, "bug": [20, 27], "cgi": 20, "bugreport": [20, 23], "833067": 20, "0pointer": 20, "blog": [20, 26, 31], "uid": [20, 26], "gid": 20, "ignor": [20, 43, 45], "lc_all": 20, "chdir": [20, 26], "chmod": [20, 59], "666": 20, "singl": [20, 39, 59], "master": [20, 22, 23, 31, 36, 60], "worker": 20, "lazi": 20, "gil": 20, "thread": [20, 56], "rememb": [20, 23, 26], "multithread": 20, "strang": 20, "behaviour": [20, 23, 29], "perform": [20, 21, 24, 28, 46, 54, 57], "reason": [20, 31, 42, 54], "wsgi": 20, "pythonhom": 20, "glob": 20, "pythonpath": [20, 26], "speak": 20, "buffer": [20, 22], "8192": 20, "expir": [20, 22, 54], "31557600": 20, "gzip": 20, "offload": 20, "k": 20, "cache2": 20, "searxngcach": 20, "2000": [20, 31], "blocksiz": 20, "4096": 20, "bitmap": 20, "logger": 20, "owner": 20, "somewhat": 20, "unusu": 20, "consider": 20, "initgroup": 20, "branch": [20, 22, 23, 27, 36], "2099": 20, "752": 20, "been": [20, 22, 23, 26, 36, 38, 54, 59], "oct": 20, "2014": 20, "had": 20, "never": [20, 26, 42], "releas": [20, 22, 27, 59], "last": [20, 24, 27, 30, 31], "major": 20, "dec": 20, "2013": 20, "bugfix": 20, "2425uwsgi": 20, "shorten": 20, "miss": [20, 28, 31], "permiss": 20, "993": 20, "fail": [20, 26, 27, 30, 42], "aef": 20, "grep": [20, 27, 59], "93": 20, "92": 20, "12": [20, 59], "43": 20, "00": [20, 59], "186": 20, "44": 20, "01": 20, "pid": 20, "cat": [20, 59], "proc": 20, "fdsize": 20, "128": 20, "j": [21, 25, 29, 30, 35], "css": [21, 27, 29, 30], "convert": [21, 24, 31, 56, 57], "digest": [21, 57], "paywal": 21, "immedi": [21, 54], "javascript": [21, 30, 35, 56], "node": [21, 25, 30, 35, 52, 56], "torproject": [21, 52], "navig": 21, "press": 21, "main": 21, "1332": 22, "456": 22, "roll": [22, 59], "opportun": 22, "filtron": [22, 26, 59], "longer": [22, 27, 59], "enough": [22, 37], "sometim": 22, "reconfigur": 22, "uninstal": [22, 27], "consid": 22, "reinstal": 22, "extent": 22, "1595": 22, "fix": [22, 23], "increas": 22, "undo": 22, "onc": [22, 26, 30, 34, 54, 56, 59], "done": [22, 27, 30, 31, 36, 50, 56, 59], "over": [22, 26, 31, 38, 46, 48, 49, 59], "deprec": 22, "move": [22, 23, 54], "three": [23, 29, 31, 39], "level": [23, 31, 46], "alter": 23, "hack": [23, 30], "lack": 23, "world": [23, 31, 39], "domin": 23, "among": [23, 39], "intent": 23, "wide": 23, "mass": 23, "adopt": 23, "corner": 23, "deserv": 23, "chapter": [23, 31, 60], "uncommon": 23, "unfortun": 23, "born": 23, "extend": [23, 28, 29, 31], "easili": 23, "maxim": 23, "reduc": 23, "preserv": [23, 31], "aspect": [23, 26], "plenti": 23, "alreadi": [23, 26, 27, 36, 56], "think": [23, 27, 30, 31], "weird": 23, "interfer": 23, "vendor": 23, "misbehav": 23, "feedback": [23, 31], "reconsid": 23, "disrespect": 23, "fanci": 23, "simpli": [23, 26, 27, 30, 32, 46, 60], "happi": [23, 30], "split": 23, "convent": 23, "practic": 23, "gitmoji": 23, "yet": [23, 24], "patch": [23, 31, 49], "pep8": [23, 27], "length": [23, 31], "cardin": 23, "rule": [23, 49, 59], "ensur": 23, "logic": 23, "break": [23, 31], "author": [23, 24, 31], "rst": [23, 27, 31], "meaning": [23, 30, 31], "scope": 23, "footer": 23, "quickstart": [23, 25, 38], "weblat": [23, 27, 36], "sphinx": 23, "much": [23, 31], "easier": 23, "makefil": [23, 25, 26, 38, 59], "dist": [23, 27], "assert": 23, "wysiwyg": 23, "target": [23, 27, 30, 31, 59], "favorit": [23, 27], "8000": 23, "watch": 23, "autobuild": [23, 25], "sphinxopt": 23, "50593": 23, "push": [23, 27, 36], "adapt": 24, "normal": [24, 31, 35, 56], "matter": [24, 26, 39], "howev": [24, 32], "boolean": [24, 29, 31], "str": [24, 34, 48, 49, 54, 55, 56], "ref": [24, 26, 29, 35, 59], "bool": [24, 29, 45, 56], "namespac": [24, 34, 42], "often": [24, 31, 58], "overwritten": 24, "assign": 24, "redefin": 24, "lead": 24, "underlin": [24, 27, 42], "veri": 24, "_non_overwritten_glob": 24, "foo": [24, 53, 54], "number_of_result": 24, "int": [24, 31, 54, 55, 56], "These": [24, 31], "construct": [24, 31], "random": [24, 39, 56, 57], "pagenumb": 24, "unspecifi": 24, "from_lang": 24, "to_lang": 24, "amount": [24, 54], "float": [24, 55, 56], "4217": 24, "from_nam": 24, "to_nam": 24, "function": [24, 28, 29, 31, 34, 38, 40, 41, 42, 44, 45, 49, 54, 59], "def": [24, 29, 31, 48, 56], "allow_redirect": 24, "hard": [24, 31], "raise_for_httperror": 24, "rais": [24, 34, 56], "300": 24, "desir": 24, "publishedd": 24, "datetim": [24, 31], "publish": 24, "partli": 24, "thumbnail_src": 24, "preview": 24, "thumbnail": [24, 45], "seed": 24, "seeder": 24, "leech": 24, "leecher": 24, "files": [24, 46, 56], "byte": [24, 46, 56], "magnetlink": 24, "torrentfil": 24, "latitud": 24, "decim": 24, "longitud": 24, "boundingbox": 24, "arrai": 24, "lat": 24, "lon": 24, "geojson": 24, "object": [24, 29, 31, 34, 46, 54, 56], "road": 24, "street": 24, "house_numb": 24, "hous": [24, 35], "citi": 24, "postcod": 24, "abstract": 24, "ital": [24, 31], "short": 24, "medium": 24, "book": 24, "editor": 24, "journal": 24, "magazin": 24, "report": [24, 26], "1038": 24, "d41586": 24, "018": 24, "07848": 24, "issn": 24, "1476": 24, "4687": 24, "isbn": 24, "9780201896831": 24, "pdf_url": 24, "html_url": 24, "prime": 25, "hackabl": 25, "translat": [25, 27, 38, 49, 52], "acknowledg": 25, "wlc": 25, "motiv": 25, "gentlemen": 25, "wrap": 25, "suit": [25, 58], "nvm": 25, "nodej": 25, "pylint": 25, "checker": 25, "primer": [25, 38], "skill": 25, "inlin": 25, "markup": [25, 48], "anchor": 25, "liter": 25, "unicod": [25, 35], "substitut": 25, "role": 25, "figur": 25, "admonit": 25, "view": [25, 39], "searxng_extra": [25, 38], "standalone_searx": [25, 33], "lxc": [26, 38, 58], "heterogen": 26, "cycl": 26, "tl": [26, 50], "dr": 26, "experienc": 26, "reader": [26, 31], "seriou": 26, "perfect": 26, "overlook": 26, "encapsul": 26, "lot": [26, 54], "prerequisit": [26, 31], "preinstal": 26, "softwar": 26, "isol": 26, "mix": 26, "divid": 26, "stack": [26, 59], "lxd": [26, 59], "snap": [26, 59], "consist": 26, "lxc_suit": [26, 59], "l19": 26, "exercis": 26, "let": [26, 31, 39], "outsid": [26, 27], "blocker": 26, "got": 26, "174": 26, "184": 26, "156": 26, "sanit": 26, "earxng": 26, "prompt": [26, 27, 58], "analog": 26, "notic": 26, "readi": 26, "ey": [26, 31], "dsitro": 26, "least": 26, "attend": 26, "rel": [26, 31, 56, 59], "transpar": [26, 59], "smylink": 26, "reposetori": 26, "becom": [26, 31, 35], "mv": 26, "daili": 26, "usag": [26, 27, 31, 34, 42, 48, 59, 60], "eth0": [26, 59], "4004": 26, "live": [26, 27, 30, 31, 54, 59], "fd42": 26, "573b": 26, "e0b3": 26, "e97": 26, "216": 26, "3eff": 26, "fea5": 26, "9b65": 26, "deeper": [27, 31], "relev": 27, "ci": [27, 33, 35], "wrapper": 27, "gnu": 27, "introduct": 27, "runner": 27, "lt": [27, 50], "counterpart": [27, 36], "engines_languag": [27, 35, 45], "userag": [27, 35, 56], "recent": 27, "prebuild": 27, "gecko": 27, "driver": 27, "geckodriv": 27, "robot_test": 27, "amd64": 27, "intermedi": 27, "upload": [27, 46], "black": [27, 31], "yamllint": 27, "yamllint_fil": 27, "pylint_fil": 27, "pyright": 27, "coverag": 27, "incl": 27, "stuff": [27, 31], "pygment": [27, 31, 35], "golang": 27, "previous": 27, "restor": [27, 30], "searxng_redis_url": 27, "py3": 27, "txt": [27, 31, 35], "argpars": 27, "initialis": 27, "sha256": 27, "sum": 27, "word": [27, 29, 31, 32], "6cea6eb6def9e14a18bf32f8a3": 27, "471efef6c73558e391c3adb35f4": 27, "goe": 27, "wrong": 27, "central": 27, "long": [27, 39], "especi": [27, 31], "pre": 27, "aka": [27, 48, 49], "public_url": 27, "higher": 27, "met": 27, "chain": [27, 57, 59], "04": [27, 57, 59], "v10": 27, "19": 27, "v16": 27, "jinja2": 27, "instant": 27, "live_them": [27, 30], "untouch": 27, "adjust": 27, "seri": 27, "pylintrc": 27, "whitespac": 27, "3xx": 27, "a1": 27, "443": 27, "life": 27, "hl": 27, "lang_en": 27, "ie": [27, 50], "utf8": 27, "oe": 27, "ceid": 27, "3aen": 27, "302": 27, "comput": [27, 39], "introduc": [28, 31], "skeleton": 28, "demo_offlin": [28, 43], "engine_set": [28, 43, 44], "omit": 28, "anyth": [28, 39], "retriev": 28, "represent": [28, 56], "publicli": 28, "default_on": 29, "js_depend": 29, "tupl": [29, 56], "css_depend": 29, "attach": 29, "callback": 29, "hook": 29, "flask": [29, 31, 48, 55], "ctx": 29, "whole": 29, "context": [29, 31, 48, 59], "post_search": 29, "result_contain": [29, 55], "green": 29, "return42": [29, 31], "tgwf": 29, "feel": [29, 31], "pre_search": 29, "continu": [29, 31], "searchwithplugin": [29, 40, 55], "on_result": 29, "parsed_url": 29, "urlpars": 29, "love": 30, "workflow": [30, 36], "receiv": 30, "end": [30, 31, 59], "wild": 30, "west": 30, "pai": [30, 37], "attent": [30, 31], "compil": [30, 56], "finish": [30, 59], "remain": 30, "rewind": 30, "encourag": 31, "contributor": 31, "principl": 31, "restructuredtext": 31, "builder": 31, "docutil": 31, "faq": 31, "doctre": 31, "cross": 31, "linuxdoc": 31, "jinja": [31, 48], "autodoc": 31, "ecosystem": 31, "therefor": 31, "spars": 31, "plaintext": 31, "intuit": 31, "learn": 31, "produc": 31, "advantag": 31, "disadvantag": 31, "bit": [31, 59], "grumpi": 31, "face": 31, "train": 31, "bring": [31, 49], "audienc": 31, "question": [31, 39], "knowledg": 31, "subject": 31, "concret": 31, "pov": 31, "heard": 31, "meta": 31, "crawler": 31, "he": [31, 47, 49, 50], "pro": 31, "con": 31, "understand": 31, "chronolog": 31, "condit": [31, 50], "asterisk": 31, "backquot": 31, "appear": 31, "confus": 31, "escap": [31, 35], "backslash": 31, "pointer": 31, "emphasi": 31, "strong": 31, "boldfac": 31, "sampl": 31, "adorn": 31, "subsect": 31, "_doc": 31, "refnam": 31, "lorem": [31, 57], "ipsum": [31, 57], "dolor": 31, "sit": 31, "amet": 31, "consectetur": 31, "adipisici": 31, "elit": 31, "_chapter": 31, "ut": 31, "enim": 31, "veniam": 31, "qui": 31, "nostrud": 31, "exercit": 31, "ullamco": 31, "labori": 31, "nisi": 31, "aliquid": 31, "ex": 31, "ea": 31, "commodi": 31, "consequat": 31, "_section": 31, "_subsect": 31, "overlin": 31, "_anchor": 31, "_rest": 31, "visist": 31, "_sphinx": 31, "_": [31, 42], "raw": [31, 48], "__": 31, "referenc": 31, "rfc": 31, "822": 31, "pep": 31, "af2cae6": 31, "oper": [31, 39], "obj": [31, 34, 56], "intersphinx_map": 31, "palletsproject": 31, "inventori": 31, "inv": 31, "simplest": 31, "indent": [31, 34, 59], "colon": 31, "literalinclud": 31, "latter": 31, "expand": 31, "consetetur": 31, "sadipsc": 31, "elitr": 31, "diam": 31, "nonumi": 31, "eirmod": 31, "tempor": 31, "invidunt": 31, "labor": 31, "variant": 31, "caption": 31, "rout": [31, 48], "statist": 31, "get_engines_stat": 31, "within": [31, 43, 44], "0xa9": 31, "copyright": 31, "sign": 31, "tm": 31, "2122": 31, "trademark": 31, "glyph": 31, "piec": 31, "explicit": 31, "signifi": 31, "enclos": 31, "rolenam": 31, "guilabel": 31, "ancel": 31, "cancel": 31, "kbd": 31, "menuselect": 31, "b": [31, 35, 50, 53, 56], "bold": 31, "subscript": 31, "o": [31, 50, 60], "sub": 31, "superscript": 31, "sup": 31, "scalabl": 31, "sens": 31, "absenc": 31, "annoi": 31, "processor": [31, 46], "inherit": [31, 55], "insert": [31, 59], "_svg": 31, "svg_imag": 31, "alt": 31, "_dot": 31, "digraph": 31, "baz": 31, "vector": 31, "nw": 31, "arrow": 31, "xmln": 31, "w3": 31, "baseprofil": 31, "70px": 31, "height": [31, 46], "40px": 31, "viewbox": 31, "700": 31, "x1": 31, "180": 31, "y1": 31, "370": 31, "x2": 31, "500": 31, "y2": 31, "50": [31, 46], "stroke": 31, "15px": 31, "polygon": 31, "585": 31, "525": 31, "25": 31, "transform": 31, "rotat": 31, "135": 31, "parent": 31, "compact": 31, "third": [31, 39, 54], "xxxx": 31, "yyyi": 31, "zzzz": 31, "distinguish": 31, "classifi": 31, "phrase": 31, "typo": 31, "That": 31, "why": [31, 38], "duref": 31, "surround": 31, "broken": 31, "fieldnam": 31, "commonli": 31, "my_funct": 31, "my_arg": 31, "my_other_arg": 31, "cours": 31, "caveat": 31, "doctest": 31, "catcher": 31, "top": 31, "kiss_": 31, "readability_": 31, "tip": 31, "caution": 31, "danger": 31, "import": [31, 34, 48, 53], "ugli": 31, "Not": [31, 45], "art": [31, 44], "comfort": 31, "huge": 31, "row": 31, "column": 31, "cell": 31, "nightmar": 31, "big": [31, 54], "diff": 31, "widen": 31, "ascrib": 31, "anywai": 31, "helper": 31, "emac": 31, "onlin": [31, 38, 40], "colspan": 31, "rowspan": 31, "metadata": 31, "front": 31, "align": 31, "span": [31, 56], "doubl": 31, "stage": 31, "cspan": 31, "rspan": 31, "rightmost": 31, "fill": 31, "behavior": [31, 45], "count": 31, "stub": 31, "morecol": 31, "morerow": 31, "col": 31, "outstand": 31, "csv_tabl": 31, "loremlorem": 31, "et": [31, 47, 50], "magna": 31, "aliquyam": 31, "erat": 31, "voluptua": 31, "vero": 31, "accusam": 31, "justo": 31, "duo": 31, "rebum": 31, "stet": 31, "clita": 31, "kasd": 31, "gubergren": 31, "sea": 31, "takimata": 31, "sanctu": 31, "est": 31, "suitabl": 31, "enabled_engine_count": 31, "categories_as_tab": 31, "group_engines_in_tab": 31, "loop": [31, 54], "endif": 31, "mod": 31, "__name__": 31, "documented_modul": 31, "els": [31, 36, 59], "upper": 31, "engine_typ": [31, 40, 46], "language_support": 31, "endfor": 31, "jinja_context": 31, "instruct": 31, "amsmath": 31, "mathemat": 31, "ctan": 31, "numref": 31, "schroeding": 31, "schr\u00f6dinger": 31, "label": [31, 35], "mathrm": 31, "hbar": 31, "dfrac": 31, "psi": 31, "rangl": 31, "hat": 31, "tfrac": 31, "textstyl": 31, "displaystyl": 31, "fraction": 31, "z": 31, "endpoint": [32, 45], "thu": [32, 39], "hash_plugin": 32, "search_on_category_select": 32, "self_inform": 32, "tracker_url_remov": 32, "ahmia_blacklist": [32, 35], "open_access_doi_rewrit": 32, "like_hotkei": 32, "tor_check_plugin": 32, "disabled_plugin": 32, "enabled_engin": 32, "disabled_engin": 32, "update_ahmia_blacklist": 33, "update_curr": 33, "update_engine_descript": 33, "update_external_bang": 33, "update_firefox_vers": 33, "update_languag": 33, "update_osm_keys_tag": 33, "update_pyg": 33, "update_wikidata_unit": 33, "get_search_queri": [33, 34], "json_seri": [33, 34], "no_parsed_url": [33, 34], "parse_argu": [33, 34], "to_dict": [33, 34], "rain": 34, "contrari": 34, "behav": 34, "track": [34, 38, 39, 47], "importlib": 34, "sy": 34, "search_queri": [34, 55], "engine_c": 34, "spec": 34, "spec_from_file_loc": 34, "sa": [34, 50], "module_from_spec": 34, "loader": [34, 42], "exec_modul": 34, "prog_arg": 34, "category_choic": 34, "search_q": 34, "engine_categori": 34, "res_dict": 34, "stdout": 34, "dump": 34, "sort_kei": 34, "ensure_ascii": 34, "infobox": [34, 35], "results_numb": 34, "820000000": 34, "timerang": 34, "arg": [34, 56], "searchqueri": [34, 40, 55], "serial": 34, "serializ": 34, "typeerror": [34, 56], "pars": [34, 44, 45, 46, 47, 52], "systemexit": 34, "ptipython": 34, "onion": 35, "fetch": [35, 44, 52, 60], "engine_descript": 35, "get_output": 35, "description_and_sourc": 35, "external_bang": [35, 55], "newbang": 35, "bv1": 35, "v260": 35, "futur": 35, "bv2": 35, "probabl": 35, "re_bang_vers": 35, "merge_when_no_leaf": 35, "child": 35, "equal": 35, "leaf_kei": 35, "dig": 35, "nood": 35, "dg": 35, "ig": 35, "signatur": 35, "intersect": 35, "unicodeescap": 35, "pprint": 35, "pformat": 35, "get_unicode_flag": 35, "lang_cod": 35, "determin": [35, 49], "emoji": 35, "i18n": [35, 36, 38, 40], "sidenot": 35, "atownsend": 35, "uk": [35, 50, 56], "osm_keys_tag": 35, "sparql_tags_request": 35, "sparql": 35, "get_tag": 35, "taginfo": 35, "3dhous": 35, "q3947": 35, "properti": [35, 45, 48, 56], "p1282": 35, "3abuild": 35, "3dbungalow": 35, "q850107": 35, "sparql_keys_request": 35, "payment": [35, 38], "3apay": 35, "q1148747": 35, "confirm": 35, "cash": [35, 37], "rdf": 35, "oppos": 35, "wikibas": 35, "wikidata_unit": 35, "extractor": [36, 38, 40], "pybabel": 36, "sync": 36, "synchron": 36, "orphan": 36, "decoupl": 36, "pot": 36, "po": 36, "job": [36, 58], "fridai": 36, "additon": 36, "mo": 36, "team": 37, "cost": 37, "vp": 37, "protonmail": 37, "ask": 37, "audit": 37, "experi": 37, "ux": 37, "credit": 37, "debit": 37, "card": 37, "bank": 37, "transfer": 37, "liberapai": 37, "recurr": 37, "bui": 37, "coffe": [37, 59], "cryptocurr": 37, "bitcoin": 37, "bc1qn3rw8t86h05cs3grx2kmwmptw9k4kt4hyzktqj": 37, "segwit": 37, "qpead2yu482e3h9amy5zk45l8qrfhk59jcpw3cth9": 37, "ethereum": 37, "0xcf82c7eb915ee70b5b69c1bbb5525e157f35fa43": 37, "dogecoin": 37, "dbcys9isstt84pddxssthpqxyqdtfp1te4": 37, "litecoin": 37, "ltc1q5j6x6f4f2htldhq570e353clc8fmw44ra5er5q": 37, "aggreg": [38, 39], "70": 38, "neither": [38, 56], "nor": [38, 56], "addition": [38, 39], "anyon": 38, "No": 38, "encrypt": 38, "middl": 38, "2021": 38, "conclus": [38, 50], "everyon": 39, "unknown": 39, "parti": 39, "peopl": 39, "vpn": 39, "laptop": 39, "gain": 39, "insight": 39, "dive": 39, "regardless": 39, "advertis": 39, "unlik": 39, "monet": 39, "besid": 39, "someon": 39, "whether": [39, 46, 50], "sent": 39, "sold": 39, "proper": 39, "vulner": 39, "abus": 39, "exchang": 39, "tailor": 39, "reset": [39, 59, 60], "clear": 39, "compromis": 39, "engine_shortcut": [40, 42], "is_missing_required_attribut": [40, 42], "load_engin": [40, 42], "download_error": [40, 46], "format_not_support": [40, 46], "no_signature_error": [40, 46], "parse_tineye_match": [40, 46], "lang2domain": [40, 47], "parse_url": [40, 47], "supported_languages_url": [40, 47], "infopag": [40, 48], "infopageset": [40, 48], "additional_transl": [40, 49], "locale_best_match": [40, 49], "locale_nam": [40, 49], "rtl_local": [40, 49], "get_engine_local": [40, 49, 50], "get_locale_descr": [40, 49], "get_transl": [40, 49], "locales_initi": [40, 49], "supported_lang": [40, 50], "preference_sect": [40, 52], "query_exampl": [40, 52], "query_keyword": [40, 52], "old_redis_url_default_url": [40, 53], "lua_script_storag": [40, 54], "drop_count": [40, 54], "incr_count": [40, 54], "incr_sliding_window": [40, 54], "purge_by_prefix": [40, 54], "secret_hash": [40, 54], "engineref": [40, 55], "convert_str_to_int": [40, 56], "detect_languag": [40, 56], "dict_subset": [40, 56], "ecma_unescap": [40, 56], "eval_xpath": [40, 56], "eval_xpath_getindex": [40, 56], "eval_xpath_list": [40, 56], "extract_text": [40, 56], "extract_url": [40, 56], "gen_userag": [40, 56], "get_engine_from_set": [40, 56], "get_torrent_s": [40, 56], "get_xpath": [40, 56], "html_to_text": [40, 56], "int_or_zero": [40, 56], "is_valid_lang": [40, 56], "match_languag": [40, 45, 56], "normalize_url": [40, 56], "to_str": [40, 56], "searxng_msg": 41, "msg": 41, "cfg": 41, "babel_extract": 41, "yield": 41, "fileobj": 41, "comment_tag": 41, "regist": 42, "engine_data": [42, 55], "engine_default_arg": 42, "underscor": [42, 49], "lowercas": 42, "engine_list": 42, "declar": 42, "request_param": 43, "assembl": 43, "institut": 44, "chicago": 44, "demo_onlin": 44, "edu": 44, "appi": 45, "use_mobile_ui": 45, "mobil": 45, "bypass": 45, "159": 45, "simul": 45, "async": 45, "use_ac": 45, "_fmt": 45, "pc": 45, "get_lang_info": 45, "lang_list": [45, 56], "custom_alias": [45, 56], "supported_any_languag": 45, "compos": 45, "alias": 45, "non": 45, "decid": 45, "deliv": 45, "dictionari": [45, 54, 56], "pair": 45, "AT": 45, "subdomain": 45, "google_domain": 45, "urllib": 45, "urlencod": 45, "intern": [45, 56], "android": 45, "protobuf": 45, "pb": 45, "compress": [45, 50], "jspb": 45, "img": 45, "scrap_out_thumb": 45, "dom": 45, "num": [45, 56], "drag": 46, "constantli": 46, "crawl": 46, "billion": 46, "offici": 46, "due": [46, 54], "unsupport": 46, "jpeg": 46, "png": 46, "gif": 46, "bmp": 46, "tiff": 46, "webp": 46, "visual": 46, "identifi": [46, 50, 59], "match_json": 46, "image_url": 46, "score": [46, 54], "pixel": 46, "area": 46, "overlai": 46, "belong": 46, "stock": 46, "backlink": 46, "crawl_dat": 46, "bg": [47, 50], "el": [47, 50], "hr": [47, 50], "sk": [47, 50], "sl": [47, 50], "zh_ch": 47, "hk": [47, 49], "zh_cht": 47, "url_str": 47, "_info_pag": 48, "mistletoepag": 48, "pagenam": 48, "get_valu": 48, "get_pag": 48, "fname": 48, "conntext": 48, "get_ctx": 48, "markdown": 48, "commonmark": 48, "raw_cont": 48, "page_class": 48, "info_fold": 48, "parser": 48, "markdwon": 48, "toc": 48, "zh_hans_cn": 48, "i18n_origin": 48, "iter_pag": 48, "fallback_to_default": 48, "iter": 48, "locale_default": 48, "dv": [49, 50], "\u078b": 49, "\u0788": 49, "\u0780": 49, "dhivehi": 49, "oc": [49, 50], "occitan": 49, "pap": 49, "papiamento": 49, "szl": 49, "\u015bl\u014dnski": 49, "silesian": 49, "si": [49, 50], "nl": [49, 50], "br": [49, 50], "pl": [49, 50], "zh": [49, 50], "hant": 49, "taiwan": 49, "hong": 49, "kong": 49, "fa": [49, 50], "ir": 49, "searxng_local": 49, "engine_local": 49, "ca_e": 49, "fr_be": 49, "fr_ca": 49, "ch": 49, "fr_ch": 49, "pl_pl": 49, "pt_pt": 49, "tri": 49, "narrow": 49, "down": 49, "approxim": 49, "attempt": 49, "assumpt": 49, "optim": 49, "territori": 49, "prioriti": 49, "terrirtori": 49, "offic": 49, "fran\u00e7ai": 49, "portugu\u00ea": 49, "brasil": 49, "pt_br": 49, "monkei": 49, "flask_babel": 49, "fasttext": [50, 56], "identif": [50, 56], "model": 50, "zip": 50, "classif": 50, "bag": 50, "trick": 50, "effici": 50, "af": 50, "am": 50, "arz": 50, "ast": 50, "av": 50, "az": 50, "azb": 50, "ba": 50, "bcl": 50, "bh": 50, "bn": 50, "bo": 50, "bpy": 50, "bxr": 50, "cbk": 50, "ce": 50, "ceb": 50, "ckb": 50, "co": 50, "cv": 50, "cy": 50, "diq": 50, "dsb": 50, "dty": 50, "eml": 50, "eu": 50, "fi": [50, 59], "frr": 50, "fy": 50, "ga": 50, "gn": 50, "gom": 50, "gu": 50, "gv": 50, "hi": 50, "hif": 50, "hsb": 50, "ht": 50, "hu": 50, "hy": 50, "ia": 50, "ilo": 50, "jbo": 50, "jv": 50, "ka": 50, "kk": 50, "km": 50, "kn": 50, "krc": 50, "ku": 50, "kv": 50, "kw": 50, "ky": 50, "lb": 50, "lez": 50, "li": 50, "lmo": 50, "lrc": 50, "mg": 50, "mhr": 50, "mk": 50, "ml": 50, "mn": 50, "mr": 50, "mrj": 50, "mt": 50, "mwl": 50, "myv": 50, "mzn": 50, "nah": 50, "nap": 50, "nd": 50, "ne": 50, "nn": 50, "pa": 50, "pam": 50, "pfl": 50, "pm": 50, "pnb": 50, "qu": 50, "ro": 50, "rue": 50, "sah": 50, "scn": 50, "sco": 50, "sd": 50, "sq": 50, "sr": 50, "sv": 50, "sw": 50, "ta": 50, "te": 50, "tg": 50, "th": 50, "tk": 50, "tr": 50, "tyv": 50, "ug": 50, "ur": 50, "uz": 50, "vec": 50, "vep": 50, "vi": 50, "vl": 50, "vo": 50, "war": 50, "wuu": 50, "xal": 50, "xmf": 50, "yi": 50, "yo": 50, "yue": 50, "harmon": 50, "discrep": 50, "cn": 50, "zh_hant": 50, "zh_han": 50, "menu": 50, "highest": 50, "win": 50, "conflict": [50, 59], "explicitli": 50, "via": 50, "thermomix": 50, "circumst": 50, "Its": 50, "decis": 50, "autodetect_search_languag": 50, "fil": 50, "language_cod": 50, "tor_check": 52, "conveni": 54, "lua": 54, "inspir": 54, "bullet": 54, "proof": 54, "redispi": 54, "redislib": 54, "counter": 54, "searxng_counter_": 54, "increment": 54, "64": 54, "incr": 54, "sec": [54, 56], "sleep": 54, "slide": 54, "window": 54, "typedur": 54, "zadd": 54, "minu": 54, "zremrangebyscor": 54, "refresh": 54, "zcount": 54, "until": 54, "instanti": 54, "register_script": 54, "searxng_": 54, "purg": 54, "zero": 54, "schema": 54, "del": 54, "anonymis": 54, "engineref_list": 55, "timeout_limit": 55, "resultcontain": 55, "ordered_plugin_list": 55, "number_str": 56, "threshold": 56, "min_prob": 56, "mutablemap": 56, "unescap": 56, "ecma": 56, "262": 56, "mozilla": 56, "objets_globaux": 56, "u5409": 56, "\u5409": 56, "f3": 56, "\u00f3": 56, "elementbas": 56, "xpath_spec": 56, "union": 56, "equival": 56, "xpath_str": 56, "xpathxslt": 56, "etre": 56, "searxxpathsyntaxexcept": 56, "searxenginexpathexcept": 56, "_notsetclass": 56, "ais": 56, "min_len": 56, "xpath_result": 56, "allow_non": 56, "concat": 56, "text_cont": 56, "htmlelement": 56, "fromstr": 56, "42": [56, 59], "parsererror": 56, "valueerror": 56, "os_str": 56, "filesize_multipli": 56, "tb": 56, "tib": 56, "gib": 56, "5368709120": 56, "mib": 56, "3140000": 56, "worst": 56, "html_str": 56, "color": 56, "red": 56, "zz": 56, "ukrainian": 56, "espa\u00f1ol": 56, "spanish": 56, "locale_cod": 56, "fallback": 56, "join": 56, "absolut": 56, "pari": 57, "wau": 57, "holland": 57, "abbrevi": 57, "inclus": 57, "wfr": 57, "uuid": 57, "averag": 57, "avg": 57, "123": 57, "548": 57, "md5": 57, "sha512": 57, "dispos": 58, "force_timeout": [58, 59], "batch": 58, "snapcraft": 59, "cup": 59, "ever": 59, "iptabl": 59, "fralef": 59, "whenev": 59, "reboot": 59, "7048": 59, "7851230": 59, "handi": 59, "ugo": 59, "ubu2110": 59, "147": 59, "ubu2004": 59, "246": 59, "searxnggfedora35": 59, "140": 59, "165": 59, "200331": 59, "15": 59, "296": 59, "explanatori": [59, 60], "launch": 59, "remot": 59, "storag": 59, "quot": 59, "prepar": 59, "fedora35": 59, "nil": 59, "spdx": 59, "agpl": 59, "manipul": 59, "subshel": 59, "lxc_set_suite_env": 59, "lxc_suite_nam": 59, "linuxcontain": 59, "linuxcontainers_org_nam": 59, "lxc_host_prefix": 59, "april": 59, "2025": 59, "21": 59, "juli": 59, "2027": 59, "eol": 59, "fedoraproject": 59, "35": 59, "releng": 59, "lxc_suite_install_info": 59, "eof": 59, "local_imag": 59, "lxc_suite_instal": 59, "lxc_repo_root": 59, "rst_titl": 59, "lxc_suite_info": 59, "global_ip": 59, "info_msg": 59, "sc2034": 59, "sc2031": 59, "localtest": 60, "searxng_check": 60, "get_set": 60, "searxng_uwsgi_socket": 60, "git_url": 60, "git_branch": 60, "fv": 60, "az105": 60, "957": 60}, "objects": {"": [[29, 0, 1, "", "on_result"], [29, 0, 1, "", "post_search"], [29, 0, 1, "", "pre_search"]], "searx": [[41, 1, 0, "-", "babel_extract"], [42, 1, 0, "-", "engines"], [48, 1, 0, "-", "infopage"], [49, 1, 0, "-", "locales"], [53, 1, 0, "-", "redisdb"], [54, 1, 0, "-", "redislib"], [56, 1, 0, "-", "utils"]], "searx.babel_extract": [[41, 0, 1, "", "extract"]], "searx.engines": [[42, 2, 1, "", "Engine"], [43, 1, 0, "-", "demo_offline"], [44, 1, 0, "-", "demo_online"], [42, 3, 1, "", "engine_shortcuts"], [45, 1, 0, "-", "google"], [45, 1, 0, "-", "google_images"], [45, 1, 0, "-", "google_news"], [45, 1, 0, "-", "google_videos"], [42, 0, 1, "", "is_missing_required_attributes"], [42, 0, 1, "", "load_engine"], [42, 0, 1, "", "load_engines"], [46, 1, 0, "-", "tineye"], [42, 0, 1, "", "using_tor_proxy"], [10, 1, 0, "-", "xpath"], [47, 1, 0, "-", "yahoo"]], "searx.engines.demo_offline": [[43, 0, 1, "", "init"], [43, 0, 1, "", "search"]], "searx.engines.demo_online": [[44, 0, 1, "", "init"], [44, 0, 1, "", "request"], [44, 0, 1, "", "response"]], "searx.engines.google": [[45, 0, 1, "", "get_lang_info"], [45, 0, 1, "", "request"], [45, 0, 1, "", "response"]], "searx.engines.google_images": [[45, 0, 1, "", "request"], [45, 0, 1, "", "response"]], "searx.engines.google_news": [[45, 0, 1, "", "request"], [45, 0, 1, "", "response"]], "searx.engines.google_videos": [[45, 0, 1, "", "request"], [45, 0, 1, "", "response"], [45, 0, 1, "", "scrap_out_thumbs"]], "searx.engines.tineye": [[46, 3, 1, "", "DOWNLOAD_ERROR"], [46, 3, 1, "", "FORMAT_NOT_SUPPORTED"], [46, 3, 1, "", "NO_SIGNATURE_ERROR"], [46, 3, 1, "", "engine_type"], [46, 0, 1, "", "parse_tineye_match"], [46, 0, 1, "", "request"], [46, 0, 1, "", "response"]], "searx.engines.xpath": [[10, 3, 1, "", "content_xpath"], [10, 3, 1, "", "first_page_num"], [10, 3, 1, "", "headers"], [10, 3, 1, "", "lang_all"], [10, 3, 1, "", "no_result_for_http_status"], [10, 3, 1, "", "page_size"], [10, 3, 1, "", "paging"], [10, 0, 1, "", "request"], [10, 0, 1, "", "response"], [10, 3, 1, "", "results_xpath"], [10, 3, 1, "", "safe_search_map"], [10, 3, 1, "", "safe_search_support"], [10, 3, 1, "", "search_url"], [10, 3, 1, "", "soft_max_redirects"], [10, 3, 1, "", "suggestion_xpath"], [10, 3, 1, "", "thumbnail_xpath"], [10, 3, 1, "", "time_range_map"], [10, 3, 1, "", "time_range_support"], [10, 3, 1, "", "time_range_url"], [10, 3, 1, "", "title_xpath"], [10, 3, 1, "", "url_xpath"]], "searx.engines.yahoo": [[47, 3, 1, "", "lang2domain"], [47, 0, 1, "", "parse_url"], [47, 0, 1, "", "request"], [47, 0, 1, "", "response"], [47, 3, 1, "", "supported_languages_url"]], "searx.infopage": [[48, 2, 1, "", "InfoPage"], [48, 2, 1, "", "InfoPageSet"]], "searx.infopage.InfoPage": [[48, 4, 1, "", "content"], [48, 5, 1, "", "get_ctx"], [48, 4, 1, "", "html"], [48, 4, 1, "", "raw_content"], [48, 4, 1, "", "title"]], "searx.infopage.InfoPageSet": [[48, 6, 1, "", "folder"], [48, 5, 1, "", "get_page"], [48, 5, 1, "", "iter_pages"], [48, 6, 1, "", "locale_default"], [48, 6, 1, "", "locales"], [48, 6, 1, "", "toc"]], "searx.locales": [[49, 3, 1, "", "ADDITIONAL_TRANSLATIONS"], [49, 3, 1, "", "LOCALE_BEST_MATCH"], [49, 3, 1, "", "LOCALE_NAMES"], [49, 3, 1, "", "RTL_LOCALES"], [49, 0, 1, "", "get_engine_locale"], [49, 0, 1, "", "get_locale_descr"], [49, 0, 1, "", "get_translations"], [49, 0, 1, "", "locales_initialize"]], "searx.plugins": [[50, 1, 0, "-", "autodetect_search_language"], [51, 1, 0, "-", "limiter"], [52, 1, 0, "-", "tor_check"]], "searx.plugins.autodetect_search_language": [[50, 3, 1, "", "supported_langs"]], "searx.plugins.tor_check": [[52, 3, 1, "", "description"], [52, 3, 1, "", "name"], [52, 3, 1, "", "preference_section"], [52, 3, 1, "", "query_examples"], [52, 3, 1, "", "query_keywords"]], "searx.redisdb": [[53, 3, 1, "", "OLD_REDIS_URL_DEFAULT_URL"]], "searx.redislib": [[54, 3, 1, "", "LUA_SCRIPT_STORAGE"], [54, 0, 1, "", "drop_counter"], [54, 0, 1, "", "incr_counter"], [54, 0, 1, "", "incr_sliding_window"], [54, 0, 1, "", "lua_script_storage"], [54, 0, 1, "", "purge_by_prefix"], [54, 0, 1, "", "secret_hash"]], "searx.search": [[55, 2, 1, "", "EngineRef"], [55, 2, 1, "", "Search"], [55, 2, 1, "", "SearchQuery"], [55, 2, 1, "", "SearchWithPlugins"]], "searx.search.Search": [[55, 6, 1, "", "result_container"], [55, 5, 1, "", "search"], [55, 6, 1, "", "search_query"]], "searx.search.SearchWithPlugins": [[55, 6, 1, "", "ordered_plugin_list"], [55, 6, 1, "", "request"], [55, 6, 1, "", "result_container"], [55, 5, 1, "", "search"], [55, 6, 1, "", "search_query"]], "searx.utils": [[56, 0, 1, "", "convert_str_to_int"], [56, 0, 1, "", "detect_language"], [56, 0, 1, "", "dict_subset"], [56, 0, 1, "", "ecma_unescape"], [56, 0, 1, "", "eval_xpath"], [56, 0, 1, "", "eval_xpath_getindex"], [56, 0, 1, "", "eval_xpath_list"], [56, 0, 1, "", "extract_text"], [56, 0, 1, "", "extract_url"], [56, 0, 1, "", "gen_useragent"], [56, 0, 1, "", "get_engine_from_settings"], [56, 0, 1, "", "get_torrent_size"], [56, 0, 1, "", "get_xpath"], [56, 0, 1, "", "html_to_text"], [56, 0, 1, "", "int_or_zero"], [56, 0, 1, "", "is_valid_lang"], [56, 0, 1, "", "match_language"], [56, 0, 1, "", "normalize_url"], [56, 0, 1, "", "searx_useragent"], [56, 0, 1, "", "to_string"]], "searxng_extra": [[34, 1, 0, "-", "standalone_searx"]], "searxng_extra.standalone_searx": [[34, 0, 1, "", "get_search_query"], [34, 0, 1, "", "json_serial"], [34, 0, 1, "", "no_parsed_url"], [34, 0, 1, "", "parse_argument"], [34, 0, 1, "", "to_dict"]], "searxng_extra.update": [[35, 1, 0, "-", "update_ahmia_blacklist"], [35, 1, 0, "-", "update_currencies"], [35, 1, 0, "-", "update_engine_descriptions"], [35, 1, 0, "-", "update_external_bangs"], [35, 1, 0, "-", "update_firefox_version"], [35, 1, 0, "-", "update_languages"], [35, 1, 0, "-", "update_osm_keys_tags"], [35, 1, 0, "-", "update_pygments"], [35, 1, 0, "-", "update_wikidata_units"]], "searxng_extra.update.update_engine_descriptions": [[35, 0, 1, "", "get_output"]], "searxng_extra.update.update_external_bangs": [[35, 0, 1, "", "merge_when_no_leaf"]], "searxng_extra.update.update_languages": [[35, 2, 1, "", "UnicodeEscape"], [35, 0, 1, "", "get_unicode_flag"]]}, "objtypes": {"0": "py:function", "1": "py:module", "2": "py:class", "3": "py:data", "4": "py:property", "5": "py:method", "6": "py:attribute"}, "objnames": {"0": ["py", "function", "Python function"], "1": ["py", "module", "Python module"], "2": ["py", "class", "Python class"], "3": ["py", "data", "Python data"], "4": ["py", "property", "Python property"], "5": ["py", "method", "Python method"], "6": ["py", "attribute", "Python attribute"]}, "titleterms": {"administr": [0, 13], "api": [0, 32, 45], "get": [0, 16], "configur": [0, 4, 6, 8, 12, 19, 21, 24], "data": 0, "sampl": 0, "respons": 0, "emb": 0, "search": [0, 4, 9, 11, 27, 32, 50, 55, 57], "bar": 0, "architectur": 1, "further": [1, 4, 5, 6, 9, 11, 12, 15, 17, 18, 20, 21, 22, 24, 29, 31, 32, 59, 60], "read": [1, 4, 5, 6, 9, 11, 12, 15, 17, 18, 20, 21, 22, 24, 29, 31, 32, 59, 60], "uwsgi": [1, 15, 20], "setup": [1, 20, 24, 59], "buildhost": [2, 59], "thi": 2, "articl": [2, 31], "need": 2, "some": 2, "work": [2, 26], "content": [2, 11, 13, 15, 17, 19, 20, 22, 23, 24, 25, 26, 27, 31, 33, 38, 40, 45, 57, 58], "build": [2, 16, 23, 27, 31], "doc": [2, 23, 27], "sphinx": [2, 31], "lint": 2, "shell": [2, 16, 27], "script": [2, 18], "command": [3, 16, 58, 59], "line": [3, 16, 31], "engin": [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 24, 26, 27, 28, 42, 43, 44, 45, 47, 56, 57], "info": [3, 6, 8, 9, 12, 16, 22, 38, 48, 51], "acknowledg": [3, 6, 7, 9, 12, 28], "gener": [4, 11, 24, 31], "web": [4, 45], "other": 4, "imag": [4, 16, 24, 31, 45], "video": [4, 24, 45], "new": [4, 45], "map": [4, 24], "music": 4, "lyric": 4, "packag": [4, 19], "q": 4, "repo": 4, "softwar": 4, "wiki": 4, "scienc": 4, "scientif": 4, "public": [4, 39], "file": [4, 24, 31], "app": 4, "social": 4, "media": [4, 24], "set": [5, 11, 24], "nosql": 6, "databas": 6, "extra": [6, 12, 28], "depend": [6, 12, 19, 28], "redi": [6, 11, 53, 54], "server": [6, 11, 15, 17], "mongodb": 6, "privat": [7, 28, 39], "token": 7, "recol": 8, "exampl": [8, 11, 29, 31], "local": [9, 49], "meilisearch": 9, "elasticsearch": 9, "solr": 9, "xpath": 10, "yml": [11, 24], "locat": 11, "global": 11, "brand": 11, "buildenv": [11, 27], "ui": 11, "tip": 11, "develop": [11, 25, 26, 30], "outgo": 11, "categories_as_tab": 11, "multilingu": 11, "use_default_set": [11, 19], "true": [11, 19], "sql": 12, "sqlite": 12, "postgresql": 12, "mysql": 12, "document": [13, 23, 25], "instal": [14, 18, 19, 22, 27, 59, 60], "apach": 15, "The": [15, 17, 23, 24, 26], "http": [15, 17], "debian": [15, 20], "": [15, 17, 20], "layout": [15, 20], "modul": [15, 24], "site": [15, 17], "searxng": [15, 16, 17, 19, 22, 26, 37, 38, 39, 59, 60], "header": 15, "disabl": [15, 17], "log": [15, 17], "docker": [16, 59], "contain": [16, 26], "hint": 16, "run": [16, 27, 59], "warn": 16, "insid": 16, "bashism": 16, "nginx": 17, "updat": [18, 22, 35], "o": 18, "first": 18, "step": 19, "creat": [19, 23], "user": [19, 57], "check": [19, 22, 52], "origin": 20, "distributor": 20, "mainten": [20, 22], "pitfal": 20, "tyrant": 20, "mode": 20, "plugin": [21, 29, 50, 51, 52], "builtin": 21, "built": 21, "time": 21, "default": [21, 24], "how": [22, 23, 39], "inspect": 22, "debug": 22, "migrat": 22, "stai": 22, "tune": 22, "remov": 22, "obsolet": 22, "servic": 22, "after": 22, "contribut": 23, "prime": 23, "direct": 23, "privaci": [23, 39], "hackabl": 23, "design": 23, "code": [23, 31, 40], "good": 23, "commit": 23, "translat": [23, 36], "rest": [23, 31], "sourc": [23, 40], "live": 23, "clean": [23, 27], "deploi": 23, "github": 23, "io": 23, "overview": [24, 59, 60], "common": [24, 58], "option": 24, "overrid": 24, "name": [24, 27, 31], "i": [24, 39], "arbitrari": 24, "recommend": 24, "ar": [24, 39], "make": [24, 27], "request": 24, "pass": 24, "argument": 24, "If": 24, "engine_typ": 24, "onlin": [24, 44, 48], "online_dictionari": 24, "addit": 24, "online_curr": 24, "specifi": 24, "type": 24, "paramet": [24, 32], "torrent": 24, "paper": 24, "see": [24, 39], "bibtex": 24, "field": [24, 31], "format": 24, "linux": 26, "audienc": 26, "motiv": 26, "gentlemen": 26, "start": 26, "your": 26, "archlinux": 26, "fulli": 26, "function": [26, 56], "suit": [26, 59], "In": 26, "usual": 26, "wrap": 26, "product": 26, "summari": 26, "makefil": 27, "environ": [27, 31, 58], "python": 27, "activ": 27, "drop": 27, "node": 27, "j": 27, "env": 27, "nvm": 27, "nodej": 27, "autobuild": 27, "gh": 27, "page": 27, "test": 27, "pylint": 27, "checker": 27, "offlin": [28, 43], "program": 28, "interfac": 28, "secur": [28, 45], "extern": [29, 57], "entri": 29, "point": 29, "quickstart": 30, "primer": 31, "kiss": 31, "readabl": 31, "matter": 31, "soft": 31, "skill": 31, "basic": 31, "inlin": 31, "markup": 31, "structur": 31, "templat": 31, "head": 31, "anchor": 31, "link": 31, "ref": 31, "role": 31, "ordinari": 31, "url": 31, "hyperlink": 31, "smart": 31, "ext": 31, "extlink": 31, "intersphinx": 31, "liter": 31, "block": 31, "syntax": [31, 57], "highlight": 31, "unicod": 31, "substitut": 31, "figur": 31, "process": 31, "dot": 31, "aka": 31, "graphviz": 31, "hello": 31, "kernel": 31, "render": 31, "svg": 31, "list": 31, "bullet": 31, "horizont": 31, "hlist": 31, "definit": 31, "note": 31, "quot": 31, "paragraph": 31, "bibliograph": 31, "admonit": 31, "sidebar": 31, "titl": 31, "specif": 31, "tabl": 31, "nest": 31, "simpl": 31, "ascii": 31, "foo": 31, "gate": 31, "truth": 31, "grid": 31, "flat": 31, "csv": 31, "tab": 31, "view": 31, "math": 31, "equat": 31, "about": [31, 39], "latex": 31, "space": 31, "tool": [33, 58], "box": [33, 58], "searxng_extra": [33, 34, 35], "standalone_searx": 34, "py": [34, 35], "update_ahmia_blacklist": 35, "update_curr": 35, "update_engine_descript": 35, "update_external_bang": 35, "update_firefox_vers": 35, "update_languag": 35, "update_osm_keys_tag": 35, "update_pyg": 35, "update_wikidata_unit": 35, "wlc": 36, "donat": 37, "org": 37, "why": [37, 39], "payment": 37, "method": 37, "welcom": 38, "featur": 38, "us": 39, "instanc": 39, "doe": 39, "protect": 39, "what": 39, "consequ": 39, "conclus": 39, "custom": 41, "messag": 41, "extractor": 41, "i18n": 41, "load": 42, "demo": [43, 44], "googl": 45, "polici": 45, "csp": 45, "tiney": 46, "yahoo": 47, "languag": [50, 57], "limit": 51, "tor": 52, "db": 53, "librari": 54, "util": [56, 59, 60], "inform": 57, "select": 57, "categori": 57, "bang": 57, "special": 57, "queri": 57, "devop": 58, "lxc": 59, "sh": [59, 60], "internet": 59, "connect": 59}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 8, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.viewcode": 1, "sphinx.ext.intersphinx": 1, "sphinx": 57}, "alltitles": {"Administration API": [[0, "administration-api"]], "Get configuration data": [[0, "get-configuration-data"]], "Sample response": [[0, "sample-response"]], "Embed search bar": [[0, "embed-search-bar"]], "Architecture": [[1, "architecture"]], "Further reading": [[1, null], [31, null], [31, null]], "uWSGI Setup": [[1, "uwsgi-setup"]], "Buildhosts": [[2, "buildhosts"]], "This article needs some work": [[2, null]], "Contents": [[2, "contents"], [11, "contents"], [13, null], [15, "contents"], [17, "contents"], [19, "contents"], [20, "contents"], [22, "contents"], [23, "contents"], [24, "contents"], [25, null], [26, "contents"], [27, "contents"], [31, "contents"], [33, null], [38, null], [40, null], [45, "contents"], [57, "contents"], [58, null]], "Build docs": [[2, "build-docs"]], "Sphinx build needs": [[2, null]], "Lint shell scripts": [[2, "lint-shell-scripts"]], "Command Line Engines": [[3, "command-line-engines"]], "info": [[3, null], [6, null], [6, null], [8, null], [9, null], [9, null], [9, null], [12, null], [12, null], [12, null], [16, null], [22, null], [38, null], [51, null]], "Acknowledgment": [[3, "acknowledgment"], [6, "acknowledgment"], [7, "acknowledgment"], [9, "acknowledgment"], [12, "acknowledgment"]], "Engines & Settings": [[5, "engines-settings"]], "Further reading ..": [[5, null], [11, null], [11, null], [21, null], [24, null], [29, null], [32, null], [4, null]], "NoSQL databases": [[6, "nosql-databases"]], "further read": [[6, null], [9, null], [12, null], [15, null], [18, null], [18, null], [18, null], [22, null], [59, null]], "Configure the engines": [[6, "configure-the-engines"], [12, "configure-the-engines"]], "Extra Dependencies": [[6, "extra-dependencies"], [12, "extra-dependencies"], [28, "extra-dependencies"]], "Redis Server": [[6, "redis-server"]], "MongoDB": [[6, "engine-mongodb"]], "Private Engines (tokens)": [[7, "private-engines-tokens"]], "Recoll Engine": [[8, "recoll-engine"]], "Configuration": [[8, "configuration"], [19, "configuration"]], "Example": [[8, "example"]], "Local Search Engines": [[9, "local-search-engines"]], "MeiliSearch": [[9, "meilisearch"]], "Elasticsearch": [[9, "elasticsearch"]], "Solr": [[9, "solr"]], "XPath Engine": [[10, "xpath-engine"]], "settings.yml": [[11, "settings-yml"]], "settings.yml location": [[11, "settings-yml-location"]], "Global Settings": [[11, "global-settings"]], "brand:": [[11, "brand"]], "general:": [[11, "general"]], "search:": [[11, "search"]], "server:": [[11, "server"]], "buildenv": [[11, null]], "ui:": [[11, "ui"]], "redis:": [[11, "redis"]], "Tip for developers": [[11, null]], "outgoing:": [[11, "outgoing"]], "categories_as_tabs:": [[11, "categories-as-tabs"]], "Engine settings": [[11, "engine-settings"]], "Example: Multilingual Search": [[11, "example-multilingual-search"]], "use_default_settings": [[11, "use-default-settings"]], "use_default_settings: true": [[11, null]], "SQL Engines": [[12, "sql-engines"]], "SQLite": [[12, "engine-sqlite"]], "PostgreSQL": [[12, "engine-postgresql"]], "MySQL": [[12, "engine-mysql-server"]], "Administrator documentation": [[13, "administrator-documentation"]], "Installation": [[14, "installation"]], "Apache": [[15, "apache"]], "The Apache HTTP server": [[15, "the-apache-http-server"]], "Debian\u2019s Apache layout": [[15, "debian-s-apache-layout"]], "Apache modules": [[15, "apache-modules"]], "Apache sites": [[15, "apache-sites"]], "Apache\u2019s SearXNG site": [[15, "apache-s-searxng-site"]], "uWSGI": [[15, null], [20, "uwsgi"]], "HTTP headers": [[15, null]], "disable logs": [[15, "disable-logs"]], "Docker Container": [[16, "docker-container"]], "hint": [[16, null]], "Get Docker": [[16, "get-docker"]], "searxng/searxng": [[16, "searxng-searxng"]], "docker run": [[16, null], [16, null]], "Warning": [[16, null]], "shell inside container": [[16, "shell-inside-container"]], "Bashism": [[16, null]], "Build the image": [[16, "build-the-image"]], "Command line": [[16, "command-line"]], "NGINX": [[17, "nginx"]], "further reading": [[17, null], [20, null], [59, null], [60, null]], "The nginx HTTP server": [[17, "the-nginx-http-server"]], "NGINX\u2019s SearXNG site": [[17, "nginx-s-searxng-site"]], "Disable logs": [[17, "disable-logs"]], "Installation Script": [[18, "installation-script"]], "Update the OS first!": [[18, null]], "Step by step installation": [[19, "step-by-step-installation"]], "Install packages": [[19, "install-packages"]], "Create user": [[19, "create-user"]], "Install SearXNG & dependencies": [[19, "install-searxng-dependencies"]], "use_default_settings: True": [[19, null]], "Check": [[19, "check"]], "Origin uWSGI": [[20, "origin-uwsgi"]], "Distributors": [[20, "distributors"]], "Debian\u2019s uWSGI layout": [[20, "debian-s-uwsgi-layout"]], "uWSGI maintenance": [[20, "uwsgi-maintenance"]], "uWSGI setup": [[20, "uwsgi-setup"]], "Pitfalls of the Tyrant mode": [[20, "pitfalls-of-the-tyrant-mode"]], "Plugins builtin": [[21, "plugins-builtin"]], "Plugins configured at built time (defaults)": [[21, "id1"]], "SearXNG maintenance": [[22, "searxng-maintenance"]], "How to update": [[22, "how-to-update"]], "How to inspect & debug": [[22, "how-to-inspect-debug"]], "Migrate and stay tuned!": [[22, "migrate-and-stay-tuned"]], "remove obsolete services": [[22, "remove-obsolete-services"]], "Check after Installation": [[22, "check-after-installation"]], "How to contribute": [[23, "how-to-contribute"]], "Prime directives: Privacy, Hackability": [[23, "prime-directives-privacy-hackability"]], "Privacy-by-design": [[23, "privacy-by-design"]], "Code": [[23, "code"]], "Create good commits!": [[23, null]], "Translation": [[23, "translation"], [36, "translation"]], "Documentation": [[23, "documentation"]], "The reST sources": [[23, null]], "live build": [[23, "live-build"]], "docs.clean": [[23, null]], "deploy on github.io": [[23, "deploy-on-github-io"]], "Engine Overview": [[24, "engine-overview"]], "General Engine Configuration": [[24, "general-engine-configuration"]], "Engine File": [[24, "engine-file"]], "Common options in the engine module": [[24, "id3"]], "Engine settings.yml": [[24, "engine-settings-yml"]], "Common options in the engine setup (settings.yml)": [[24, "id4"]], "Overrides": [[24, "overrides"]], "The naming of overrides is arbitrary / recommended overrides are:": [[24, "id5"]], "Making a Request": [[24, "making-a-request"]], "Passed Arguments (request)": [[24, "passed-arguments-request"]], "If the engine_type is online": [[24, "id6"]], "If the engine_type is online_dictionary, in addition to the\n online arguments:": [[24, "id7"]], "If the engine_type is online_currency`, in addition to the\n online arguments:": [[24, "id8"]], "Specify Request": [[24, "specify-request"]], "Media Types": [[24, "media-types"]], "Parameter of the default media type:": [[24, "id9"]], "Parameter of the images media type:": [[24, "id10"]], "Parameter of the videos media type:": [[24, "id11"]], "Parameter of the torrent media type:": [[24, "id12"]], "Parameter of the map media type:": [[24, "id13"]], "Parameter of the paper media type /\n see BibTeX field types and BibTeX format": [[24, "id14"]], "Developer documentation": [[25, "developer-documentation"]], "Developing in Linux Containers": [[26, "developing-in-linux-containers"]], "Audience": [[26, null]], "Motivation": [[26, "motivation"]], "Gentlemen, start your engines!": [[26, "gentlemen-start-your-engines"]], "The searxng-archlinux container": [[26, null]], "Fully functional SearXNG suite": [[26, null]], "In containers, work as usual": [[26, "in-containers-work-as-usual"]], "Wrap production into developer suite": [[26, "wrap-production-into-developer-suite"]], "Summary": [[26, "summary"]], "Makefile": [[27, "makefile"]], "build environment": [[27, null]], "Python environment (make install)": [[27, "python-environment-make-install"]], "activate environment": [[27, null]], "drop environment": [[27, null]], "make buildenv": [[27, "make-buildenv"]], "Node.js environment (make node.env)": [[27, "node-js-environment-make-node-env"]], "make nvm.nodejs": [[27, "make-nvm-nodejs"]], "make run": [[27, "make-run"]], "make clean": [[27, "make-clean"]], "make docs docs.autobuild docs.clean": [[27, "make-docs-docs-autobuild-docs-clean"]], "make docs.gh-pages": [[27, "make-docs-gh-pages"]], "make test": [[27, "make-test"]], "make test.shell": [[27, "make-test-shell"]], "make test.pylint": [[27, "make-test-pylint"]], "search.checker.{engine name}": [[27, "search-checker-engine-name"]], "Offline Engines": [[28, "offline-engines"]], "offline engines": [[28, null]], "Programming Interface": [[28, "programming-interface"]], "Private engines (Security)": [[28, "private-engines-security"]], "Acknowledgement": [[28, "acknowledgement"]], "Plugins": [[29, "plugins"]], "Example plugin": [[29, "example-plugin"]], "External plugins": [[29, "external-plugins"]], "Plugin entry points": [[29, "plugin-entry-points"]], "Development Quickstart": [[30, "development-quickstart"]], "reST primer": [[31, "rest-primer"]], "KISS and readability": [[31, null]], "Content matters": [[31, null]], "Soft skills": [[31, "soft-skills"]], "Basic inline markup": [[31, "basic-inline-markup"]], "Inline markup": [[31, null]], "basic inline markup": [[31, "id4"]], "Basic article structure": [[31, "basic-article-structure"]], "reST template": [[31, "rest-template"]], "Headings": [[31, "headings"]], "Anchors & Links": [[31, "anchors-links"]], "Anchors": [[31, "anchors"]], ":ref: role": [[31, null]], "Link ordinary URL": [[31, "link-ordinary-url"]], "Named hyperlink": [[31, null]], "Smart refs": [[31, "smart-refs"]], "smart refs with sphinx.ext.extlinks and intersphinx": [[31, "id5"], [31, "id6"]], "Literal blocks": [[31, "literal-blocks"]], "::": [[31, "rest-literal"]], "Literal block": [[31, null]], "code-block": [[31, "code-block"]], "Syntax highlighting": [[31, null]], "Code block": [[31, null]], "Unicode substitution": [[31, "unicode-substitution"]], "Unicode": [[31, null]], "Roles": [[31, "roles"]], "Figures & Images": [[31, "figures-images"]], "Image processing": [[31, null]], "DOT files (aka Graphviz)": [[31, "dot-files-aka-graphviz"]], "hello.dot": [[31, null]], "kernel-render DOT": [[31, "kernel-render-dot"], [31, null]], "kernel-render SVG": [[31, "kernel-render-svg"], [31, null]], "List markups": [[31, "list-markups"]], "Bullet list": [[31, "bullet-list"]], "bullet list": [[31, null]], "Horizontal list": [[31, "horizontal-list"]], "hlist": [[31, null]], "Definition list": [[31, "definition-list"]], "Note ..": [[31, null]], "definition list": [[31, null]], "Quoted paragraphs": [[31, "quoted-paragraphs"]], "Quoted paragraph and line block": [[31, null]], "Field Lists": [[31, "field-lists"]], "bibliographic fields": [[31, null]], "Field List": [[31, null]], "Further list blocks": [[31, "further-list-blocks"]], "Admonitions": [[31, "admonitions"]], "Sidebar": [[31, "sidebar"]], "Generic admonition": [[31, "generic-admonition"]], "generic admonition title": [[31, null]], "Specific admonitions": [[31, "specific-admonitions"]], "Tables": [[31, "tables"]], "Nested tables": [[31, null]], "List tables": [[31, null]], "Simple tables": [[31, "simple-tables"]], "Simple ASCII table": [[31, null]], "foo gate truth table": [[31, "id11"]], "Grid tables": [[31, "grid-tables"]], "ASCII grid table": [[31, null]], "grid table example": [[31, "id12"]], "flat-table": [[31, "flat-table"]], "List table": [[31, null]], "flat-table example": [[31, "id13"]], "CSV table": [[31, "csv-table"], [31, null]], "CSV table example": [[31, "id14"]], "Templating": [[31, "templating"]], "Build environment": [[31, null]], "Tabbed views": [[31, "tabbed-views"]], "Math equations": [[31, "math-equations"]], "About LaTeX": [[31, null]], "LaTeX math equation": [[31, null]], "Line spacing": [[31, null]], "Search API": [[32, "search-api"]], "Parameters": [[32, "parameters"]], "Tooling box searxng_extra": [[33, "tooling-box-searxng-extra"]], "searxng_extra/standalone_searx.py": [[34, "module-searxng_extra.standalone_searx"]], "searxng_extra/update/": [[35, "searxng-extra-update"]], "update_ahmia_blacklist.py": [[35, "update-ahmia-blacklist-py"]], "update_currencies.py": [[35, "update-currencies-py"]], "update_engine_descriptions.py": [[35, "update-engine-descriptions-py"]], "update_external_bangs.py": [[35, "update-external-bangs-py"]], "update_firefox_version.py": [[35, "update-firefox-version-py"]], "update_languages.py": [[35, "update-languages-py"]], "update_osm_keys_tags.py": [[35, "update-osm-keys-tags-py"]], "update_pygments.py": [[35, "update-pygments-py"]], "update_wikidata_units.py": [[35, "update-wikidata-units-py"]], "translated": [[36, null]], "wlc": [[36, "id2"]], "Donate to searxng.org": [[37, "donate-to-searxng-org"]], "Why donating?": [[37, "why-donating"]], "Payment methods": [[37, "payment-methods"]], "Welcome to SearXNG": [[38, "welcome-to-searxng"]], "Features": [[38, null]], "Why use a private instance?": [[39, "why-use-a-private-instance"]], "How does SearXNG protect privacy?": [[39, "how-does-searxng-protect-privacy"]], "What are the consequences of using public instances?": [[39, "what-are-the-consequences-of-using-public-instances"]], "I see. What about private instances?": [[39, "i-see-what-about-private-instances"]], "Conclusion": [[39, "conclusion"]], "Source-Code": [[40, "source-code"]], "Custom message extractor (i18n)": [[41, "module-searx.babel_extract"]], "Load Engines": [[42, "module-searx.engines"]], "Demo Offline Engine": [[43, "demo-offline-engine"]], "Demo Online Engine": [[44, "demo-online-engine"]], "Google Engines": [[45, "google-engines"]], "google API": [[45, "google-api"]], "Google WEB": [[45, "module-searx.engines.google"]], "Google Images": [[45, "module-searx.engines.google_images"]], "Google Videos": [[45, "module-searx.engines.google_videos"]], "Content-Security-Policy (CSP)": [[45, null]], "Google News": [[45, "module-searx.engines.google_news"]], "Tineye": [[46, "module-searx.engines.tineye"]], "Yahoo Engine": [[47, "yahoo-engine"]], "Online /info": [[48, "module-searx.infopage"]], "Locales": [[49, "module-searx.locales"]], "Search language plugin": [[50, "module-searx.plugins.autodetect_search_language"]], "Limiter Plugin": [[51, "limiter-plugin"]], "Tor check plugin": [[52, "tor-check-plugin"]], "Redis DB": [[53, "redis-db"]], "Redis Library": [[54, "module-searx.redislib"]], "Search": [[55, "search"]], "Utility functions for the engines": [[56, "module-searx.utils"]], "User information": [[57, "user-information"]], "Search syntax": [[57, "search-syntax"]], "! select engine and category": [[57, "select-engine-and-category"]], ": select language": [[57, "select-language"]], "!! external bangs": [[57, "external-bangs"]], "Special Queries": [[57, "special-queries"]], "DevOps tooling box": [[58, "devops-tooling-box"]], "Common command environments": [[58, "common-command-environments"]], "utils/lxc.sh": [[59, "utils-lxc-sh"]], "Internet Connectivity & Docker": [[59, "internet-connectivity-docker"]], "Install suite": [[59, "install-suite"]], "Running commands": [[59, "running-commands"]], "Setup SearXNG buildhost": [[59, "setup-searxng-buildhost"]], "Overview": [[59, "overview"], [60, "overview"]], "SearXNG suite": [[59, "searxng-suite"]], "utils/searxng.sh": [[60, "utils-searxng-sh"]], "Install": [[60, "install"]], "Configured Engines": [[4, "configured-engines"]], "general search engines": [[4, "general-search-engines"]], "web": [[4, "web"], [4, "id2"], [4, "id4"]], "others": [[4, "others"], [4, "id3"], [4, "id5"], [4, "id6"], [4, "id9"], [4, "id10"], [4, "id11"]], "images search engines": [[4, "images-search-engines"]], "videos search engines": [[4, "videos-search-engines"]], "news search engines": [[4, "news-search-engines"]], "map search engines": [[4, "map-search-engines"]], "music search engines": [[4, "music-search-engines"]], "lyrics": [[4, "lyrics"]], "it search engines": [[4, "it-search-engines"]], "packages": [[4, "packages"]], "q&a": [[4, "q-a"]], "repos": [[4, "repos"]], "software wikis": [[4, "software-wikis"]], "science search engines": [[4, "science-search-engines"]], "scientific publications": [[4, "scientific-publications"]], "files search engines": [[4, "files-search-engines"]], "apps": [[4, "apps"]], "social media search engines": [[4, "social-media-search-engines"]]}, "indexentries": {"content_xpath (in module searx.engines.xpath)": [[10, "searx.engines.xpath.content_xpath"]], "first_page_num (in module searx.engines.xpath)": [[10, "searx.engines.xpath.first_page_num"]], "headers (in module searx.engines.xpath)": [[10, "searx.engines.xpath.headers"]], "lang_all (in module searx.engines.xpath)": [[10, "searx.engines.xpath.lang_all"]], "module": [[10, "module-searx.engines.xpath"], [34, "module-searxng_extra.standalone_searx"], [35, "module-searxng_extra.update.update_ahmia_blacklist"], [35, "module-searxng_extra.update.update_currencies"], [35, "module-searxng_extra.update.update_engine_descriptions"], [35, "module-searxng_extra.update.update_external_bangs"], [35, "module-searxng_extra.update.update_firefox_version"], [35, "module-searxng_extra.update.update_languages"], [35, "module-searxng_extra.update.update_osm_keys_tags"], [35, "module-searxng_extra.update.update_pygments"], [35, "module-searxng_extra.update.update_wikidata_units"], [41, "module-searx.babel_extract"], [42, "module-searx.engines"], [43, "module-searx.engines.demo_offline"], [44, "module-searx.engines.demo_online"], [45, "module-searx.engines.google"], [45, "module-searx.engines.google_images"], [45, "module-searx.engines.google_news"], [45, "module-searx.engines.google_videos"], [46, "module-searx.engines.tineye"], [47, "module-searx.engines.yahoo"], [48, "module-searx.infopage"], [49, "module-searx.locales"], [50, "module-searx.plugins.autodetect_search_language"], [51, "module-searx.plugins.limiter"], [52, "module-searx.plugins.tor_check"], [53, "module-searx.redisdb"], [54, "module-searx.redislib"], [56, "module-searx.utils"]], "no_result_for_http_status (in module searx.engines.xpath)": [[10, "searx.engines.xpath.no_result_for_http_status"]], "page_size (in module searx.engines.xpath)": [[10, "searx.engines.xpath.page_size"]], "paging (in module searx.engines.xpath)": [[10, "searx.engines.xpath.paging"]], "request() (in module searx.engines.xpath)": [[10, "searx.engines.xpath.request"]], "response() (in module searx.engines.xpath)": [[10, "searx.engines.xpath.response"]], "results_xpath (in module searx.engines.xpath)": [[10, "searx.engines.xpath.results_xpath"]], "safe_search_map (in module searx.engines.xpath)": [[10, "searx.engines.xpath.safe_search_map"]], "safe_search_support (in module searx.engines.xpath)": [[10, "searx.engines.xpath.safe_search_support"]], "search_url (in module searx.engines.xpath)": [[10, "searx.engines.xpath.search_url"]], "searx.engines.xpath": [[10, "module-searx.engines.xpath"]], "soft_max_redirects (in module searx.engines.xpath)": [[10, "searx.engines.xpath.soft_max_redirects"]], "suggestion_xpath (in module searx.engines.xpath)": [[10, "searx.engines.xpath.suggestion_xpath"]], "thumbnail_xpath (in module searx.engines.xpath)": [[10, "searx.engines.xpath.thumbnail_xpath"]], "time_range_map (in module searx.engines.xpath)": [[10, "searx.engines.xpath.time_range_map"]], "time_range_support (in module searx.engines.xpath)": [[10, "searx.engines.xpath.time_range_support"]], "time_range_url (in module searx.engines.xpath)": [[10, "searx.engines.xpath.time_range_url"]], "title_xpath (in module searx.engines.xpath)": [[10, "searx.engines.xpath.title_xpath"]], "url_xpath (in module searx.engines.xpath)": [[10, "searx.engines.xpath.url_xpath"]], "built-in function": [[29, "on_result"], [29, "post_search"], [29, "pre_search"]], "on_result()": [[29, "on_result"]], "post_search()": [[29, "post_search"]], "pre_search()": [[29, "pre_search"]], "pep 8": [[31, "index-1"]], "python enhancement proposals": [[31, "index-1"]], "rfc": [[31, "index-0"]], "rfc 822": [[31, "index-0"]], "get_search_query() (in module searxng_extra.standalone_searx)": [[34, "searxng_extra.standalone_searx.get_search_query"]], "json_serial() (in module searxng_extra.standalone_searx)": [[34, "searxng_extra.standalone_searx.json_serial"]], "no_parsed_url() (in module searxng_extra.standalone_searx)": [[34, "searxng_extra.standalone_searx.no_parsed_url"]], "parse_argument() (in module searxng_extra.standalone_searx)": [[34, "searxng_extra.standalone_searx.parse_argument"]], "searxng_extra.standalone_searx": [[34, "module-searxng_extra.standalone_searx"]], "to_dict() (in module searxng_extra.standalone_searx)": [[34, "searxng_extra.standalone_searx.to_dict"]], "unicodeescape (class in searxng_extra.update.update_languages)": [[35, "searxng_extra.update.update_languages.UnicodeEscape"]], "get_output() (in module searxng_extra.update.update_engine_descriptions)": [[35, "searxng_extra.update.update_engine_descriptions.get_output"]], "get_unicode_flag() (in module searxng_extra.update.update_languages)": [[35, "searxng_extra.update.update_languages.get_unicode_flag"]], "merge_when_no_leaf() (in module searxng_extra.update.update_external_bangs)": [[35, "searxng_extra.update.update_external_bangs.merge_when_no_leaf"]], "searxng_extra.update.update_ahmia_blacklist": [[35, "module-searxng_extra.update.update_ahmia_blacklist"]], "searxng_extra.update.update_currencies": [[35, "module-searxng_extra.update.update_currencies"]], "searxng_extra.update.update_engine_descriptions": [[35, "module-searxng_extra.update.update_engine_descriptions"]], "searxng_extra.update.update_external_bangs": [[35, "module-searxng_extra.update.update_external_bangs"]], "searxng_extra.update.update_firefox_version": [[35, "module-searxng_extra.update.update_firefox_version"]], "searxng_extra.update.update_languages": [[35, "module-searxng_extra.update.update_languages"]], "searxng_extra.update.update_osm_keys_tags": [[35, "module-searxng_extra.update.update_osm_keys_tags"]], "searxng_extra.update.update_pygments": [[35, "module-searxng_extra.update.update_pygments"]], "searxng_extra.update.update_wikidata_units": [[35, "module-searxng_extra.update.update_wikidata_units"]], "extract() (in module searx.babel_extract)": [[41, "searx.babel_extract.extract"]], "searx.babel_extract": [[41, "module-searx.babel_extract"]], "engine (class in searx.engines)": [[42, "searx.engines.Engine"]], "engine_shortcuts (in module searx.engines)": [[42, "searx.engines.engine_shortcuts"]], "is_missing_required_attributes() (in module searx.engines)": [[42, "searx.engines.is_missing_required_attributes"]], "load_engine() (in module searx.engines)": [[42, "searx.engines.load_engine"]], "load_engines() (in module searx.engines)": [[42, "searx.engines.load_engines"]], "searx.engines": [[42, "module-searx.engines"]], "using_tor_proxy() (in module searx.engines)": [[42, "searx.engines.using_tor_proxy"]], "init() (in module searx.engines.demo_offline)": [[43, "searx.engines.demo_offline.init"]], "search() (in module searx.engines.demo_offline)": [[43, "searx.engines.demo_offline.search"]], "searx.engines.demo_offline": [[43, "module-searx.engines.demo_offline"]], "init() (in module searx.engines.demo_online)": [[44, "searx.engines.demo_online.init"]], "request() (in module searx.engines.demo_online)": [[44, "searx.engines.demo_online.request"]], "response() (in module searx.engines.demo_online)": [[44, "searx.engines.demo_online.response"]], "searx.engines.demo_online": [[44, "module-searx.engines.demo_online"]], "get_lang_info() (in module searx.engines.google)": [[45, "searx.engines.google.get_lang_info"]], "request() (in module searx.engines.google)": [[45, "searx.engines.google.request"]], "request() (in module searx.engines.google_images)": [[45, "searx.engines.google_images.request"]], "request() (in module searx.engines.google_news)": [[45, "searx.engines.google_news.request"]], "request() (in module searx.engines.google_videos)": [[45, "searx.engines.google_videos.request"]], "response() (in module searx.engines.google)": [[45, "searx.engines.google.response"]], "response() (in module searx.engines.google_images)": [[45, "searx.engines.google_images.response"]], "response() (in module searx.engines.google_news)": [[45, "searx.engines.google_news.response"]], "response() (in module searx.engines.google_videos)": [[45, "searx.engines.google_videos.response"]], "scrap_out_thumbs() (in module searx.engines.google_videos)": [[45, "searx.engines.google_videos.scrap_out_thumbs"]], "searx.engines.google": [[45, "module-searx.engines.google"]], "searx.engines.google_images": [[45, "module-searx.engines.google_images"]], "searx.engines.google_news": [[45, "module-searx.engines.google_news"]], "searx.engines.google_videos": [[45, "module-searx.engines.google_videos"]], "download_error (in module searx.engines.tineye)": [[46, "searx.engines.tineye.DOWNLOAD_ERROR"]], "format_not_supported (in module searx.engines.tineye)": [[46, "searx.engines.tineye.FORMAT_NOT_SUPPORTED"]], "no_signature_error (in module searx.engines.tineye)": [[46, "searx.engines.tineye.NO_SIGNATURE_ERROR"]], "engine_type (in module searx.engines.tineye)": [[46, "searx.engines.tineye.engine_type"]], "parse_tineye_match() (in module searx.engines.tineye)": [[46, "searx.engines.tineye.parse_tineye_match"]], "request() (in module searx.engines.tineye)": [[46, "searx.engines.tineye.request"]], "response() (in module searx.engines.tineye)": [[46, "searx.engines.tineye.response"]], "searx.engines.tineye": [[46, "module-searx.engines.tineye"]], "lang2domain (in module searx.engines.yahoo)": [[47, "searx.engines.yahoo.lang2domain"]], "parse_url() (in module searx.engines.yahoo)": [[47, "searx.engines.yahoo.parse_url"]], "request() (in module searx.engines.yahoo)": [[47, "searx.engines.yahoo.request"]], "response() (in module searx.engines.yahoo)": [[47, "searx.engines.yahoo.response"]], "searx.engines.yahoo": [[47, "module-searx.engines.yahoo"]], "supported_languages_url (in module searx.engines.yahoo)": [[47, "searx.engines.yahoo.supported_languages_url"]], "infopage (class in searx.infopage)": [[48, "searx.infopage.InfoPage"]], "infopageset (class in searx.infopage)": [[48, "searx.infopage.InfoPageSet"]], "content (searx.infopage.infopage property)": [[48, "searx.infopage.InfoPage.content"]], "folder (searx.infopage.infopageset attribute)": [[48, "searx.infopage.InfoPageSet.folder"]], "get_ctx() (searx.infopage.infopage method)": [[48, "searx.infopage.InfoPage.get_ctx"]], "get_page() (searx.infopage.infopageset method)": [[48, "searx.infopage.InfoPageSet.get_page"]], "html (searx.infopage.infopage property)": [[48, "searx.infopage.InfoPage.html"]], "iter_pages() (searx.infopage.infopageset method)": [[48, "searx.infopage.InfoPageSet.iter_pages"]], "locale_default (searx.infopage.infopageset attribute)": [[48, "searx.infopage.InfoPageSet.locale_default"]], "locales (searx.infopage.infopageset attribute)": [[48, "searx.infopage.InfoPageSet.locales"]], "raw_content (searx.infopage.infopage property)": [[48, "searx.infopage.InfoPage.raw_content"]], "searx.infopage": [[48, "module-searx.infopage"]], "title (searx.infopage.infopage property)": [[48, "searx.infopage.InfoPage.title"]], "toc (searx.infopage.infopageset attribute)": [[48, "searx.infopage.InfoPageSet.toc"]], "additional_translations (in module searx.locales)": [[49, "searx.locales.ADDITIONAL_TRANSLATIONS"]], "locale_best_match (in module searx.locales)": [[49, "searx.locales.LOCALE_BEST_MATCH"]], "locale_names (in module searx.locales)": [[49, "searx.locales.LOCALE_NAMES"]], "rtl_locales (in module searx.locales)": [[49, "searx.locales.RTL_LOCALES"]], "get_engine_locale() (in module searx.locales)": [[49, "searx.locales.get_engine_locale"]], "get_locale_descr() (in module searx.locales)": [[49, "searx.locales.get_locale_descr"]], "get_translations() (in module searx.locales)": [[49, "searx.locales.get_translations"]], "locales_initialize() (in module searx.locales)": [[49, "searx.locales.locales_initialize"]], "searx.locales": [[49, "module-searx.locales"]], "searx.plugins.autodetect_search_language": [[50, "module-searx.plugins.autodetect_search_language"]], "supported_langs (in module searx.plugins.autodetect_search_language)": [[50, "searx.plugins.autodetect_search_language.supported_langs"]], "searx.plugins.limiter": [[51, "module-searx.plugins.limiter"]], "description (in module searx.plugins.tor_check)": [[52, "searx.plugins.tor_check.description"]], "name (in module searx.plugins.tor_check)": [[52, "searx.plugins.tor_check.name"]], "preference_section (in module searx.plugins.tor_check)": [[52, "searx.plugins.tor_check.preference_section"]], "query_examples (in module searx.plugins.tor_check)": [[52, "searx.plugins.tor_check.query_examples"]], "query_keywords (in module searx.plugins.tor_check)": [[52, "searx.plugins.tor_check.query_keywords"]], "searx.plugins.tor_check": [[52, "module-searx.plugins.tor_check"]], "old_redis_url_default_url (in module searx.redisdb)": [[53, "searx.redisdb.OLD_REDIS_URL_DEFAULT_URL"]], "searx.redisdb": [[53, "module-searx.redisdb"]], "lua_script_storage (in module searx.redislib)": [[54, "searx.redislib.LUA_SCRIPT_STORAGE"]], "drop_counter() (in module searx.redislib)": [[54, "searx.redislib.drop_counter"]], "incr_counter() (in module searx.redislib)": [[54, "searx.redislib.incr_counter"]], "incr_sliding_window() (in module searx.redislib)": [[54, "searx.redislib.incr_sliding_window"]], "lua_script_storage() (in module searx.redislib)": [[54, "searx.redislib.lua_script_storage"]], "purge_by_prefix() (in module searx.redislib)": [[54, "searx.redislib.purge_by_prefix"]], "searx.redislib": [[54, "module-searx.redislib"]], "secret_hash() (in module searx.redislib)": [[54, "searx.redislib.secret_hash"]], "engineref (class in searx.search)": [[55, "searx.search.EngineRef"]], "search (class in searx.search)": [[55, "searx.search.Search"]], "searchquery (class in searx.search)": [[55, "searx.search.SearchQuery"]], "searchwithplugins (class in searx.search)": [[55, "searx.search.SearchWithPlugins"]], "ordered_plugin_list (searx.search.searchwithplugins attribute)": [[55, "searx.search.SearchWithPlugins.ordered_plugin_list"]], "request (searx.search.searchwithplugins attribute)": [[55, "searx.search.SearchWithPlugins.request"]], "result_container (searx.search.search attribute)": [[55, "searx.search.Search.result_container"]], "result_container (searx.search.searchwithplugins attribute)": [[55, "searx.search.SearchWithPlugins.result_container"]], "search() (searx.search.search method)": [[55, "searx.search.Search.search"]], "search() (searx.search.searchwithplugins method)": [[55, "searx.search.SearchWithPlugins.search"]], "search_query (searx.search.search attribute)": [[55, "searx.search.Search.search_query"]], "search_query (searx.search.searchwithplugins attribute)": [[55, "searx.search.SearchWithPlugins.search_query"]], "convert_str_to_int() (in module searx.utils)": [[56, "searx.utils.convert_str_to_int"]], "detect_language() (in module searx.utils)": [[56, "searx.utils.detect_language"]], "dict_subset() (in module searx.utils)": [[56, "searx.utils.dict_subset"]], "ecma_unescape() (in module searx.utils)": [[56, "searx.utils.ecma_unescape"]], "eval_xpath() (in module searx.utils)": [[56, "searx.utils.eval_xpath"]], "eval_xpath_getindex() (in module searx.utils)": [[56, "searx.utils.eval_xpath_getindex"]], "eval_xpath_list() (in module searx.utils)": [[56, "searx.utils.eval_xpath_list"]], "extract_text() (in module searx.utils)": [[56, "searx.utils.extract_text"]], "extract_url() (in module searx.utils)": [[56, "searx.utils.extract_url"]], "gen_useragent() (in module searx.utils)": [[56, "searx.utils.gen_useragent"]], "get_engine_from_settings() (in module searx.utils)": [[56, "searx.utils.get_engine_from_settings"]], "get_torrent_size() (in module searx.utils)": [[56, "searx.utils.get_torrent_size"]], "get_xpath() (in module searx.utils)": [[56, "searx.utils.get_xpath"]], "html_to_text() (in module searx.utils)": [[56, "searx.utils.html_to_text"]], "int_or_zero() (in module searx.utils)": [[56, "searx.utils.int_or_zero"]], "is_valid_lang() (in module searx.utils)": [[56, "searx.utils.is_valid_lang"]], "match_language() (in module searx.utils)": [[56, "searx.utils.match_language"]], "normalize_url() (in module searx.utils)": [[56, "searx.utils.normalize_url"]], "searx.utils": [[56, "module-searx.utils"]], "searx_useragent() (in module searx.utils)": [[56, "searx.utils.searx_useragent"]], "to_string() (in module searx.utils)": [[56, "searx.utils.to_string"]]}}) \ No newline at end of file diff --git a/src/index.html b/src/index.html new file mode 100644 index 00000000..9771d5af --- /dev/null +++ b/src/index.html @@ -0,0 +1,275 @@ + + + + + + + + + Source-Code — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Source-Code

+

This is a partial documentation of our source code. We are not aiming to document +every item from the source code, but we will add documentation when requested.

+
+

Contents

+ +
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.babel_extract.html b/src/searx.babel_extract.html new file mode 100644 index 00000000..2ae9d4f8 --- /dev/null +++ b/src/searx.babel_extract.html @@ -0,0 +1,173 @@ + + + + + + + + + Custom message extractor (i18n) — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Custom message extractor (i18n)

+

This module implements the searxng_msg extractor to +extract messages from:

+ +

The searxng.msg files are selected by Babel, see Babel’s configuration in +git://babel.cfg:

+
searxng_msg = searx.babel_extract.extract
+...
+[searxng_msg: **/searxng.msg]
+
+
+

A searxng.msg file is a python file that is executed by the +extract function. Additional searxng.msg files can be added by:

+
    +

  1. Adding a searxng.msg file in one of the SearXNG python packages and

    2. +

  2. implement a method in extract that yields messages from this file.

    3. +
+
+
+searx.babel_extract.extract(fileobj, keywords, comment_tags, options)[source]
+

Extract messages from searxng.msg files by a custom extractor.

+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.engines.demo_offline.html b/src/searx.engines.demo_offline.html new file mode 100644 index 00000000..0e37695a --- /dev/null +++ b/src/searx.engines.demo_offline.html @@ -0,0 +1,175 @@ + + + + + + + + + Demo Offline Engine — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Demo Offline Engine

+

Within this module we implement a demo offline engine. Do not look to +close to the implementation, its just a simple example. To get in use of this +demo engine add the following entry to your engines list in settings.yml:

+
- name: my offline engine
+  engine: demo_offline
+  shortcut: demo
+  disabled: false
+
+
+
+
+searx.engines.demo_offline.init(engine_settings=None)[source]
+

Initialization of the (offline) engine. The origin of this demo engine is a +simple json string which is loaded in this example while the engine is +initialized.

+
+ +
+
+searx.engines.demo_offline.search(query, request_params)[source]
+

Query (offline) engine and return results. Assemble the list of results from +your local engine. In this demo engine we ignore the ‘query’ term, usual +you would pass the ‘query’ term to your local engine to filter out the +results.

+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.engines.demo_online.html b/src/searx.engines.demo_online.html new file mode 100644 index 00000000..d74b7d99 --- /dev/null +++ b/src/searx.engines.demo_online.html @@ -0,0 +1,182 @@ + + + + + + + + + Demo Online Engine — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Demo Online Engine

+

Within this module we implement a demo online engine. Do not look to +close to the implementation, its just a simple example which queries The Art +Institute of Chicago

+

To get in use of this demo engine add the following entry to your engines +list in settings.yml:

+
- name: my online engine
+  engine: demo_online
+  shortcut: demo
+  disabled: false
+
+
+
+
+searx.engines.demo_online.init(engine_settings)[source]
+

Initialization of the (online) engine. If no initialization is needed, drop +this init function.

+
+ +
+
+searx.engines.demo_online.request(query, params)[source]
+

Build up the params for the online request. In this example we build a +URL to fetch images from artic.edu

+
+ +
+
+searx.engines.demo_online.response(resp)[source]
+

Parse out the result items from the response. In this example we parse the +response from api.artic.edu and filter out all +images.

+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.engines.google.html b/src/searx.engines.google.html new file mode 100644 index 00000000..055f7276 --- /dev/null +++ b/src/searx.engines.google.html @@ -0,0 +1,324 @@ + + + + + + + + + Google Engines — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Google Engines

+ +
+

google API

+

For detailed description of the REST-full API see: Query Parameter +Definitions. Not all parameters can be appied and some engines are special +(e.g. Google News).

+
+
+

Google WEB

+

This is the implementation of the google WEB engine. Some of this +implementations are shared by other engines:

+ +

The google WEB engine itself has a special setup option:

+
- name: google
+  ...
+  use_mobile_ui: false
+
+
+
+
use_mobile_ui: (default: false)

Enables to use mobile endpoint to bypass the google blocking (see +#159). On the mobile UI of Google Search, the button More +results is not affected by Google rate limiting and we can still do requests +while actively blocked by the original Google search. By activate +use_mobile_ui this behavior is simulated by adding the parameter +async=use_ac:true,_fmt:pc to the request().

+
+
+
+
+searx.engines.google.get_lang_info(params, lang_list, custom_aliases, supported_any_language)[source]
+

Composing various language properties for the google engines.

+

This function is called by the various google engines (Google WEB, Google Images, Google News and +Google Videos).

+
+
Parameters:
+
    +

  • param (dict) – request parameters of the engine

    • +

  • lang_list (dict) – list of supported languages of the engine +ENGINES_LANGUAGES[engine-name]

    • +

  • lang_list – custom aliases for non standard language codes +(used when calling searx.utils.match_language())

    • +

  • supported_any_language (bool) – When a language is not specified, the +language interpretation is left up to Google to decide how the search +results should be delivered. This argument is True for the google +engine and False for the other engines (google-images, -news, +-scholar, -videos).

    • +
+
+
Return type:
+

dict

+
+
Returns:
+

Py-Dictionary with the key/value pairs:

+
+
language:

Return value from searx.utils.match_language()

+
+
country:

The country code (e.g. US, AT, CA, FR, DE ..)

+
+
subdomain:

Google subdomain google_domains that fits to the country +code.

+
+
params:

Py-Dictionary with additional request arguments (can be passed to +urllib.parse.urlencode()).

+
+
headers:

Py-Dictionary with additional HTTP headers (can be passed to +request’s headers)

+
+
+

+
+
+
+ +
+
+searx.engines.google.request(query, params)[source]
+

Google search request

+
+ +
+
+searx.engines.google.response(resp)[source]
+

Get response from google’s search request

+
+ +
+
+

Google Images

+

This is the implementation of the google images engine using the google +internal API used the Google Go Android app.

+

This internal API offer results in

+
    +

  • JSON (_fmt:json)

    • +

  • Protobuf (_fmt:pb)

    • +

  • Protobuf compressed? (_fmt:pc)

    • +

  • HTML (_fmt:html)

    • +

  • Protobuf encoded in JSON (_fmt:jspb).

    • +
+
+
+searx.engines.google_images.request(query, params)[source]
+

Google-Image search request

+
+ +
+
+searx.engines.google_images.response(resp)[source]
+

Get response from google’s search request

+
+ +
+
+

Google Videos

+

This is the implementation of the google videos engine.

+
+

Content-Security-Policy (CSP)

+

This engine needs to allow images from the data URLs (prefixed with the +data: scheme):

+
Header set Content-Security-Policy "img-src 'self' data: ;"
+
+
+
+
+
+searx.engines.google_videos.request(query, params)[source]
+

Google-Video search request

+
+ +
+
+searx.engines.google_videos.response(resp)[source]
+

Get response from google’s search request

+
+ +
+
+searx.engines.google_videos.scrap_out_thumbs(dom)[source]
+

Scrap out thumbnail data from <script> tags.

+
+ +
+
+

Google News

+

This is the implementation of the google news engine. The google news API +ignores some parameters from the common google API:

+
    +

  • num : the number of search results is ignored

    • +

  • save : is ignored / Google-News results are always SafeSearch

    • +
+
+
+searx.engines.google_news.request(query, params)[source]
+

Google-News search request

+
+ +
+
+searx.engines.google_news.response(resp)[source]
+

Get response from google’s search request

+
+ +
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.engines.html b/src/searx.engines.html new file mode 100644 index 00000000..8333e551 --- /dev/null +++ b/src/searx.engines.html @@ -0,0 +1,220 @@ + + + + + + + + + Load Engines — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Load Engines

+

This module implements the engine loader.

+

Load and initialize the engines, see load_engines() and register +engine_shortcuts.

+

usage:

+
load_engines( settings['engines'] )
+
+
+
+
+class searx.engines.Engine[source]
+

This class is currently never initialized and only used for type hinting.

+
+ +
+
+searx.engines.engine_shortcuts
+

Simple map of registered shortcuts to name of the engine (or None).

+
engine_shortcuts[engine.shortcut] = engine.name
+
+
+
+
+
+ +
+
+searx.engines.is_missing_required_attributes(engine)[source]
+

An attribute is required when its name doesn’t start with _ (underline). +Required attributes must not be None.

+
+ +
+
+searx.engines.load_engine(engine_data: dict) → Optional[Engine][source]
+

Load engine from engine_data.

+
+
Parameters:
+

engine_data (dict) – Attributes from YAML settings:engines/<engine>

+
+
Returns:
+

initialized namespace of the <engine>.

+
+
+
    +

  1. create a namespace and load module of the <engine>

    2. +

  2. update namespace with the defaults from ENGINE_DEFAULT_ARGS

    3. +

  3. update namespace with values from engine_data

    4. +
+

If engine is active, return namespace of the engine, otherwise return +None.

+

This function also returns None if initialization of the namespace fails +for one of the following reasons:

+ +
+ +
+
+searx.engines.load_engines(engine_list)[source]
+

usage: engine_list = settings['engines']

+
+ +
+
+searx.engines.using_tor_proxy(engine: Engine)[source]
+

Return True if the engine configuration declares to use Tor.

+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.engines.tineye.html b/src/searx.engines.tineye.html new file mode 100644 index 00000000..4c39ec25 --- /dev/null +++ b/src/searx.engines.tineye.html @@ -0,0 +1,224 @@ + + + + + + + + + Tineye — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Tineye

+

This engine implements Tineye - reverse image search

+

Using TinEye, you can search by image or perform what we call a reverse image +search. You can do that by uploading an image or searching by URL. You can also +simply drag and drop your images to start your search. TinEye constantly crawls +the web and adds images to its index. Today, the TinEye index is over 50.2 +billion images [tineye.com].

+
+

Hint

+

This SearXNG engine only supports ‘searching by URL’ and it does not use +the official API [api.tineye.com].

+
+
+
+searx.engines.tineye.DOWNLOAD_ERROR = 'The image could not be downloaded.'
+

TinEye error message

+
+ +
+
+searx.engines.tineye.FORMAT_NOT_SUPPORTED = 'Could not read that image url. This may be due to an unsupported file format. TinEye only supports images that are JPEG, PNG, GIF, BMP, TIFF or WebP.'
+

TinEye error message

+
+ +
+
+searx.engines.tineye.NO_SIGNATURE_ERROR = 'The image is too simple to find matches. TinEye requires a basic level of visual detail to successfully identify matches.'
+

TinEye error message

+
+ +
+
+searx.engines.tineye.engine_type = 'online_url_search'
+

searx.search.processors.online_url_search

+
+ +
+
+searx.engines.tineye.parse_tineye_match(match_json)[source]
+

Takes parsed JSON from the API server and turns it into a dict +object.

+

Attributes (class Match)

+
    +

  • image_url, link to the result image.

    • +

  • domain, domain this result was found on.

    • +

  • score, a number (0 to 100) that indicates how closely the images match.

    • +

  • width, image width in pixels.

    • +

  • height, image height in pixels.

    • +

  • size, image area in pixels.

    • +

  • format, image format.

    • +

  • filesize, image size in bytes.

    • +

  • overlay, overlay URL.

    • +

  • tags, whether this match belongs to a collection or stock domain.

    • +

  • backlinks, a list of Backlink objects pointing to the original websites +and image URLs. List items are instances of dict, (Backlink):

    +
      +

    • url, the image URL to the image.

      • +

    • backlink, the original website URL.

      • +

    • crawl_date, the date the image was crawled.

      • +
    +
    • +
+
+ +
+
+searx.engines.tineye.request(query, params)[source]
+

Build TinEye HTTP request using search_urls of a engine_type.

+
+ +
+
+searx.engines.tineye.response(resp)[source]
+

Parse HTTP response from TinEye.

+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.engines.yahoo.html b/src/searx.engines.yahoo.html new file mode 100644 index 00000000..df21bd9c --- /dev/null +++ b/src/searx.engines.yahoo.html @@ -0,0 +1,182 @@ + + + + + + + + + Yahoo Engine — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Yahoo Engine

+

Yahoo Search (Web)

+

Languages are supported by mapping the language to a domain. If domain is not +found in lang2domain URL <lang>.search.yahoo.com is used.

+
+
+searx.engines.yahoo.lang2domain = {'bg': 'search.yahoo.com', 'cs': 'search.yahoo.com', 'da': 'search.yahoo.com', 'el': 'search.yahoo.com', 'en': 'search.yahoo.com', 'et': 'search.yahoo.com', 'he': 'search.yahoo.com', 'hr': 'search.yahoo.com', 'ja': 'search.yahoo.com', 'ko': 'search.yahoo.com', 'sk': 'search.yahoo.com', 'sl': 'search.yahoo.com', 'zh_chs': 'hk.search.yahoo.com', 'zh_cht': 'tw.search.yahoo.com'}
+

Map language to domain

+
+ +
+
+searx.engines.yahoo.parse_url(url_string)[source]
+

remove yahoo-specific tracking-url

+
+ +
+
+searx.engines.yahoo.request(query, params)[source]
+

build request

+
+ +
+
+searx.engines.yahoo.response(resp)[source]
+

parse response

+
+ +
+
+searx.engines.yahoo.supported_languages_url = 'https://search.yahoo.com/preferences/languages'
+

Supported languages are read from Yahoo preference page.

+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.infopage.html b/src/searx.infopage.html new file mode 100644 index 00000000..142cfbd0 --- /dev/null +++ b/src/searx.infopage.html @@ -0,0 +1,257 @@ + + + + + + + + + Online /info — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Online /info

+

Render SearXNG instance documentation.

+

Usage in a Flask app route:

+
from searx import infopage
+
+_INFO_PAGES = infopage.InfoPageSet(infopage.MistletoePage)
+
+@app.route('/info/<pagename>', methods=['GET'])
+def info(pagename):
+
+    locale = request.preferences.get_value('locale')
+    page = _INFO_PAGES.get_page(pagename, locale)
+
+
+
+
+class searx.infopage.InfoPage(fname)[source]
+

A page of the online documentation.

+
+
+property content
+

Content of the page (rendered in a Jinja conntext)

+
+ +
+
+get_ctx()[source]
+

Jinja context to render InfoPage.content

+
+ +
+
+property html
+

Render Markdown (CommonMark) to HTML by using markdown-it-py.

+
+ +
+
+property raw_content
+

Raw content of the page (without any jinja rendering)

+
+ +
+
+property title
+

Title of the content (without any markup)

+
+ +
+ +
+
+class searx.infopage.InfoPageSet(page_class: Optional[Type[InfoPage]] = None, info_folder: Optional[str] = None)[source]
+

Cached rendering of the online documentation a SearXNG instance has.

+
+
Parameters:
+
    +

  • page_class (InfoPage) – render online documentation by InfoPage parser.

    • +

  • info_folder (str) – information directory

    • +
+
+
+
+
+folder: str
+

location of the Markdwon files

+
+ +
+
+get_page(pagename: str, locale: Optional[str] = None)[source]
+

Return pagename instance of InfoPage

+
+
Parameters:
+
    +

  • pagename (str) – name of the page, a value from InfoPageSet.toc

    • +

  • locale (str) – language of the page, e.g. en, zh_Hans_CN +(default: InfoPageSet.i18n_origin)

    • +
+
+
+
+ +
+
+iter_pages(locale: Optional[str] = None, fallback_to_default=False)[source]
+

Iterate over all pages of the TOC

+
+ +
+
+locale_default: str
+

default language

+
+ +
+
+locales: List[str]
+

list of supported languages (aka locales)

+
+ +
+
+toc: List[str]
+

list of articles in the online documentation

+
+ +
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.locales.html b/src/searx.locales.html new file mode 100644 index 00000000..aca64fdc --- /dev/null +++ b/src/searx.locales.html @@ -0,0 +1,261 @@ + + + + + + + + + Locales — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Locales

+

Initialize LOCALE_NAMES, RTL_LOCALES.

+
+
+searx.locales.ADDITIONAL_TRANSLATIONS = {'dv': 'ދިވެހި (Dhivehi)', 'oc': 'Occitan', 'pap': 'Papiamento', 'szl': 'Ślōnski (Silesian)'}
+

Additional languages SearXNG has translations for but not supported by +python-babel (see locales_initialize).

+
+ +
+
+searx.locales.LOCALE_BEST_MATCH = {'dv': 'si', 'nl-BE': 'nl', 'oc': 'fr-FR', 'pap': 'pt-BR', 'szl': 'pl', 'zh-HK': 'zh-Hant-TW'}
+

Map a locale we do not have a translations for to a locale we have a +translation for. By example: use Taiwan version of the translation for Hong +Kong.

+
+ +
+
+searx.locales.LOCALE_NAMES
+

Mapping of locales and their description. Locales e.g. ‘fr’ or ‘pt-BR’ (see +locales_initialize).

+
+
+
+ +
+
+searx.locales.RTL_LOCALES: Set[str] = {'ar', 'fa-IR', 'he'}
+

List of Right-To-Left locales e.g. ‘he’ or ‘fa-IR’ (see +locales_initialize).

+
+ +
+
+searx.locales.get_engine_locale(searxng_locale, engine_locales, default=None)[source]
+

Return engine’s language (aka locale) string that best fits to argument +searxng_locale.

+

Argument engine_locales is a python dict that maps SearXNG locales to +corresponding engine locales:

+
<engine>: {
+    # SearXNG string : engine-string
+    'ca-ES'          : 'ca_ES',
+    'fr-BE'          : 'fr_BE',
+    'fr-CA'          : 'fr_CA',
+    'fr-CH'          : 'fr_CH',
+    'fr'             : 'fr_FR',
+    ...
+    'pl-PL'          : 'pl_PL',
+    'pt-PT'          : 'pt_PT'
+}
+
+
+
+

Hint

+

The SearXNG locale string has to be known by babel!

+
+

If there is no direct 1:1 mapping, this functions tries to narrow down +engine’s language (locale). If no value can be determined by these +approximation attempts the default value is returned.

+

Assumptions:

+
    +

  1. When user select a language the results should be optimized according to +the selected language.

    2. +

  2. When user select a language and a territory the results should be +optimized with first priority on terrirtory and second on language.

    3. +
+

First approximation rule (by territory):

+
+

When the user selects a locale with terrirtory (and a language), the +territory has priority over the language. If any of the offical languages +in the terrirtory is supported by the engine (engine_locales) it will +be used.

+
+

Second approximation rule (by language):

+
+

If “First approximation rule” brings no result or the user selects only a +language without a terrirtory. Check in which territories the language +has an offical status and if one of these territories is supported by the +engine.

+
+
+ +
+
+searx.locales.get_locale_descr(locale, locale_name)[source]
+

Get locale name e.g. ‘Français - fr’ or ‘Português (Brasil) - pt-BR’

+
+
Parameters:
+
    +

  • locale – instance of Locale

    • +

  • locale_name – name e.g. ‘fr’ or ‘pt_BR’ (delimiter is underscore)

    • +
+
+
+
+ +
+
+searx.locales.get_translations()[source]
+

Monkey patch of flask_babel.get_translations

+
+ +
+
+searx.locales.locales_initialize(directory=None)[source]
+

Initialize locales environment of the SearXNG session.

+ +
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.plugins.autodetect_search_language.html b/src/searx.plugins.autodetect_search_language.html new file mode 100644 index 00000000..6552455a --- /dev/null +++ b/src/searx.plugins.autodetect_search_language.html @@ -0,0 +1,208 @@ + + + + + + + + + Search language plugin — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Search language plugin

+

Plugin to detect the search language from the search query.

+

The language detection is done by using the fastText library (python +fasttext). fastText distributes the language identification model, for +reference:

+ +

The language identification model support the language codes (ISO-639-3):

+
af als am an ar arz as ast av az azb ba bar bcl be bg bh bn bo bpy br bs bxr
+ca cbk ce ceb ckb co cs cv cy da de diq dsb dty dv el eml en eo es et eu fa
+fi fr frr fy ga gd gl gn gom gu gv he hi hif hr hsb ht hu hy ia id ie ilo io
+is it ja jbo jv ka kk km kn ko krc ku kv kw ky la lb lez li lmo lo lrc lt lv
+mai mg mhr min mk ml mn mr mrj ms mt mwl my myv mzn nah nap nds ne new nl nn
+no oc or os pa pam pfl pl pms pnb ps pt qu rm ro ru rue sa sah sc scn sco sd
+sh si sk sl so sq sr su sv sw ta te tg th tk tl tr tt tyv ug uk ur uz vec vep
+vi vls vo wa war wuu xal xmf yi yo yue zh
+
+
+

The language identification model is harmonized with the SearXNG’s language +(locale) model. General conditions of SearXNG’s locale model are:

+
    +

  1. SearXNG’s locale of a query is passed to the +searx.locales.get_engine_locale to get a language and/or region +code that is used by an engine.

    2. +

  2. SearXNG and most of the engines do not support all the languages from +language model and there might be also a discrepancy in the ISO-639-3 and +ISO-639-2 handling (searx.locales.get_engine_locale). Further +more, in SearXNG the locales like zh-TH (zh-CN) are mapped to +zh_Hant (zh_Hans).

    3. +
+

Conclusion: This plugin does only auto-detect the languages a user can select in +the language menu (supported_langs).

+

SearXNG’s locale of a query comes from (highest wins):

+
    +

  1. The Accept-Language header from user’s HTTP client.

    2. +

  2. The user select a locale in the preferences.

    3. +

  3. The user select a locale from the menu in the query form (e.g. :zh-TW)

    4. +

  4. This plugin is activated in the preferences and the locale (only the language +code / none region code) comes from the fastText’s language detection.

    5. +
+

Conclusion: There is a conflict between the language selected by the user and +the language from language detection of this plugin. For example, the user +explicitly selects the German locale via the search syntax to search for a term +that is identified as an English term (try :de-DE thermomix, for example).

+
+

Hint

+

To SearXNG maintainers; please take into account: under some circumstances +the auto-detection of the language of this plugin could be detrimental to +users expectations. Its not recommended to activate this plugin by +default. It should always be the user’s decision whether to activate this +plugin or not.

+
+
+
+searx.plugins.autodetect_search_language.supported_langs = {'af', 'ar', 'be', 'bg', 'ca', 'cs', 'da', 'de', 'el', 'en', 'es', 'et', 'fa', 'fi', 'fil', 'fr', 'he', 'hi', 'hr', 'hu', 'id', 'is', 'it', 'ja', 'ko', 'lt', 'lv', 'nl', 'no', 'pl', 'pt', 'ro', 'ru', 'sk', 'sl', 'sr', 'sv', 'sw', 'th', 'tr', 'uk', 'vi', 'zh'}
+

Languages supported by most searxng engines (searx.languages.language_codes).

+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.plugins.limiter.html b/src/searx.plugins.limiter.html new file mode 100644 index 00000000..3424c4f7 --- /dev/null +++ b/src/searx.plugins.limiter.html @@ -0,0 +1,162 @@ + + + + + + + + + Limiter Plugin — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Limiter Plugin

+ +

Some bot protection / rate limitation

+

To monitor rate limits and protect privacy the IP addresses are getting stored +with a hash so the limiter plugin knows who to block. A redis database is +needed to store the hash values.

+

Enable the plugin in settings.yml:

+
    +

  • server.limiter: true

    • +

  • redis.url: ... check the value, see redis:

    • +
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.plugins.tor_check.html b/src/searx.plugins.tor_check.html new file mode 100644 index 00000000..09d44fd2 --- /dev/null +++ b/src/searx.plugins.tor_check.html @@ -0,0 +1,189 @@ + + + + + + + + + Tor check plugin — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Tor check plugin

+

A plugin to check if the ip address of the request is a TOR exit node if the +user searches for tor-check. It fetches the tor exit node list from +https://check.torproject.org/exit-addresses and parses all the IPs into a list, +then checks if the user’s IP address is in it.

+

Enable in settings.yml:

+
enabled_plugins:
+  ..
+  - 'Tor check plugin'
+
+
+
+
+searx.plugins.tor_check.description = 'This plugin checks if the address of the request is a TOR exit node, and informs the user if it is, like check.torproject.org but from searxng.'
+

Translated description of the plugin.

+
+ +
+
+searx.plugins.tor_check.name = 'Tor check plugin'
+

Translated name of the plugin

+
+ +
+
+searx.plugins.tor_check.preference_section = 'query'
+

The preference section where the plugin is shown.

+
+ +
+
+searx.plugins.tor_check.query_examples = ''
+

Query examples shown in the preferences.

+
+ +
+
+searx.plugins.tor_check.query_keywords = ['tor-check']
+

Query keywords shown in the preferences.

+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.redisdb.html b/src/searx.redisdb.html new file mode 100644 index 00000000..5e0ec7ec --- /dev/null +++ b/src/searx.redisdb.html @@ -0,0 +1,169 @@ + + + + + + + + + Redis DB — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Redis DB

+

Implementation of the redis client (redis-py).

+

This implementation uses the redis: setup from settings.yml. +A redis DB connect can be tested by:

+
>>> from searx import redisdb
+>>> redisdb.initialize()
+True
+>>> db = redisdb.client()
+>>> db.set("foo", "bar")
+True
+>>> db.get("foo")
+b'bar'
+>>>
+
+
+
+
+searx.redisdb.OLD_REDIS_URL_DEFAULT_URL = 'unix:///usr/local/searxng-redis/run/redis.sock?db=0'
+

This was the default Redis URL in settings.yml.

+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.redislib.html b/src/searx.redislib.html new file mode 100644 index 00000000..28b9e099 --- /dev/null +++ b/src/searx.redislib.html @@ -0,0 +1,295 @@ + + + + + + + + + Redis Library — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Redis Library

+

A collection of convenient functions and redis/lua scripts.

+

This code was partial inspired by the Bullet-Proofing Lua Scripts in RedisPy +article.

+
+
+searx.redislib.LUA_SCRIPT_STORAGE = {}
+

A global dictionary to cache client’s Script objects, used by +lua_script_storage

+
+ +
+
+searx.redislib.drop_counter(client, name)[source]
+

Drop counter with redis key SearXNG_counter_<name>

+

The replacement <name> is a secret hash of the value from argument +name (see incr_counter() and incr_sliding_window()).

+
+ +
+
+searx.redislib.incr_counter(client, name: str, limit: int = 0, expire: int = 0)[source]
+

Increment a counter and return the new value.

+

If counter with redis key SearXNG_counter_<name> does not exists it is +created with initial value 1 returned. The replacement <name> is a +secret hash of the value from argument name (see +secret_hash()).

+

The implementation of the redis counter is the lua script from string +INCR_COUNTER.

+
+
Parameters:
+
    +

  • name (str) – name of the counter

    • +

  • expire (int / see EXPIRE) – live-time of the counter in seconds (default None means +infinite).

    • +

  • limit (int / limit is 2^64 see INCR) – limit where the counter stops to increment (default None)

    • +
+
+
Returns:
+

value of the incremented counter

+
+
+

A simple demo of a counter with expire time and limit:

+
>>> for i in range(6):
+...   i, incr_counter(client, "foo", 3, 5) # max 3, duration 5 sec
+...   time.sleep(1) # from the third call on max has been reached
+...
+(0, 1)
+(1, 2)
+(2, 3)
+(3, 3)
+(4, 3)
+(5, 1)
+
+
+
+ +
+
+searx.redislib.incr_sliding_window(client, name: str, duration: int)[source]
+

Increment a sliding-window counter and return the new value.

+

If counter with redis key SearXNG_counter_<name> does not exists it is +created with initial value 1 returned. The replacement <name> is a +secret hash of the value from argument name (see +secret_hash()).

+
+
Parameters:
+
    +

  • name (str) – name of the counter

    • +

  • duration – live-time of the sliding window in seconds

    • +
+
+
Typeduration:
+

int

+
+
Returns:
+

value of the incremented counter

+
+
+

The implementation of the redis counter is the lua script from string +INCR_SLIDING_WINDOW. The lua script uses sorted sets in Redis +to implement a sliding window for the redis key SearXNG_counter_<name> +(ZADD). The current TIME is used to score the items in the sorted set and +the time window is moved by removing items with a score lower current time +minus duration time (ZREMRANGEBYSCORE).

+

The EXPIRE time (the duration of the sliding window) is refreshed on each +call (incrementation) and if there is no call in this duration, the sorted +set expires from the redis DB.

+

The return value is the amount of items in the sorted set (ZCOUNT), what +means the number of calls in the sliding window.

+

A simple demo of the sliding window:

+
>>> for i in range(5):
+...   incr_sliding_window(client, "foo", 3) # duration 3 sec
+...   time.sleep(1) # from the third call (second) on the window is moved
+...
+1
+2
+3
+3
+3
+>>> time.sleep(3)  # wait until expire
+>>> incr_sliding_window(client, "foo", 3)
+1
+
+
+
+ +
+
+searx.redislib.lua_script_storage(client, script)[source]
+

Returns a redis Script instance.

+

Due to performance reason the Script object is instantiated only once +for a client (client.register_script(..)) and is cached in +LUA_SCRIPT_STORAGE.

+
+ +
+
+searx.redislib.purge_by_prefix(client, prefix: str = 'SearXNG_')[source]
+

Purge all keys with prefix from database.

+

Queries all keys in the database by the given prefix and set expire time to +zero. The default prefix will drop all keys which has been set by SearXNG +(drops SearXNG schema entirely from database).

+

The implementation is the lua script from string PURGE_BY_PREFIX. +The lua script uses EXPIRE instead of DEL: if there are a lot keys to +delete and/or their values are big, DEL could take more time and blocks +the command loop while EXPIRE turns back immediate.

+
+
Parameters:
+

prefix – prefix of the key to delete (default: SearXNG_)

+
+
+
+ +
+
+searx.redislib.secret_hash(name: str)[source]
+

Creates a hash of the name.

+

Combines argument name with the secret_key from server:. This function can be used to get a more anonymised name of a Redis +KEY.

+
+
Parameters:
+

name (str) – the name to create a secret hash for

+
+
+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.search.html b/src/searx.search.html new file mode 100644 index 00000000..f2917771 --- /dev/null +++ b/src/searx.search.html @@ -0,0 +1,213 @@ + + + + + + + + + Search — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ + + + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/src/searx.utils.html b/src/searx.utils.html new file mode 100644 index 00000000..7a186a34 --- /dev/null +++ b/src/searx.utils.html @@ -0,0 +1,493 @@ + + + + + + + + + Utility functions for the engines — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

Utility functions for the engines

+

Utility functions for the engines

+
+
+searx.utils.convert_str_to_int(number_str: str) → int[source]
+

Convert number_str to int or 0 if number_str is not a number.

+
+ +
+
+searx.utils.detect_language(text: str, threshold: float = 0.3, min_probability: float = 0.5) → Optional[str][source]
+

https://fasttext.cc/docs/en/language-identification.html

+
+ +
+
+searx.utils.dict_subset(dictionary: MutableMapping, properties: Set[str]) → Dict[source]
+

Extract a subset of a dict

+
+
Examples:
>>> dict_subset({'A': 'a', 'B': 'b', 'C': 'c'}, ['A', 'C'])
+{'A': 'a', 'C': 'c'}
+>>> >> dict_subset({'A': 'a', 'B': 'b', 'C': 'c'}, ['A', 'D'])
+{'A': 'a'}
+
+
+
+
+
+ +
+
+searx.utils.ecma_unescape(string: str) → str[source]
+

Python implementation of the unescape javascript function

+

https://www.ecma-international.org/ecma-262/6.0/#sec-unescape-string +https://developer.mozilla.org/fr/docs/Web/JavaScript/Reference/Objets_globaux/unescape

+
+
Examples:
>>> ecma_unescape('%u5409')
+'吉'
+>>> ecma_unescape('%20')
+' '
+>>> ecma_unescape('%F3')
+'ó'
+
+
+
+
+
+ +
+
+searx.utils.eval_xpath(element: ElementBase, xpath_spec: Union[str, XPath])[source]
+

Equivalent of element.xpath(xpath_str) but compile xpath_str once for all. +See https://lxml.de/xpathxslt.html#xpath-return-values

+
+
Args:
    +

  • element (ElementBase): [description]

    • +

  • xpath_spec (str|lxml.etree.XPath): XPath as a str or lxml.etree.XPath

    • +
+
+
Returns:
    +

  • result (bool, float, list, str): Results.

    • +
+
+
Raises:
    +

  • TypeError: Raise when xpath_spec is neither a str nor a lxml.etree.XPath

    • +

  • SearxXPathSyntaxException: Raise when there is a syntax error in the XPath

    • +

  • SearxEngineXPathException: Raise when the XPath can’t be evaluated.

    • +
+
+
+
+ +
+
+searx.utils.eval_xpath_getindex(elements: ~lxml.etree.ElementBase, xpath_spec: ~typing.Union[str, ~lxml.etree.XPath], index: int, default=<searx.utils._NotSetClass object>)[source]
+

Call eval_xpath_list then get one element using the index parameter. +If the index does not exist, either aise an exception is default is not set, +other return the default value (can be None).

+
+
Args:
    +

  • elements (ElementBase): lxml element to apply the xpath.

    • +

  • xpath_spec (str|lxml.etree.XPath): XPath as a str or lxml.etree.XPath.

    • +

  • index (int): index to get

    • +

  • default (Object, optional): Defaults if index doesn’t exist.

    • +
+
+
Raises:
    +

  • TypeError: Raise when xpath_spec is neither a str nor a lxml.etree.XPath

    • +

  • SearxXPathSyntaxException: Raise when there is a syntax error in the XPath

    • +

  • SearxEngineXPathException: if the index is not found. Also see eval_xpath.

    • +
+
+
Returns:
    +

  • result (bool, float, list, str): Results.

    • +
+
+
+
+ +
+
+searx.utils.eval_xpath_list(element: ElementBase, xpath_spec: Union[str, XPath], min_len: Optional[int] = None)[source]
+

Same as eval_xpath, check if the result is a list

+
+
Args:
    +

  • element (ElementBase): [description]

    • +

  • xpath_spec (str|lxml.etree.XPath): XPath as a str or lxml.etree.XPath

    • +

  • min_len (int, optional): [description]. Defaults to None.

    • +
+
+
Raises:
    +

  • TypeError: Raise when xpath_spec is neither a str nor a lxml.etree.XPath

    • +

  • SearxXPathSyntaxException: Raise when there is a syntax error in the XPath

    • +

  • SearxEngineXPathException: raise if the result is not a list

    • +
+
+
Returns:
    +

  • result (bool, float, list, str): Results.

    • +
+
+
+
+ +
+
+searx.utils.extract_text(xpath_results, allow_none: bool = False) → Optional[str][source]
+

Extract text from a lxml result

+
    +

  • if xpath_results is list, extract the text from each result and concat the list

    • +

  • if xpath_results is a xml element, extract all the text node from it +( text_content() method from lxml )

    • +

  • if xpath_results is a string element, then it’s already done

    • +
+
+ +
+
+searx.utils.extract_url(xpath_results, base_url) → str[source]
+

Extract and normalize URL from lxml Element

+
+
Args:
    +

  • xpath_results (Union[List[html.HtmlElement], html.HtmlElement]): lxml Element(s)

    • +

  • base_url (str): Base URL

    • +
+
+
Example:
>>> def f(s, search_url):
+>>>    return searx.utils.extract_url(html.fromstring(s), search_url)
+>>> f('<span id="42">https://example.com</span>', 'http://example.com/')
+'https://example.com/'
+>>> f('https://example.com', 'http://example.com/')
+'https://example.com/'
+>>> f('//example.com', 'http://example.com/')
+'http://example.com/'
+>>> f('//example.com', 'https://example.com/')
+'https://example.com/'
+>>> f('/path?a=1', 'https://example.com')
+'https://example.com/path?a=1'
+>>> f('', 'https://example.com')
+raise lxml.etree.ParserError
+>>> searx.utils.extract_url([], 'https://example.com')
+raise ValueError
+
+
+
+
Raises:
    +

  • ValueError

    • +

  • lxml.etree.ParserError

    • +
+
+
Returns:
    +

  • str: normalized URL

    • +
+
+
+
+ +
+
+searx.utils.gen_useragent(os_string: Optional[str] = None) → str[source]
+

Return a random browser User Agent

+

See searx/data/useragents.json

+
+ +
+
+searx.utils.get_engine_from_settings(name: str) → Dict[source]
+

Return engine configuration from settings.yml of a given engine name

+
+ +
+
+searx.utils.get_torrent_size(filesize: str, filesize_multiplier: str) → Optional[int][source]
+
+
Args:
    +

  • filesize (str): size

    • +

  • filesize_multiplier (str): TB, GB, …. TiB, GiB…

    • +
+
+
Returns:
    +

  • int: number of bytes

    • +
+
+
Example:
>>> get_torrent_size('5', 'GB')
+5368709120
+>>> get_torrent_size('3.14', 'MiB')
+3140000
+
+
+
+
+
+ +
+
+searx.utils.get_xpath(xpath_spec: Union[str, XPath]) → XPath[source]
+

Return cached compiled XPath

+

There is no thread lock. +Worst case scenario, xpath_str is compiled more than one time.

+
+
Args:
    +

  • xpath_spec (str|lxml.etree.XPath): XPath as a str or lxml.etree.XPath

    • +
+
+
Returns:
    +

  • result (bool, float, list, str): Results.

    • +
+
+
Raises:
    +

  • TypeError: Raise when xpath_spec is neither a str nor a lxml.etree.XPath

    • +

  • SearxXPathSyntaxException: Raise when there is a syntax error in the XPath

    • +
+
+
+
+ +
+
+searx.utils.html_to_text(html_str: str) → str[source]
+

Extract text from a HTML string

+
+
Args:
    +

  • html_str (str): string HTML

    • +
+
+
Returns:
    +

  • str: extracted text

    • +
+
+
Examples:
>>> html_to_text('Example <span id="42">#2</span>')
+'Example #2'
+
+
+
>>> html_to_text('<style>.span { color: red; }</style><span>Example</span>')
+'Example'
+
+
+
+
+
+ +
+
+searx.utils.int_or_zero(num: Union[List[str], str]) → int[source]
+

Convert num to int or 0. num can be either a str or a list. +If num is a list, the first element is converted to int (or return 0 if the list is empty). +If num is a str, see convert_str_to_int

+
+ +
+
+searx.utils.is_valid_lang(lang) → Optional[Tuple[bool, str, str]][source]
+

Return language code and name if lang describe a language.

+
+
Examples:
>>> is_valid_lang('zz')
+None
+>>> is_valid_lang('uk')
+(True, 'uk', 'ukrainian')
+>>> is_valid_lang(b'uk')
+(True, 'uk', 'ukrainian')
+>>> is_valid_lang('en')
+(True, 'en', 'english')
+>>> searx.utils.is_valid_lang('Español')
+(True, 'es', 'spanish')
+>>> searx.utils.is_valid_lang('Spanish')
+(True, 'es', 'spanish')
+
+
+
+
+
+ +
+
+searx.utils.match_language(locale_code, lang_list=[], custom_aliases={}, fallback: Optional[str] = 'en-US') → Optional[str][source]
+

get the language code from lang_list that best matches locale_code

+
+ +
+
+searx.utils.normalize_url(url: str, base_url: str) → str[source]
+

Normalize URL: add protocol, join URL with base_url, add trailing slash if there is no path

+
+
Args:
    +

  • url (str): Relative URL

    • +

  • base_url (str): Base URL, it must be an absolute URL.

    • +
+
+
Example:
>>> normalize_url('https://example.com', 'http://example.com/')
+'https://example.com/'
+>>> normalize_url('//example.com', 'http://example.com/')
+'http://example.com/'
+>>> normalize_url('//example.com', 'https://example.com/')
+'https://example.com/'
+>>> normalize_url('/path?a=1', 'https://example.com')
+'https://example.com/path?a=1'
+>>> normalize_url('', 'https://example.com')
+'https://example.com/'
+>>> normalize_url('/test', '/path')
+raise ValueError
+
+
+
+
Raises:
    +

  • lxml.etree.ParserError

    • +
+
+
Returns:
    +

  • str: normalized URL

    • +
+
+
+
+ +
+
+searx.utils.searx_useragent() → str[source]
+

Return the searx User Agent

+
+ +
+
+searx.utils.to_string(obj: Any) → str[source]
+

Convert obj to its string representation.

+
+ +
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/user/index.html b/user/index.html new file mode 100644 index 00000000..877682b0 --- /dev/null +++ b/user/index.html @@ -0,0 +1,226 @@ + + + + + + + + + User information — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

User information

+ +
+

Search syntax

+

SearXNG comes with a search syntax by with you can modify the categories, +engines, languages and more. See the preferences for +the list of engines, categories and languages.

+
+

! select engine and category

+

To set category and/or engine names use a ! prefix. To give a few examples:

+
    +

  • search in Wikipedia for paris

    +
      +

    • !wp paris

      • +

    • !wikipedia paris

      • +
    +
    • +

  • search in category map for paris

    +
      +

    • !map paris

      • +
    +
    • +

  • image search

    +
      +

    • !images Wau Holland

      • +
    +
    • +
+

Abbreviations of the engines and languages are also accepted. Engine/category +modifiers are chain able and inclusive. E.g. with !map !ddg !wp paris search in map category and DuckDuckGo and Wikipedia for paris.

+
+
+

: select language

+

To select language filter use a : prefix. To give an example:

+
    +

  • search Wikipedia by a custom language

    +
      +

    • :fr !wp Wau Holland

      • +
    +
    • +
+
+
+

!! external bangs

+

SearXNG supports the external bangs from DuckDuckGo. To directly jump to a +external search page use the !! prefix. To give an example:

+
    +

  • search Wikipedia by a custom language

    +
      +

    • !!wfr Wau Holland

      • +
    +
    • +
+

Please note, your search will be performed directly in the external search +engine, SearXNG cannot protect your privacy on this.

+
+
+

Special Queries

+

In the preferences page you find keywords for +special queries. To give a few examples:

+
    +

  • generate a random UUID

    +
      +

    • random uuid

      • +
    +
    • +

  • find the average

    +
      +

    • avg 123 548 2.04 24.2

      • +
    +
    • +

  • show user agent of your browser (needs to be activated)

    +
      +

    • user-agent

      • +
    +
    • +

  • convert strings to different hash digests (needs to be activated)

    +
      +

    • md5 lorem ipsum

      • +

    • sha512 lorem ipsum

      • +
    +
    • +
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/utils/index.html b/utils/index.html new file mode 100644 index 00000000..ae4d9f8a --- /dev/null +++ b/utils/index.html @@ -0,0 +1,167 @@ + + + + + + + + + DevOps tooling box — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

DevOps tooling box

+

In the folder git://utils/ we maintain some tools useful for administrators +and developers.

+
+

Contents

+ +
+
+

Common command environments

+

The scripts in our tooling box often dispose of common environments:

+
+
FORCE_TIMEOUTenvironment

Sets timeout for interactive prompts. If you want to run a script in batch +job, with defaults choices, set FORCE_TIMEOUT=0. By example; to install a +SearXNG server and nginx proxy on all containers of the SearXNG suite use:

+
sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install all
+sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
+
+
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/utils/lxc.sh.html b/utils/lxc.sh.html new file mode 100644 index 00000000..4a76d3dd --- /dev/null +++ b/utils/lxc.sh.html @@ -0,0 +1,404 @@ + + + + + + + + + utils/lxc.sh — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

utils/lxc.sh

+ +

With the use of Linux Containers (LXC) we can scale our tasks over a stack of +containers, what we call the: lxc suite. The SearXNG suite +(lxc-searxng.env) is loaded by default, every time +you start the lxc.sh script (you do not need to care about).

+

Before you can start with containers, you need to install and initiate LXD +once:

+
$ snap install lxd
+$ lxd init --auto
+
+
+

To make use of the containers from the SearXNG suite, you have to build the +LXC suite containers initial. But be warned, this might +take some time:

+
$ sudo -H ./utils/lxc.sh build
+
+
+

A cup of coffee later, your LXC suite is build up and you can run whatever task +you want / in a selected or even in all LXC suite containers.

+
+

Hint

+

If you see any problems with the internet connectivity of your +containers read section Internet Connectivity & Docker.

+
+

If you do not want to build all containers, you can build just one:

+
$ sudo -H ./utils/lxc.sh build searxng-archlinux
+
+
+

Good to know …

+

Each container shares the root folder of the repository and the command +utils/lxc.sh cmd handles relative path names transparent, compare output +of:

+
$ sudo -H ./utils/lxc.sh cmd -- ls -la Makefile
+...
+
+
+

In the containers, you can run what ever you want, e.g. to start a bash use:

+
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
+INFO:  [searxng-archlinux] bash
+[root@searxng-archlinux SearXNG]#
+
+
+

If there comes the time you want to get rid off all the containers and +clean up local images just type:

+
$ sudo -H ./utils/lxc.sh remove
+$ sudo -H ./utils/lxc.sh remove images
+
+
+
+

Internet Connectivity & Docker

+ +

There is a conflict in the iptables setup of Docker & LXC. If you have +docker installed, you may find that the internet connectivity of your LXD +containers no longer work.

+

Whenever docker is started (reboot) it sets the iptables policy for the +FORWARD chain to DROP [ref]:

+
$ sudo -H iptables-save | grep FORWARD
+:FORWARD ACCEPT [7048:7851230]
+:FORWARD DROP [7048:7851230]
+
+
+

A handy solution of this problem might be to reset the policy for the +FORWARD chain after the network has been initialized. For this create a +file in the if-up section of the network (/etc/network/if-up.d/iptable) +and insert the following lines:

+
#!/bin/sh
+iptables -F FORWARD
+iptables -P FORWARD ACCEPT
+
+
+

Don’t forget to set the execution bit:

+
sudo chmod ugo+x /etc/network/if-up.d/iptable
+
+
+

Reboot your system and check the iptables rules:

+
$ sudo -H iptables-save | grep FORWARD
+:FORWARD ACCEPT [7048:7851230]
+:FORWARD ACCEPT [7048:7851230]
+
+
+
+
+

Install suite

+

To install the complete SearXNG suite (includes searx, morty & filtron) into all LXC use:

+
$ sudo -H ./utils/lxc.sh install suite
+
+
+

The command above installs a SearXNG suite (see Installation Script). +To install a nginx reverse proxy (or alternatively +use apache):

+
sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
+
+
+

To get the IP (URL) of the SearXNG service in the containers use show suite +command. To test instances from containers just open the URLs in your +WEB-Browser:

+
$ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL
+
+[searxng-ubu2110]      SEARXNG_URL          : http://n.n.n.147/searxng
+[searxng-ubu2004]      SEARXNG_URL          : http://n.n.n.246/searxng
+[searxnggfedora35]     SEARXNG_URL          : http://n.n.n.140/searxng
+[searxng-archlinux]    SEARXNG_URL          : http://n.n.n.165/searxng
+
+
+
+
+

Running commands

+

Inside containers, you can use make or run scripts from the +DevOps tooling box. By example: to setup a Buildhosts and run the +Makefile target test in the archlinux container:

+
sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh install buildhost
+sudo -H ./utils/lxc.sh cmd searxng-archlinux make test
+
+
+
+
+

Setup SearXNG buildhost

+

You can install the SearXNG buildhost environment into one or all containers. +The installation procedure to set up a build host takes its +time. Installation in all containers will take more time (time for another cup +of coffee).:

+
sudo -H ./utils/lxc.sh cmd -- ./utils/searxng.sh install buildhost
+
+
+

To build (live) documentation inside a archlinux container:

+
sudo -H ./utils/lxc.sh cmd searxng-archlinux make docs.clean docs.live
+...
+[I 200331 15:00:42 server:296] Serving on http://0.0.0.0:8080
+
+
+

To get IP of the container and the port number live docs is listening:

+
$ sudo ./utils/lxc.sh show suite | grep docs.live
+...
+[searxng-archlinux]  INFO:  (eth0) docs.live:  http://n.n.n.12:8080/
+
+
+
+
+

Overview

+

The --help output of the script is largely self-explanatory:

+
usage::
+  lxc.sh build        [containers|<name>]
+  lxc.sh copy         [images]
+  lxc.sh remove       [containers|<name>|images]
+  lxc.sh [start|stop] [containers|<name>]
+  lxc.sh show         [images|suite|info|config [<name>]]
+  lxc.sh cmd          [--|<name>] '...'
+  lxc.sh install      [suite|base [<name>]]
+
+build
+  :containers:   build, launch all containers and 'install base' packages
+  :<name>:       build, launch container <name>  and 'install base' packages
+copy:
+  :images:       copy remote images of the suite into local storage
+remove
+  :containers:   delete all 'containers' or only <container-name>
+  :images:       delete local images of the suite
+start/stop
+  :containers:   start/stop all 'containers' from the suite
+  :<name>:       start/stop container <name> from suite
+show
+  :info:         show info of all (or <name>) containers from LXC suite
+  :config:       show config of all (or <name>) containers from the LXC suite
+  :suite:        show services of all (or <name>) containers from the LXC suite
+  :images:       show information of local images
+cmd
+  use single quotes to evaluate in container's bash, e.g.: 'echo $(hostname)'
+  --             run command '...' in all containers of the LXC suite
+  :<name>:       run command '...' in container <name>
+install
+  :base:         prepare LXC; install basic packages
+  :suite:        install LXC searxng suite into all (or <name>) containers
+
+LXC suite: searxng
+  Suite includes installation of SearXNG
+  images:     ubu2004 ubu2110 fedora35 archlinux
+  containers: searxng-ubu2004 searxng-ubu2110 searxng-fedora35 searxng-archlinux
+
+
+
+
+

SearXNG suite

+
# -*- coding: utf-8; mode: sh indent-tabs-mode: nil -*-
+# SPDX-License-Identifier: AGPL-3.0-or-later
+# shellcheck shell=bash
+
+# This file is a setup of a LXC suite.  It is sourced from different context, do
+# not manipulate the environment directly, implement functions and manipulate
+# environment only in subshells.
+
+lxc_set_suite_env() {
+
+    export LXC_SUITE_NAME="searxng"
+
+    # name of https://images.linuxcontainers.org
+    export LINUXCONTAINERS_ORG_NAME="${LINUXCONTAINERS_ORG_NAME:-images}"
+    export LXC_HOST_PREFIX="${LXC_SUITE_NAME:-searx}"
+    export LXC_SUITE=(
+
+        # end of standard support see https://wiki.ubuntu.com/Releases
+        "$LINUXCONTAINERS_ORG_NAME:ubuntu/20.04"  "ubu2004" # April 2025
+        "$LINUXCONTAINERS_ORG_NAME:ubuntu/21.10"  "ubu2110" # July 2027
+
+        # EOL see https://fedoraproject.org/wiki/Releases
+        "$LINUXCONTAINERS_ORG_NAME:fedora/35"     "fedora35"
+
+        # rolling releases see https://www.archlinux.org/releng/releases/
+        "$LINUXCONTAINERS_ORG_NAME:archlinux"     "archlinux"
+    )
+}
+
+lxc_suite_install_info() {
+    (
+        lxc_set_suite_env
+        cat <<EOF
+LXC suite: ${LXC_SUITE_NAME}
+  Suite includes installation of SearXNG
+  images:     ${LOCAL_IMAGES[*]}
+  containers: ${CONTAINERS[*]}
+EOF
+    )
+}
+
+lxc_suite_install() {
+    (
+        lxc_set_suite_env
+        FORCE_TIMEOUT=0
+        export FORCE_TIMEOUT
+        "${LXC_REPO_ROOT}/utils/searxng.sh" install all
+        rst_title "suite installation finished ($(hostname))" part
+        lxc_suite_info
+        echo
+    )
+}
+
+lxc_suite_info() {
+    (
+        lxc_set_suite_env
+        for ip in $(global_IPs) ; do
+            if [[ $ip =~ .*:.* ]]; then
+                info_msg "(${ip%|*}) IPv6:       http://[${ip#*|}]"
+            else
+                # IPv4:
+                # shellcheck disable=SC2034,SC2031
+                info_msg "(${ip%|*}) docs-live:  http://${ip#*|}:8080/"
+            fi
+        done
+        "${LXC_REPO_ROOT}/utils/searxng.sh" searxng.instance.env
+    )
+}
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file diff --git a/utils/searxng.sh.html b/utils/searxng.sh.html new file mode 100644 index 00000000..39b87731 --- /dev/null +++ b/utils/searxng.sh.html @@ -0,0 +1,192 @@ + + + + + + + + + utils/searxng.sh — SearXNG Documentation (2023.1.23+522ba9a1) + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

utils/searxng.sh

+ +

To simplify the installation and maintenance of a SearXNG instance you can use the +script git://utils/searxng.sh.

+
+

Install

+

In most cases you will install SearXNG simply by running the command:

+
sudo -H ./utils/searx.sh install all
+
+
+

The installation is described in chapter Step by step installation.

+
+
+

Overview

+

The --help output of the script is largely self-explanatory:

+
usage:
+  searxng.sh install    [all|user|pyenv|settings|uwsgi|redis|nginx|apache|searxng-src|packages|buildhost]
+  searxng.sh remove     [all|user|pyenv|settings|uwsgi|redis|nginx|apache]
+  searxng.sh instance   [cmd|update|check|localtest|inspect]
+install|remove:
+  all           : complete (de-) installation of the SearXNG service
+  user          : service user 'searxng' (/usr/local/searxng)
+  pyenv         : virtualenv (python) in /usr/local/searxng/searx-pyenv
+  settings      : settings from /etc/searxng/settings.yml
+  uwsgi         : SearXNG's uWSGI app searxng.ini
+  redis         : build & install or remove a local redis server /usr/local/searxng-redis/run/redis.sock
+  nginx         : HTTP site /etc/nginx/default.apps-available/searxng.conf
+  apache        : HTTP site /etc/apache2/sites-available/searxng.conf
+install:
+  searxng-src   : clone https://github.com/searxng/searxng into /usr/local/searxng/searxng-src
+  packages      : installs packages from OS package manager required by SearXNG
+  buildhost     : installs packages from OS package manager required by a SearXNG buildhost
+instance:
+  update        : update SearXNG instance (git fetch + reset & update settings.yml)
+  check         : run checks from utils/searxng_check.py in the active installation
+  inspect       : run some small tests and inspect SearXNG's server status and log
+  get_setting   : get settings value from running SearXNG instance
+  cmd           : run command in SearXNG instance's environment (e.g. bash)
+uWSGI:
+  SEARXNG_UWSGI_SOCKET : /usr/local/searxng/run/socket
+environment /usr/local/searxng/searxng-src/utils/brand.env:
+  GIT_URL              : https://github.com/searxng/searxng
+  GIT_BRANCH           : master
+  SEARXNG_URL          : http://fv-az105-957/searxng
+  SEARXNG_PORT         : 8888
+  SEARXNG_BIND_ADDRESS : 127.0.0.1
+
+
+
+
+ + +
+
+
+
+ + +
+
+ + + + + + \ No newline at end of file