searxng/docs/admin/engines/sql-engines.rst
Markus Heiser 1b3f0c848e [docs] move blog article "SQL engines" to admin/engines/
The article "SQL engines" should be in admin's engine documentation (like the
recoll engine).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:04:38 +02:00

170 lines
5.0 KiB
ReStructuredText

.. _sql engines:
===========
SQL Engines
===========
.. sidebar:: further read
- `SQLite <https://www.sqlite.org/index.html>`_
- `PostgreSQL <https://www.postgresql.org>`_
- `MySQL <https://www.mysql.com>`_
With the *SQL engines* you can bind SQL databases into SearxNG. The following
Relational Database Management System (RDBMS) are supported:
- :ref:`engine sqlite`
- :ref:`engine postgresql`
- :ref:`engine mysql_server`
All of the engines above are just commented out in the :origin:`settings.yml
<searx/settings.yml>`, as you have to set the required attributes for the
engines, e.g. ``database:`` ...
.. code:: yaml
- name: ...
engine: {sqlite|postgresql|mysql_server}
database: ...
result_template: {template_name}
query_str: ...
By default, the engines use the ``key-value`` template for displaying results /
see :origin:`oscar <searx/templates/oscar/result_templates/key-value.html>` &
:origin:`simple <searx/templates/simple/result_templates/key-value.html>`
themes. If you are not satisfied with the original result layout, you can use
your own template, set ``result_template`` attribute to ``{template_name}`` and
place the templates at::
searx/templates/{theme_name}/result_templates/{template_name}
If you do not wish to expose these engines on a public instance, you can still
add them and limit the access by setting ``tokens`` as described in section
:ref:`private engines`.
Configure the engines
=====================
The configuration of the new database engines are similar. You must put a valid
SQL-SELECT query in ``query_str``. At the moment you can only bind at most one
parameter in your query. By setting the attribute ``limit`` you can define how
many results you want from the SQL server. Basically, it is the same as the
``LIMIT`` keyword in SQL.
Please, do not include ``LIMIT`` or ``OFFSET`` in your SQL query as the engines
rely on these keywords during paging. If you want to configure the number of
returned results use the option ``limit``.
.. _engine sqlite:
SQLite
------
.. sidebar:: info
- :origin:`sqlite.py <searx/engines/sqlite.py>`
.. _MediathekView: https://mediathekview.de/
SQLite is a small, fast and reliable SQL database engine. It does not require
any extra dependency. To demonstrate the power of database engines, here is a
more complex example which reads from a MediathekView_ (DE) movie database. For
this example of the SQlite engine download the database:
- https://liste.mediathekview.de/filmliste-v2.db.bz2
and unpack into ``searx/data/filmliste-v2.db``. To search the database use e.g
Query to test: ``!mediathekview concert``
.. code:: yaml
- name: mediathekview
engine: sqlite
disabled: False
categories: general
result_template: default.html
database: searx/data/filmliste-v2.db
query_str: >-
SELECT title || ' (' || time(duration, 'unixepoch') || ')' AS title,
COALESCE( NULLIF(url_video_hd,''), NULLIF(url_video_sd,''), url_video) AS url,
description AS content
FROM film
WHERE title LIKE :wildcard OR description LIKE :wildcard
ORDER BY duration DESC
Extra Dependencies
------------------
For using :ref:`engine postgresql` or :ref:`engine mysql_server` you need to
install additional packages in Python's Virtual Environment of your SearxNG
instance. To switch into the environment (:ref:`searx-src`) you can use
:ref:`searx.sh`::
$ sudo utils/searx.sh shell
(searx-pyenv)$ pip install ...
.. _engine postgresql:
PostgreSQL
----------
.. _psycopg2: https://www.psycopg.org/install
.. sidebar:: info
- :origin:`postgresql.py <searx/engines/postgresql.py>`
- ``pip install`` psycopg2_
PostgreSQL is a powerful and robust open source database. Before configuring
the PostgreSQL engine, you must install the dependency ``psychopg2``. You can
find an example configuration below:
.. code:: yaml
- name: my_database
engine: postgresql
database: my_database
username: searx
password: password
query_str: 'SELECT * from my_table WHERE my_column = %(query)s'
.. _engine mysql_server:
MySQL
-----
.. _mysql-connector-python: https://pypi.org/project/mysql-connector-python
.. sidebar:: info
- :origin:`mysql_server.py <searx/engines/mysql_server.py>`
- ``pip install`` mysql-connector-python_
MySQL is said to be the most popular open source database. Before enabling MySQL
engine, you must install the package ``mysql-connector-python``.
The authentication plugin is configurable by setting ``auth_plugin`` in the
attributes. By default it is set to ``caching_sha2_password``. This is an
example configuration for quering a MySQL server:
.. code:: yaml
- name: my_database
engine: mysql_server
database: my_database
username: searx
password: password
limit: 5
query_str: 'SELECT * from my_table WHERE my_column=%(query)s'
Acknowledgment
==============
This development was sponsored by `Search and Discovery Fund
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.