mirror of https://github.com/searxng/searxng
Merge branch 'master' into bug/oscar-theme
commit
1d86d0054f
@ -0,0 +1,33 @@
|
||||
digraph G {
|
||||
|
||||
node [style=filled, shape=box, fillcolor="#ffffcc", fontname="Sans"];
|
||||
edge [fontname="Sans"];
|
||||
|
||||
browser [label="Browser", shape=Mdiamond];
|
||||
rp [label="Reverse Proxy", href="url to configure reverse proxy"];
|
||||
filtron [label="Filtron", href="https://github.com/asciimoo/filtron"];
|
||||
morty [label="Morty", href="https://github.com/asciimoo/morty"];
|
||||
static [label="Static files", href="url to configure static files"];
|
||||
uwsgi [label="uwsgi", href="url to configure uwsgi"]
|
||||
searx1 [label="Searx #1"];
|
||||
searx2 [label="Searx #2"];
|
||||
searx3 [label="Searx #3"];
|
||||
searx4 [label="Searx #4"];
|
||||
|
||||
browser -> rp [label="HTTPS"]
|
||||
|
||||
subgraph cluster_searx {
|
||||
label = "Searx instance" fontname="Sans";
|
||||
bgcolor="#fafafa";
|
||||
{ rank=same; static rp };
|
||||
rp -> morty [label="optional: images and HTML pages proxy"];
|
||||
rp -> static [label="optional: reverse proxy serves directly static files"];
|
||||
rp -> filtron [label="HTTP"];
|
||||
filtron -> uwsgi [label="HTTP"];
|
||||
uwsgi -> searx1;
|
||||
uwsgi -> searx2;
|
||||
uwsgi -> searx3;
|
||||
uwsgi -> searx4;
|
||||
}
|
||||
|
||||
}
|
@ -0,0 +1,24 @@
|
||||
.. _architecture:
|
||||
|
||||
============
|
||||
Architecture
|
||||
============
|
||||
|
||||
.. sidebar:: Needs work!
|
||||
|
||||
This article needs some work / Searx is a collaborative effort. If you have
|
||||
any contribution, feel welcome to send us your :pull:`PR <../pulls>`, see
|
||||
:ref:`how to contribute`.
|
||||
|
||||
Herein you will find some hints and suggestions about typical architectures of
|
||||
searx infrastructures.
|
||||
|
||||
We start with a contribution from :pull:`@dalf <1776#issuecomment-567917320>`.
|
||||
It shows a *reference* setup for public searx instances.
|
||||
|
||||
.. _arch public:
|
||||
|
||||
.. kernel-figure:: arch_public.dot
|
||||
:alt: arch_public.dot
|
||||
|
||||
Reference architecture of a public searx setup.
|
@ -0,0 +1,103 @@
|
||||
.. _buildhosts:
|
||||
|
||||
==========
|
||||
Buildhosts
|
||||
==========
|
||||
|
||||
.. sidebar:: This article needs some work
|
||||
|
||||
If you have any contribution send us your :pull:`PR <../pulls>`, see
|
||||
:ref:`how to contribute`.
|
||||
|
||||
To get best results from build, its recommend to install additional packages
|
||||
on build hosts.
|
||||
|
||||
.. _docs build:
|
||||
|
||||
Build docs
|
||||
==========
|
||||
|
||||
.. _Graphviz: https://graphviz.gitlab.io
|
||||
.. _ImageMagick: https://www.imagemagick.org
|
||||
.. _XeTeX: https://tug.org/xetex/
|
||||
.. _dvisvgm: https://dvisvgm.de/
|
||||
|
||||
.. sidebar:: Sphinx build needs
|
||||
|
||||
- ImageMagick_
|
||||
- Graphviz_
|
||||
- XeTeX_
|
||||
- dvisvgm_
|
||||
|
||||
Most of the sphinx requirements are installed from :origin:`setup.py` and the
|
||||
docs can be build from scratch with ``make docs``. For better math and image
|
||||
processing additional packages are needed. The XeTeX_ needed not only for PDF
|
||||
creation, its also needed for :ref:`math` when HTML output is build.
|
||||
|
||||
To be able to do :ref:`sphinx:math-support` without CDNs, the math are rendered
|
||||
as images (``sphinx.ext.imgmath`` extension). If your docs build (``make
|
||||
docs``) shows warnings like this::
|
||||
|
||||
WARNING: dot(1) not found, for better output quality install \
|
||||
graphviz from http://www.graphviz.org
|
||||
..
|
||||
WARNING: LaTeX command 'latex' cannot be run (needed for math \
|
||||
display), check the imgmath_latex setting
|
||||
|
||||
you need to install additional packages on your build host, to get better HTML
|
||||
output.
|
||||
|
||||
.. _system requirements:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: Ubuntu / debian
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ sudo apt install graphviz imagemagick texlive-xetex librsvg2-bin
|
||||
|
||||
.. group-tab:: Arch Linux
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ sudo pacman -S graphviz imagemagick texlive-bin extra/librsvg
|
||||
|
||||
.. group-tab:: Fedora / RHEL
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ sudo dnf install graphviz graphviz-gd texlive-xetex-bin librsvg2-tools
|
||||
|
||||
|
||||
For PDF output you also need:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: Ubuntu / debian
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ sudo apt texlive-latex-recommended texlive-extra-utils ttf-dejavu
|
||||
|
||||
.. group-tab:: Arch Linux
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ sudo pacman -S texlive-core texlive-latexextra ttf-dejavu
|
||||
|
||||
.. group-tab:: Fedora / RHEL
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ sudo dnf install \
|
||||
texlive-collection-fontsrecommended texlive-collection-latex \
|
||||
dejavu-sans-fonts dejavu-serif-fonts dejavu-sans-mono-fonts
|
||||
|
||||
.. _system requirements END:
|
||||
|
||||
.. literalinclude:: ../conf.py
|
||||
:language: python
|
||||
:start-after: # sphinx.ext.imgmath setup
|
||||
:end-before: # sphinx.ext.imgmath setup END
|
||||
|
@ -0,0 +1,68 @@
|
||||
.. _engines generic:
|
||||
|
||||
=======
|
||||
engines
|
||||
=======
|
||||
|
||||
.. sidebar:: Further reading ..
|
||||
|
||||
- :ref:`engines generic`
|
||||
- :ref:`configured engines`
|
||||
- :ref:`engine settings`
|
||||
- :ref:`engine file`
|
||||
|
||||
============= =========== ==================== ============
|
||||
:ref:`engine settings` :ref:`engine file`
|
||||
------------------------- ---------------------------------
|
||||
Name (cfg) Categories
|
||||
------------------------- ---------------------------------
|
||||
Engine .. Paging support **P**
|
||||
------------------------- -------------------- ------------
|
||||
Shortcut **S** Language support **L**
|
||||
Timeout **TO** Time range support **TR**
|
||||
Disabled **D** Offline **O**
|
||||
------------- ----------- -------------------- ------------
|
||||
Suspend end **SE**
|
||||
------------- ----------- ---------------------------------
|
||||
Safe search **SS**
|
||||
============= =========== =================================
|
||||
|
||||
Configuration defaults (at built time):
|
||||
|
||||
.. _configured engines:
|
||||
|
||||
.. jinja:: webapp
|
||||
|
||||
.. flat-table:: Engines configured at built time (defaults)
|
||||
:header-rows: 1
|
||||
:stub-columns: 2
|
||||
|
||||
* - Name (cfg)
|
||||
- S
|
||||
- Engine
|
||||
- TO
|
||||
- Categories
|
||||
- P
|
||||
- L
|
||||
- SS
|
||||
- D
|
||||
- TR
|
||||
- O
|
||||
- SE
|
||||
|
||||
{% for name, mod in engines.items() %}
|
||||
|
||||
* - {{name}}
|
||||
- !{{mod.shortcut}}
|
||||
- {{mod.__name__}}
|
||||
- {{mod.timeout}}
|
||||
- {{", ".join(mod.categories)}}
|
||||
- {{(mod.paging and "y") or ""}}
|
||||
- {{(mod.language_support and "y") or ""}}
|
||||
- {{(mod.safesearch and "y") or ""}}
|
||||
- {{(mod.disabled and "y") or ""}}
|
||||
- {{(mod.time_range_support and "y") or ""}}
|
||||
- {{(mod.offline and "y") or ""}}
|
||||
- {{mod.suspend_end_time}}
|
||||
|
||||
{% endfor %}
|
@ -0,0 +1,39 @@
|
||||
.. _plugins generic:
|
||||
|
||||
===============
|
||||
Plugins builtin
|
||||
===============
|
||||
|
||||
.. sidebar:: Further reading ..
|
||||
|
||||
- :ref:`dev plugin`
|
||||
|
||||
Configuration defaults (at built time):
|
||||
|
||||
:DO: Default on
|
||||
|
||||
.. _configured plugins:
|
||||
|
||||
.. jinja:: webapp
|
||||
|
||||
.. flat-table:: Plugins configured at built time (defaults)
|
||||
:header-rows: 1
|
||||
:stub-columns: 1
|
||||
:widths: 3 1 9
|
||||
|
||||
* - Name
|
||||
- DO
|
||||
- Description
|
||||
|
||||
JS & CSS dependencies
|
||||
|
||||
{% for plgin in plugins %}
|
||||
|
||||
* - {{plgin.name}}
|
||||
- {{(plgin.default_on and "y") or ""}}
|
||||
- {{plgin.description}}
|
||||
|
||||
{% for dep in (plgin.js_dependencies + plgin.css_dependencies) %}
|
||||
| ``{{dep}}`` {% endfor %}
|
||||
|
||||
{% endfor %}
|
@ -0,0 +1,6 @@
|
||||
stub col row 1, column, "loremLorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy
|
||||
eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam
|
||||
voluptua."
|
||||
stub col row 1, "At vero eos et accusam et justo duo dolores et ea rebum. Stet clita
|
||||
kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.", column
|
||||
stub col row 1, column, column
|
@ -0,0 +1,3 @@
|
||||
graph G {
|
||||
Hello -- World
|
||||
}
|
@ -0,0 +1,221 @@
|
||||
.. _makefile:
|
||||
|
||||
================
|
||||
Makefile Targets
|
||||
================
|
||||
|
||||
.. _gnu-make: https://www.gnu.org/software/make/manual/make.html#Introduction
|
||||
|
||||
.. sidebar:: build environment
|
||||
|
||||
Before looking deeper at the targets, first read about :ref:`makefile setup`
|
||||
and :ref:`make pyenv`.
|
||||
|
||||
With the aim to simplify development cycles, started with :pull:`1756` a
|
||||
``Makefile`` based boilerplate was added. If you are not familiar with
|
||||
Makefiles, we recommend to read gnu-make_ introduction.
|
||||
|
||||
The usage is simple, just type ``make {target-name}`` to *build* a target.
|
||||
Calling the ``help`` target gives a first overview::
|
||||
|
||||
$ make help
|
||||
test - run developer tests
|
||||
docs - build documentation
|
||||
docs-live - autobuild HTML documentation while editing
|
||||
run - run developer instance
|
||||
install - developer install (./local)
|
||||
uninstall - uninstall (./local)
|
||||
gh-pages - build docs & deploy on gh-pages branch
|
||||
clean - drop builds and environments
|
||||
...
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
|
||||
.. _makefile setup:
|
||||
|
||||
Setup
|
||||
=====
|
||||
|
||||
.. _git stash: https://git-scm.com/docs/git-stash
|
||||
|
||||
The main setup is done in the :origin:`Makefile`::
|
||||
|
||||
export GIT_URL=https://github.com/asciimoo/searx
|
||||
export SEARX_URL=https://searx.me
|
||||
export DOCS_URL=https://asciimoo.github.io/searx
|
||||
|
||||
.. sidebar:: fork & upstream
|
||||
|
||||
Commit changes in your (local) branch, fork or whatever, but do not push them
|
||||
upstream / `git stash`_ is your friend.
|
||||
|
||||
:GIT_URL: Changes this, to point to your searx fork.
|
||||
|
||||
:SEARX_URL: Changes this, to point to your searx instance.
|
||||
|
||||
:DOCS_URL: If you host your own (branded) documentation, change this URL.
|
||||
|
||||
.. _make pyenv:
|
||||
|
||||
Python environment
|
||||
==================
|
||||
|
||||
.. sidebar:: activate environment
|
||||
|
||||
``source ./local/py3/bin/activate``
|
||||
|
||||
With Makefile we do no longer need to build up the virualenv manually (as
|
||||
described in the :ref:`devquickstart` guide). Jump into your git working tree
|
||||
and release a ``make pyenv``:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ cd ~/searx-clone
|
||||
$ make pyenv
|
||||
PYENV usage: source ./local/py3/bin/activate
|
||||
...
|
||||
|
||||
With target ``pyenv`` a development environment (aka virtualenv) was build up in
|
||||
``./local/py3/``. To make a *developer install* of searx (:origin:`setup.py`)
|
||||
into this environment, use make target ``install``:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ make install
|
||||
PYENV usage: source ./local/py3/bin/activate
|
||||
PYENV using virtualenv from ./local/py3
|
||||
PYENV install .
|
||||
|
||||
You have never to think about intermediate targets like ``pyenv`` or
|
||||
``install``, the ``Makefile`` chains them as requisites. Just run your main
|
||||
target.
|
||||
|
||||
.. sidebar:: drop environment
|
||||
|
||||
To get rid of the existing environment before re-build use :ref:`clean target
|
||||
<make clean>` first.
|
||||
|
||||
If you think, something goes wrong with your ./local environment or you change
|
||||
the :origin:`setup.py` file (or the requirements listed in
|
||||
:origin:`requirements-dev.txt` and :origin:`requirements.txt`), you have to call
|
||||
:ref:`make clean`.
|
||||
|
||||
|
||||
.. _make run:
|
||||
|
||||
``make run``
|
||||
============
|
||||
|
||||
To get up a running a developer instance simply call ``make run``. This enables
|
||||
*debug* option in :origin:`searx/settings.yml`, starts a ``./searx/webapp.py``
|
||||
instance, disables *debug* option again and opens the URL in your favorite WEB
|
||||
browser (:man:`xdg-open`):
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ make run
|
||||
PYENV usage: source ./local/py3/bin/activate
|
||||
PYENV install .
|
||||
./local/py3/bin/python ./searx/webapp.py
|
||||
...
|
||||
INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
|
||||
...
|
||||
|
||||
.. _make clean:
|
||||
|
||||
``make clean``
|
||||
==============
|
||||
|
||||
Drop all intermediate files, all builds, but keep sources untouched. Includes
|
||||
target ``pyclean`` which drops ./local environment. Before calling ``make
|
||||
clean`` stop all processes using :ref:`make pyenv`.
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ make clean
|
||||
CLEAN pyclean
|
||||
CLEAN clean
|
||||
|
||||
.. _make docs:
|
||||
|
||||
``make docs docs-live docs-clean``
|
||||
==================================
|
||||
|
||||
We describe the usage of the ``doc*`` targets in the :ref:`How to contribute /
|
||||
Documentation <contrib docs>` section. If you want to edit the documentation
|
||||
read our :ref:`make docs-live` section. If you are working in your own brand,
|
||||
adjust your :ref:`Makefile setup <makefile setup>`.
|
||||
|
||||
|
||||
.. _make gh-pages:
|
||||
|
||||
``make gh-pages``
|
||||
=================
|
||||
|
||||
To deploy on github.io first adjust your :ref:`Makefile setup <makefile
|
||||
setup>`. For any further read :ref:`deploy on github.io`.
|
||||
|
||||
.. _make test:
|
||||
|
||||
``make test``
|
||||
=============
|
||||
|
||||
Runs a series of tests: ``test.pep8``, ``test.unit``, ``test.robot`` and does
|
||||
additional :ref:`pylint checks <make pylint>`. You can run tests selective,
|
||||
e.g.:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ make test.pep8 test.unit
|
||||
. ./local/py3/bin/activate; ./manage.sh pep8_check
|
||||
[!] Running pep8 check
|
||||
. ./local/py3/bin/activate; ./manage.sh unit_tests
|
||||
[!] Running unit tests
|
||||
|
||||
.. _make pylint:
|
||||
|
||||
``make pylint``
|
||||
===============
|
||||
|
||||
.. _Pylint: https://www.pylint.org/
|
||||
|
||||
Before commiting its recommend to do some (more) linting. Pylint_ is known as
|
||||
one of the best source-code, bug and quality checker for the Python programming
|
||||
language. Pylint_ is not yet a quality gate within our searx project (like
|
||||
:ref:`test.pep8 <make test>` it is), but Pylint_ can help to improve code
|
||||
quality anyway. The pylint profile we use at searx project is found in
|
||||
project's root folder :origin:`.pylintrc`.
|
||||
|
||||
Code quality is a ongoing process. Don't try to fix all messages from Pylint,
|
||||
run Pylint and check if your changed lines are bringing up new messages. If so,
|
||||
fix it. By this, code quality gets incremental better and if there comes the
|
||||
day, the linting is balanced out, we might decide to add Pylint as a quality
|
||||
gate.
|
||||
|
||||
|
||||
``make pybuild``
|
||||
================
|
||||
|
||||
.. _PyPi: https://pypi.org/
|
||||
.. _twine: https://twine.readthedocs.io/en/latest/
|
||||
|
||||
Build Python packages in ``./dist/py``.
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ make pybuild
|
||||
...
|
||||
BUILD pybuild
|
||||
running sdist
|
||||
running egg_info
|
||||
...
|
||||
$ ls ./dist/py/
|
||||
searx-0.15.0-py3-none-any.whl searx-0.15.0.tar.gz
|
||||
|
||||
To upload packages to PyPi_, there is also a ``upload-pypi`` target. It needs
|
||||
twine_ to be installed. Since you are not the owner of :pypi:`searx` you will
|
||||
never need the latter.
|
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,10 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!-- originate: https://commons.wikimedia.org/wiki/File:Variable_Resistor.svg -->
|
||||
<svg xmlns="http://www.w3.org/2000/svg"
|
||||
version="1.1" baseProfile="full"
|
||||
width="70px" height="40px" viewBox="0 0 700 400">
|
||||
<line x1="0" y1="200" x2="700" y2="200" stroke="black" stroke-width="20px"/>
|
||||
<rect x="100" y="100" width="500" height="200" fill="white" stroke="black" stroke-width="20px"/>
|
||||
<line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
|
||||
<polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
|
||||
</svg>
|
After Width: | Height: | Size: 580 B |
Loading…
Reference in New Issue