forked from Archives/searxng
b6a55e223c
Not all settings from the 'brand:' section of the YAML files are needed in the
shell scripts. This patch reduce the variables in ./utils/brand.env to what is
needed. The following ('brand:' settings) can be removed from this file:
- ISSUE_URL
- DOCS_URL
- PUBLIC_INSTANCES
- WIKI_URL
Tasks running outside of an *installed instance*, need the following settings
from the YAML configuration:
- GIT_URL <--> brand.git_url
- GIT_BRANCH <--> brand.git_branch
- SEARX_URL <--> server.base_url (aka PUBLIC_URL)
- SEARX_PORT <--> server.port
- SEARX_BIND_ADDRESS <--> server.bind_address
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
257 lines
7.9 KiB
ReStructuredText
257 lines
7.9 KiB
ReStructuredText
.. _makefile:
|
|
|
|
========
|
|
Makefile
|
|
========
|
|
|
|
.. _gnu-make: https://www.gnu.org/software/make/manual/make.html#Introduction
|
|
|
|
.. sidebar:: build environment
|
|
|
|
Before looking deeper at the targets, first read about :ref:`make
|
|
install`.
|
|
|
|
To install system requirements follow :ref:`buildhosts`.
|
|
|
|
All relevant build tasks are implemented in :origin:`manage.sh` and for CI or
|
|
IDE integration a small ``Makefile`` wrapper is available. If you are not
|
|
familiar with Makefiles, we recommend to read gnu-make_ introduction.
|
|
|
|
The usage is simple, just type ``make {target-name}`` to *build* a target.
|
|
Calling the ``help`` target gives a first overview (``make help``):
|
|
|
|
.. program-output:: bash -c "cd ..; make --no-print-directory help"
|
|
|
|
.. contents:: Contents
|
|
:depth: 2
|
|
:local:
|
|
:backlinks: entry
|
|
|
|
.. _make install:
|
|
|
|
Python Environment (``make install``)
|
|
=====================================
|
|
|
|
.. sidebar:: activate environment
|
|
|
|
``source ./local/py3/bin/activate``
|
|
|
|
We do no longer need to build up the virtualenv manually. Jump into your git
|
|
working tree and release a ``make install`` to get a virtualenv with a
|
|
*developer install* of searx (:origin:`setup.py`). ::
|
|
|
|
$ cd ~/searx-clone
|
|
$ make install
|
|
PYENV [virtualenv] installing ./requirements*.txt into local/py3
|
|
...
|
|
PYENV OK
|
|
PYENV [install] pip install -e 'searx[test]'
|
|
...
|
|
Successfully installed argparse-1.4.0 searx
|
|
BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
|
|
BUILDENV INFO:searx:Initialisation done
|
|
BUILDENV build utils/brand.env
|
|
|
|
If you release ``make install`` multiple times the installation will only
|
|
rebuild if the sha256 sum of the *requirement files* fails. With other words:
|
|
the check fails if you edit the requirements listed in
|
|
:origin:`requirements-dev.txt` and :origin:`requirements.txt`). ::
|
|
|
|
$ make install
|
|
PYENV OK
|
|
PYENV [virtualenv] requirements.sha256 failed
|
|
[virtualenv] - 6cea6eb6def9e14a18bf32f8a3e... ./requirements-dev.txt
|
|
[virtualenv] - 471efef6c73558e391c3adb35f4... ./requirements.txt
|
|
...
|
|
PYENV [virtualenv] installing ./requirements*.txt into local/py3
|
|
...
|
|
PYENV OK
|
|
PYENV [install] pip install -e 'searx[test]'
|
|
...
|
|
Successfully installed argparse-1.4.0 searx
|
|
BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
|
|
BUILDENV INFO:searx:Initialisation done
|
|
BUILDENV build utils/brand.env
|
|
|
|
.. sidebar:: drop environment
|
|
|
|
To get rid of the existing environment before re-build use :ref:`clean target
|
|
<make clean>` first.
|
|
|
|
If you think, something goes wrong with your ./local environment or you change
|
|
the :origin:`setup.py` file, you have to call :ref:`make clean`.
|
|
|
|
.. _make buildenv:
|
|
|
|
``make buildenv``
|
|
=================
|
|
|
|
Rebuild instance's environment with the modified settings from the
|
|
:ref:`settings global brand` and :ref:`settings global server` section of your
|
|
:ref:`settings.yml <settings location>`.
|
|
|
|
We have all SearXNG setups are centralized in the :ref:`settings.yml` file.
|
|
This setup is available as long we are in a *installed instance*. E.g. the
|
|
*installed instance* on the server or the *installed developer instance* at
|
|
``./local`` (the later one is created by a :ref:`make install <make
|
|
install>` or :ref:`make run <make run>`).
|
|
|
|
Tasks running outside of an *installed instance*, especially those tasks and
|
|
scripts running at (pre-) installation time do not have access to the SearXNG
|
|
setup (from a *installed instance*). Those tasks need a *build environment*.
|
|
|
|
The ``make buildenv`` target will update the *build environment* in:
|
|
|
|
- :origin:`utils/brand.env`
|
|
|
|
Tasks running outside of an *installed instance*, need the following settings
|
|
from the YAML configuration:
|
|
|
|
- ``GIT_URL`` from :ref:`brand.git_url <settings global brand>`
|
|
- ``GIT_BRANCH`` from :ref:`brand.git_branch <settings global brand>`
|
|
|
|
- ``SEARX_URL`` from :ref:`server.base_url <settings global server>` (aka
|
|
``PUBLIC_URL``)
|
|
- ``SEARX_BIND_ADDRESS`` from :ref:`server.bind_address <settings global server>`
|
|
- ``SEARX_PORT`` from :ref:`server.port <settings global server>`
|
|
|
|
.. _make run:
|
|
|
|
``make run``
|
|
============
|
|
|
|
To get up a running a developer instance simply call ``make run``. This enables
|
|
*debug* option in :origin:`searx/settings.yml`, starts a ``./searx/webapp.py``
|
|
instance, disables *debug* option again and opens the URL in your favorite WEB
|
|
browser (:man:`xdg-open`)::
|
|
|
|
$ make run
|
|
PYENV OK
|
|
SEARX_DEBUG=1 ./manage.sh pyenv.cmd python ./searx/webapp.py
|
|
...
|
|
INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
|
|
|
|
.. _make clean:
|
|
|
|
``make clean``
|
|
==============
|
|
|
|
Drop all intermediate files, all builds, but keep sources untouched. Before
|
|
calling ``make clean`` stop all processes using :ref:`make install`. ::
|
|
|
|
$ make clean
|
|
CLEAN pyenv
|
|
PYENV [virtualenv] drop ./local/py3
|
|
CLEAN docs -- ./build/docs ./dist/docs
|
|
CLEAN locally installed npm dependencies
|
|
CLEAN test stuff
|
|
CLEAN common files
|
|
|
|
.. _make docs:
|
|
|
|
``make docs docs.autobuild docs.clean``
|
|
=======================================
|
|
|
|
We describe the usage of the ``doc.*`` targets in the :ref:`How to contribute /
|
|
Documentation <contrib docs>` section. If you want to edit the documentation
|
|
read our :ref:`make docs.live` section. If you are working in your own brand,
|
|
adjust your :ref:`settings global`.
|
|
|
|
.. _make docs.gh-pages:
|
|
|
|
``make docs.gh-pages``
|
|
======================
|
|
|
|
To deploy on github.io first adjust your :ref:`settings global`. For any
|
|
further read :ref:`deploy on github.io`.
|
|
|
|
.. _make test:
|
|
|
|
``make test``
|
|
=============
|
|
|
|
Runs a series of tests: :ref:`make test.pylint`, ``test.pep8``, ``test.unit``
|
|
and ``test.robot``. You can run tests selective, e.g.::
|
|
|
|
$ make test.pep8 test.unit test.sh
|
|
TEST test.pep8 OK
|
|
...
|
|
TEST test.unit OK
|
|
...
|
|
TEST test.sh OK
|
|
|
|
.. _make test.sh:
|
|
|
|
``make test.sh``
|
|
================
|
|
|
|
:ref:`sh lint` / if you have changed some bash scripting run this test before
|
|
commit.
|
|
|
|
.. _make test.pylint:
|
|
|
|
``make test.pylint``
|
|
====================
|
|
|
|
.. _Pylint: https://www.pylint.org/
|
|
|
|
Pylint_ is known as one of the best source-code, bug and quality checker for the
|
|
Python programming language. The pylint profile we use at searx project is
|
|
found in project's root folder :origin:`.pylintrc`.
|
|
|
|
.. _make search.checker:
|
|
|
|
``search.checker.{engine name}``
|
|
================================
|
|
|
|
To check all engines::
|
|
|
|
make search.checker
|
|
|
|
To check a engine with whitespace in the name like *google news* replace space
|
|
by underline::
|
|
|
|
make search.checker.google_news
|
|
|
|
To see HTTP requests and more use SEARX_DEBUG::
|
|
|
|
make SEARX_DEBUG=1 search.checker.google_news
|
|
|
|
.. _3xx: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_redirection
|
|
|
|
To filter out HTTP redirects (3xx_)::
|
|
|
|
make SEARX_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
|
|
...
|
|
Engine google news Checking
|
|
https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
|
|
https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
|
|
--
|
|
https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
|
|
https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
|
|
--
|
|
|
|
|
|
``make pybuild``
|
|
================
|
|
|
|
.. _PyPi: https://pypi.org/
|
|
.. _twine: https://twine.readthedocs.io/en/latest/
|
|
|
|
Build Python packages in ``./dist/py``::
|
|
|
|
$ make pybuild
|
|
...
|
|
BUILD pybuild
|
|
running sdist
|
|
running egg_info
|
|
...
|
|
running bdist_wheel
|
|
|
|
$ ls ./dist
|
|
searx-0.18.0-py3-none-any.whl searx-0.18.0.tar.gz
|
|
|
|
To upload packages to PyPi_, there is also a ``pypi.upload`` target (to test use
|
|
``pypi.upload.test``). Since you are not the owner of :pypi:`searx` you will
|
|
never need to upload.
|