**new** utils/lib_install.sh:
Used to initialize installation procedures
- Modified source_dot_config function that
- loads .config.sh from an existing installation (at SEARX_SRC).
- initialize **SEARX_SRC_INIT_FILES**
- functions like:
- install_log_searx_instance()
- install_searx_get_state()
**modified** utils/searx.sh
- obsolete environment SEARX_INSTANCE_NAME has been replaced
by string 'SearXNG'.
**modified** utils/filtron.sh, utils/morty.sh, utils/searx.sh
- source utils/lib_install.sh
- normalize logging of environment variables using new function
install_log_searx_instance()
**modified** utils/lib.sh
- fix marginal typos
**Installation scripts**
The utils/lib_install.sh is sourced by the installations scripts:
- utils/searx.sh
- utils/morty.sh
- utils/filtron.sh
If '${SEARX_SRC}/.config.sh' exists, the modified source_dot_config() function
loads this configuration (instead of './.config.sh').
**SEARX_SRC_INIT_FILES**
Array of file names to sync into a installation at $SEARX_SRC. The file names
are relative to the $REPO_ROOT. Set by function init_SEARX_SRC_INIT_FILES().
Most often theses are files like:
- .config.sh
- searx/settings.yml
- utils/brand.env
- ...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Prompt the verbose messages ...
PYENV [check] import yaml --> OK
PYENV OK
only when environment $VERBOSE is active.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Added function searx.get_setting(name, default=_unset):
Returns the value to which ``name`` point. If there is no such name in the
settings and the ``default`` is unset, a KeyError exception is raised.
In all the python processes ..
- make docs
- make buildenv
- make install (setup.py)
the usage of the 'brand.*' name space is replaced by 'searx.get_setting'
function.
- brand.SEARX_URL --> get_setting('server.base_url')
- brand.GIT_URL --> get_setting('brand.git_url')
- brand.GIT_BRANCH' --> get_setting('server.base_url')
- brand.ISSUE_URL --> get_setting('brand.issue_url')
- brand.DOCS_URL --> get_setting('brand.docs_url')
- brand.PUBLIC_INSTANCES --> get_setting('brand.public_instances')
- brand.CONTACT_URL --> get_setting('general.contact_url', '')
- brand.WIKI_URL --> get_setting('brand.wiki_url')
- brand.TWITTER_URL --> get_setting('brand.twitter_url', '')
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
From pacman's man page:
--noprogressbar
Do not show a progress bar when downloading files. This can be useful for
scripts that call pacman and capture the output.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In 2a12637 an ASCII escape code was added, such escape codes should not be
written to pipes (when the output is not a terminal).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Some piped commands hide the cursor, show cursory when the stream ends.
Most often this is a bug of the command which piped. The command should not
hide the cursor when it writes to a pipe. I have seen this bug with the package
manager (pacman) from ArchLinux.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The functions:
- static.build.commit
- static.build.commit.drop
- static.build.restore
are imported into the ./manage script. To avoid name collisions some variables
and fucntions has been renamed by adding the prefix *static_*.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Marginal simplification of the procedure to get only the last local commit which
is not in remotes.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
- add script to shellcheck,
- fix error messages from shellcheck and
- moved global variables to local variables (lower case)
No functional change!
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The idea is to avoid conflict when there are differents branches with changes are made on the static files.
A solution is to ask the administrators to build the files from the sources, but it requires to install the npm dependencies.
So the solution in this commit keep the sources and the build files in the same git repository.
In one branch, the modification of the source (*.less, *.js) are commited without the built files.
The built files are commited in a uniq commit, with a commit message "Static build"
In case of merge or rebase, this commit can be dropped.
New make targets:
* static.build.commit.drop:
drop the last "Static build".
The command checks that there are only build files in the commit.
* static.build.commit :
call "make static.build.commit.drop"
call "make themes.all"
commit the files
* static.git.restore.staged:
git restore --staged <build files>
* static.git.restore:
git restore <build files>
Related to https://github.com/searxng/searxng/issues/137
Docker is blocking network of existing LXC containers / there is a conflict in
the iptables setup of Docker & LXC. With this patch:
- utils/lxc.sh checks internet connectivity (instead of silently hang)
- Chapter "Internet Connectivity & Docker" describes the problem and made a
suggestion for a solution a solution
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In 5a7b12ee we changed the signature of the YAML settings, this patch takes this
into account.
Related-to: 5a7b12ee [yamllint] searx/settings.yml
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The node.env build environment require npm, ttfautohint and fontforge installed
in the OS. These tools can be installed by::
sudo -H ./utils/searx.sh install buildhost
If one of the tools is not installed, the script node.env stops with a
appropriate message.
BTW: We ignore CentOS-7 as developer & build platform
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This file is generated by webfont.
* It is now generated as searx/static/themes/simple/ion.less
* It is generated before the .less compilation.
* .gitignore includes this file
Add two new package depedencies: fontforge ttfautohint
See utils/searx.sh
The public URL of the searx instance: PUBLIC_URL="https://mydomain.xy/searx"
The default is SEARX_URL which is taken from ./utils/brand.env.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Access to formats can be denied by settings configuration::
search:
formats: [html, csv, json, rss]
Closes: https://github.com/searxng/searxng/issues/95
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
To select a different file with filtron rules, set environment
FILTRON_RULES_TEMPLATE
the default is
utils/templates/etc/filtron/rules.json
The installation is done by the new function install_rules() which offers a
multiple choice in case of collisions (known from searx.sh install setup).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Implement a boilerplate to manage performance optimized virtualenv builds.
Shell scripts can use (e.g.) 'pyenv.cmd' to execute command in the virtualenv
without having to worry about whether and how the environment is provided. ::
pyenv.cmd which python
..../local/py3/bin/python
pyenv.cmd which pip
..../local/py3/bin/pip
If pyenv.cmd released multiple times the installation will only rebuild if the
function 'pyenv.OK' fails. Function 'pyenv.OK' make some test to validate that
the virtualenv exists and works as expected. The check also fails if
requirements listed requirements-dev.txt and requirements.txt has been edited.
Among these tests 'pyenv.OK' calls 'pyenv.check' which implements a python
script that validate the python installation. Here is an example how a
'pyenv.check' implementation could look like::
pyenv.check() {
cat <<EOF
import yaml
print('import yaml --> OK')
EOF
}
Signed-off-by: Markus Heiser <markus@darmarit.de>
Newer CentOS-7 images from https://images.linuxcontainers.org do no longer
include the which command.
Issue:
$ sudo -H ./utils/lxc.sh cmd searx-centos7 ./utils/filtron.sh install all
INFO: [searx-centos7] ./utils/filtron.sh install all
...
Install Go in user's HOME
-------------------------
download and install go binary ..
...
-bash: line 1: which: command not found
-->|ERROR - Go Installation not found in PATH!?!
-bash: line 2: which: command not found
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The wrong and unnecessary `pip install .` is executed in /usr/local/searx and is
responsible for the error message:
ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: /usr/local/searx
The correct pip-install comes right after changing to `cd ${SEARX_SRC}`.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
LTS distros like Ubuntu 18.04 do not ship a up-to-date version of git.::
$ sudo -H ./utils/lxc.sh cmd searx-ubu1804 git --version
...
git version 2.17.1
The option `--show-current` was added in git v2.22, the alternative to this
option is::
git rev-parse --abbrev-ref HEAD
Issue when using option `--show-current`::
[searx-ubu1804] Clone searx sources
[searx-ubu1804] -------------------
[searx-ubu1804]
[searx-ubu1804] error: unknown option `show-current'
[searx-ubu1804] usage: git branch [<options>] [-r | -a] [--merged | --no-merged]
....
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
EOL of Ubuntu 16.04 (Xenial Xerus) is in April 2021 but we do not support 16.04
since a longer time. Issues are comming from dependencies (e.g. pip)::
[searx-ubu1604] |searx| SyntaxError: invalid syntax
[searx-ubu1604] |searx| Traceback (most recent call last):
[searx-ubu1604] |searx| File "/usr/local/searx/searx-pyenv/bin/pip", line 7, in <module>
[searx-ubu1604] |searx| from pip._internal.cli.main import main
[searx-ubu1604] |searx| File "/usr/local/searx/searx-pyenv/lib/python3.5/site-packages/pip/_internal/cli/main.py", line 60
[searx-ubu1604] |searx| sys.stderr.write(f"ERROR: {exc}")
[searx-ubu1604] |searx| ^
[searx-ubu1604] |searx| SyntaxError: invalid syntax
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
use a sparql request on wikidata to get the list of currencies.
currencies.json contains the translation for all supported searx languages.
Supersede #993
