Commit Graph

356 Commits (master)

Author SHA1 Message Date
Milad-Laly 0af41ab15c [fix] typos in utils/searxng.sh 1 year ago
llmII 4e2eb9922e Fix checksum virtualenv checks.
There's already precedence for not using GNUism sha256sum longopts as
seen in searxng/utils/lib_go.sh so update lib.sh to not use them either.
A nice side effect is now the sha256sum usage doesn't care if you're
using BSD sha256sum or GNU sha256sum which makes this work under FreeBSD.
1 year ago
Alexandre Flament b971167ced move searx.shared.redisdb to searx.redisdb 1 year ago
Alexandre FLAMENT e92755d358 Initialize Redis in searx/webapp.py
settings.yml:
* The default URL was unix:///usr/local/searxng-redis/run/redis.sock?db=0
* The default URL is now "false"

The default URL makes the log difficult to deal with:
if the admin didn't install a Redis instance, the logs record a false error.

It worked before because SearXNG initialized the Redis connection when the limiter started.

In this commit, SearXNG initializes Redis in searx/webapp.py
so various components can use Redis without taking care of the initialization step.
2 years ago
Markus Heiser ba8959ad7c [fix] typos / reported by @kianmeng in searx PR-3366
[PR-3366] https://github.com/searx/searx/pull/3366

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Léon Tiekötter 2b94fef7ec [fix] uWSGI: increase buffer-size
Increase max size of a request, by default it is 4k [1].  4096 as buffer-size is
too small and will result in the preference urls not working.

[1] https://uwsgi-docs.readthedocs.io/en/latest/Options.html#buffer-size
2 years ago
Markus Heiser bded8750d5 [mod] fix minor leftovers from PR #1332
Related: https://github.com/searxng/searxng/pull/1332
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 692708aa77 [clean up] drop obsolete searx, filtron and morty install scripts
Since ./utils/searxng.sh is implemented, the old installation procedures from
filtron, morty and searx can be removed.

For users who want to upgrade, the procedures for removing old installations
have still been retained.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser ed8a169029 [doc] update documentation of the installation procedures
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 782f73540e [utils/searxng.sh] implement new script to install SearXNG
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 81bba44869 [install scripts] rename SEARX_<name> variables to SEARXNG_<name>
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Alexandre Flament 9b3efa6d8a theme: remove __common__ 2 years ago
Markus Heiser e8541b6006 [theme] peel out oscar from SearXNG development
This is the first step of removing oscar theme

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 463254e3b4 [fix] since git v2.35.2 root can't read from unsafe repository
Git v2.35.2 closes an security issue, it is no longer possible that root uses a
git repo that is owned by someone else, the error message is::

    fatal: unsafe repository ('/share/darmarit.org/cache/searxng' is owned by someone else)

The fix is to run the `git diff --name-only` not as root in a sudo command.

[1] https://github.blog/2022-04-12-git-security-vulnerability-announced/
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
HZH 095beb8605
typo: wanr_msg -> warn_msg 2 years ago
Jabster28 89bcea56ec
don't reinstall existing packages on arch 2 years ago
Markus Heiser 7308a6e33c [LXC] ubuntu 20.10 EOL, upgrade to ubunut 21.10 [1]
[1] https://wiki.ubuntu.com/Releases

Tested by::

    # build the container ..
    $ sudo -H ./utils/lxc.sh build searx-ubu2110

    # open a shell in the container
    $ sudo -H ./utils/lxc.sh cmd searx-ubu2110 bash
    [root@searx-ubu2110 SearXNG]#

    # install a complete SearXNG suite ..
    [root@searx-ubu2110 SearXNG]# ./utils/searx.sh install all
    ...

    # install apache to export the SearXNG instance by HTTP
    [root@searx-ubu2110 SearXNG]# ./utils/searx.sh apache install
    ...
    INFO:  got 200 from http://10.174.184.94/searx

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 1c79c1a3d2 [LXC] fedora33 EOL [1], upgrade to fedora35 [2]
[1] https://docs.fedoraproject.org/en-US/releases/eol/
[2] https://docs.fedoraproject.org/en-US/releases/f35/

Tested by::

    # build the container ..
    $ sudo -H ./utils/lxc.sh build searx-fedora35

    # open a shell in the container
    $ sudo -H ./utils/lxc.sh cmd searx-fedora35 bash
    [root@searx-fedora35 SearXNG]#

    # install a complete SearXNG suite ..
    [root@searx-fedora35 SearXNG]# ./utils/searx.sh install all
    ...

    # install apache to export the SearXNG instance by HTTP
    [root@searx-fedora35 SearXNG]# ./utils/searx.sh apache install
    ...
    INFO:  got 200 from http://10.174.184.94/searx

To build wheel `python3-devel` needs to be added to SEARX_PACKAGES_fedora::

    |searx|   × Building wheel for setproctitle (pyproject.toml) did not run successfully.
    |searx|   │ exit code: 1
    ...
    |searx|       In file included from src/spt.h:15,
    |searx|                        from src/setproctitle.c:14:
    |searx|       src/spt_python.h:16:10: fatal error: Python.h: No such file or directory
    |searx|          16 | #include <Python.h>
    |searx|             |          ^~~~~~~~~~
    |searx|       compilation terminated.
    |searx|       error: command '/usr/bin/gcc' failed with exit code 1
    |searx|       [end of output]
    ...
    |searx|   ERROR: Failed building wheel for setproctitle

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 8d69ee5e7f [mod] Serving static files with uWSGI (searxng.ini)
1. Serving static files with uWSGI by using static file mount points [1].
2. Expires set to one year since there are hashes [2]

[1] https://uwsgi-docs.readthedocs.io/en/latest/StaticFiles.html#mode-3-using-static-file-mount-points
[2] https://github.com/searxng/searxng/pull/932

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 5eedd5b72a [fix] socket in SearXNG's uWSGI app (searxng.ini)
Use SEARX_UWSGI_SOCKET in uWSGI systemd service

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser e48e2a9914 [fix] APACHE_SEARX_SITE=searxng.conf renamed in 2b125214
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Martin Fischer 7ce7625117
Merge pull request #804 from return42/minor-fix
[mod] lib_nvm.sh: minor improvements / no functional change
2 years ago
Markus Heiser 309147d86f [mod] introduce node.env.devtools function 2 years ago
Martin Fischer cac0352986 [mod] remove .nvm_packages, add eslint to package.json 2 years ago
Markus Heiser a163385474 [mod] lib_nvm.sh: minor improvements / no functional change
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 977e9a4330
Merge pull request #686 from return42/lib_redis
Add redis DB and connector
2 years ago
Markus Heiser fe8b88c8a4 [mod] script to build & install a redis instance
A script to build & install a simple & isolated redis service, dedicated to
SearXNG and connected via Unix socket.

    $ ./manage redis.help
    redis.:
      devpkg    : install essential packages to compile redis
      build     : build redis binaries at /800GBPCIex4/share/SearXNG/dist/redis/6.2.6/amd64
      install   : create user (searxng-redis) and install systemd service (searxng-redis)
      remove    : delete user (searxng-redis) and remove service (searxng-redis)
      shell     : start bash interpreter from user searxng-redis
      src       : clone redis source code to <path> and checkput 6.2.6
      useradd   : create user (searxng-redis) at /usr/local/searxng-redis
      userdel   : delete user (searxng-redis)
      addgrp    : add <user> to group (searxng-redis)
      rmgrp     : remove <user> from group (searxng-redis)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Alexandre Flament 76cbfbbdda reference docs.searxng.org 2 years ago
Markus Heiser c89d2aab3f [mod] make static.build.commit more robust
- use single quote in the STATIC_BUILT_PATHS to avoid bash globbing
- don't try to commit if no files have been changed

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 9349c71c54 [mod] themes/simple/img/searxng.svg -> src/brand/searxng.svg
* move `searx/static/themes/simple/img/searxng.svg` to `src/brand/searxng.svg`

* README.rst can use it without a reference to a theme.

* the simple theme can create `searx/static/themes/simple/img/searxng.png` using
  the svg2png task

Suggested-by: @dalf https://github.com/searxng/searxng/pull/561#issuecomment-981747902
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser ee8b6aac36 [fix] searx.sh update: replace git pull by a hard reset
If the fetched branch has been rebased a 'git pull' will fails.  To get fetched
branch in the working tree, a git reset is needed.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser ef3f084c49
Merge pull request #516 from return42/searxng/filtron
[mod] filtron.sh - install github.com/searxng/filtron
2 years ago
Markus Heiser 93e933278a [fix] filtron.sh & morty.sh: quote "$GO_ENV"
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser c1bee0d96f [mod] filtron.sh - install github.com/searxng/filtron
SearXNG maintains a fork of filtron at [1]

[1] https://github.com/searxng/filtron

Related: https://github.com/searxng/searxng/issues/507#issuecomment-970254027
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 5242a841a5 [mod] NVM: dev-tools pre-installed in NVM's node installation
The Node.js installation in the NVM environment can be used by IDEs and other
developer tasks.  The required developer packagaes are added to the file
./.nvm_packages and will be installed when Node.js is installed.  Initial we
start with:

- eslint

Having a dedicated developer enviroment, provided by nvm makes it easy to
integrate Node.js packages into various IDEs.  One example is shown in the
.dir-locals.el which is used by emacs.

[1] https://github.com/nvm-sh/nvm#default-global-packages-from-file-while-installing
[2] https://eslint.org

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 69dd025da9 [mod] nvm.cmd ... : run command ... in NVM environment
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 6b4cad768d [mod] NVM: pin Node.js to v16.13.0
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 3e2f3900b8 [fix] nvm.install: add missing popd / node.env exit with error (254)
Issue::

    $ make clean node.env
    ...
    CLEAN     [NVM] drop .nvm/
    ...
    INFO:  install Node.js by NVM
    ...
    Now using node v16.13.0 (npm v8.1.0)
    ...
    INSTALL   searx/static/themes/oscar/package.json
    npm ERR! code ENOENT
    npm ERR! syscall open
        # Here now comes the issue, caused by the missing 'popd' ..
    npm ERR! path SearXNG/.nvm/searx/static/themes/oscar/package.json
    npm ERR! errno -2
    npm ERR! enoent ENOENT: no such file or directory, open 'SearXNG/.nvm/searx/static/themes/oscar/package.json'
    ERROR: node.env exit with error (254)
    make: *** [Makefile:99: node.env] Error 254

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser c681570bb9 [fix] brand - partial revert of e82e37ce6
In commit e82e37ce6 accidentally the brand file was touched / partial
revert::

  export GIT_URL='https://github.com/searxng/searxng'
  export GIT_BRANCH='master'

Related: https://github.com/searxng/searxng/pull/515

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 48752815ed [fix] make nvm.clean - don't raise error when NVM is not installed
Issue was::

    $ LANG=C make nvm.clean
    INFO:  NVM is not installed
    make: *** [Makefile:99: nvm.clean] Error 42

Now::

    $ LANG=C make nvm.clean
    CLEAN     [NVM] not installed

BTW: change info_msg to build_msg

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser e82e37ce6d
Merge pull request #515 from return42/doc-theme
Various modification of the documentation and the theme sphinx-theme
2 years ago
Markus Heiser 1e997386e9
Merge pull request #508 from return42/arm64
[mod] Tools to install and maintain golang binaries & packages.
2 years ago
Markus Heiser 1dae0c0be0 [brand] SearXNG - docs rename links and fix documentation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser dc1442a2d1 [mod] Tools to install and maintain NVM versions manager for Node.js
[1] https://github.com/nvm-sh/nvm

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 79b41478ac [fix] filtron & morty - install golang binary that fits to arch & os
Closes: https://github.com/searxng/searxng/issues/507
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser abd5eee392 [mod] Tools to install and maintain golang [1] binaries & packages.
[1] https://golang.org/doc/devel/release#policy

A simple *helloworld* test with user 'my_user' :

  sudo -H adduser my_user
  ./manage go.golang go1.17.3 my_user
  ./manage go.install github.com/go-training/helloworld@latest my_user
  ./manage go.bash my_user
  $ helloword
  Hello World!!

Don't forget to remove 'my_user':  sudo -H deluser --remove-home my_user

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
MrPaulBlack 566dfe3330 [simple theme] update to ion icons 5 and drop webfont
* update from ionicons-npm to ionicons ver.5
* drop the webfont built by grunt for icons
* built icons.html template for inlining icons with jinja2 into html
* update icon to use mostly the outline version
* add icons to categories and do not display them on mobile to save space
* remove all legacy ion icon font files from simple theme
* icons.html is added in this commit since make statc.build.restore requires git to know the file already
* cleanup error-dialog
3 years ago
Markus Heiser 3a981257de [simple theme] create favicon.png from src/svg/searxng-wordmark.svg
new bash function convert_if_newer() / usage::

    convert_if_newer <origfile> <outfile> [<options>, ...]
    convert_if_newer "path/to/origin.svg" "path/to/converted.png" -transparent white -resize 64x64

Run's ImageMagik' convert comand to generate <outfile> from <origfile>, if
<origfile> is newer than <outfile>.  The command line is to convert is::

    convert <origfile> [<options>, ...] <outfile>

PNG 'searx/static/themes/simple/img/favicon.png' has been created by::

  $ make themes.simple
  CONVERT   searx/static/themes/simple/src/svg/searxng-wordmark.svg -transparent white -resize 64x64 searx/static/themes/simple/img/favicon.png
  ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser bf52353a5c [enh] minify searxng-wordmark.svg (HTMLMinifier)
Command::

  ./node_modules/.bin/html-minifier \
      --remove-comments \
      --collapse-whitespace \
      src/svg/searxng-wordmark.svg \
      -o ../../../templates/__common__/searxng-wordmark.min.svg

- html-minifier: https://github.com/kangax/html-minifier
  - onilne: https://kangax.github.io/html-minifier
  - grunt: https://www.npmjs.com/package/grunt-contrib-htmlmin
  - grunt-contrib-htmlmin: https://github.com/gruntjs/grunt-contrib-htmlmin
  - npm: https://www.npmjs.com/package/html-minifier

To test, rebuild your node environment::

  make node.env

Alternatives:

- pretty-data: https://github.com/vkiryukhin/pretty-data
  - grunt: https://www.npmjs.com/package/grunt-xmlmin
  - grunt-xmlming: https://github.com/dtrunk90/grunt-xmlmin
  - npm: https://www.npmjs.com/package/grunt-xmlmin

- minify-xml: https://github.com/kristian/minify-xml
  - no grunt package available
  - npm: https://www.npmjs.com/package/minify-xml

src/svg/searxng-wordmark.svg':
'../../../templates/__common__/searxng-wordmark.min.svg'

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 9f30d5b1f8 [fix] morty & filtron: don't use golang installed by package manager
When golang is installed via a package manager the local version, which filtron
downloads, is ignored.

BTW: With the new go1.17.2 (cfea51f4), 'go get' is depricated::

  go get: installing executables with 'go get' in module mode is deprecated.
    Use 'go install pkg@version' instead.
    For more information, see https://golang.org/doc/go-get-install-deprecation
    or run 'go help get' or 'go help install'.

Reported-by: @tiekoetter https://github.com/searxng/searxng/pull/455#issuecomment-954918411
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago