Archives/searxng
2
0
Fork 1
mirror of https://github.com/searxng/searxng synced 2024-11-09 01:10:26 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,580 Commits 6 Branches 17 Tags 106 MiB
2b93e70a26
Commit Graph

169 Commits

Author SHA1 Message Date
Markus Heiser
f27f6c6c5d docs: use make and uitls/searx.sh in containers 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-31 18:25:40 +02:00
Markus Heiser
dd53c45a2c docs: add utils/lxc.sh docs, normalize filtron, morty & searx docs 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-30 18:47:01 +02:00
Markus Heiser
3e4d022d04 [fix] make test.sh: fix various shellcheck error messages 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-30 11:03:46 +02:00
Markus Heiser
5e2b161379 Makefile: marginal fix 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-29 16:46:07 +02:00
Markus Heiser
0742c2504d build environment: add GIT_BRANCH variable 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-29 16:28:17 +02:00
Markus Heiser
7b4cf2eb48 tooling box: simplify build enviroments 
- no more need for a .config.mk
- docs: use searx.brands environment
- searx.sh, filtron.sh & morty.sh are sourcing utils/brand.env

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-29 15:09:34 +02:00
Markus Heiser
65b22be1f4 Merge PR #1900 of https://github.com/asciimoo/searx into filtron 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-29 13:03:27 +02:00
Markus Heiser
442b27ac1b bash: use build environment from Makefile 
To Makefile target brand, add creation of bash environment in::

  utils/brand.env

In bash scripts (manage.sh) source env by::

  . utils/brand.env

manage.sh help: show GIT_URL and more environment

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-29 11:41:59 +02:00
Markus Heiser
1fcec0bbda lxc.sh install base: add bash and python3 to LXC_BASE_PACKAGES 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-23 18:06:35 +01:00
Markus Heiser
baf1ffd1fa LXC: set LXC_ENV_FOLDER=lxc/$(shell hostname)/ 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-23 18:05:46 +01:00
Markus Heiser
21ba25ebb8 Merge branch 'filtron' of https://github.com/return42/searx into filtron 2020-03-23 15:02:32 +01:00
Markus Heiser
2d956696e9 docs: building (PDF) books / build user book 
BTW: cleaned up Makefile target help

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-21 18:45:38 +01:00
Markus Heiser
62da769952 ./utils/lxc.sh: add command 'install base' 
to install basic packages like git and *build essentials*::

  $ ./utils/lxc.sh install base

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-18 17:47:48 +01:00
Markus Heiser
d2cfe9ce5b LXC: add /.lxcenv 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-18 15:34:46 +01:00
Markus Heiser
86e79488aa LXC: utils/makefile.lxc (inital) add /.lxcenv.mk to contaiiners 
Get LXC environment when building make targets.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-15 17:01:36 +01:00
Markus Heiser
80f7b65804 searx.sh: add buildhost cmd, installs OS packages for builds 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-09 01:37:26 +01:00
Markus Heiser
f5f83b8a99 shellcheck: minor fixes to support all dsitros 
checked with::

  sudo ./utils/lxc.sh cmd -- make test.sh

on ubu1804, ubu1910, fedora31 and archlinux.  All checks OK except:

ubu1604: uses shellcheck v0.3.7 (from 04/2015) which is no longer supported!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-08 18:37:16 +01:00
Markus Heiser
04ad648105 makefile.python: remove python2 support 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-08 18:35:38 +01:00
Markus Heiser
938bb02550 searx.sh: add commandline 'install packages' 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-08 18:32:30 +01:00
Markus Heiser
d48c7bf678 LXC: fixed raw.idmap & removed obsolete uid/gid subordinate 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-08 18:30:25 +01:00
Markus Heiser
e7f69b63f1 LXC: from searx.sh, morty.sh and filtron.sh tests 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-08 02:41:45 +01:00
Markus Heiser
b1e90cff23 LXC: separate lxc-suite from lxc & improved command line. 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-07 20:24:08 +01:00
Markus Heiser
c15337850e fix: minor typos 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-06 22:06:19 +01:00
Markus Heiser
387c6a7769 docs: improve description of uwsgi & ngingx setup 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-06 14:47:00 +01:00
Markus Heiser
cbc08fdc26 docs: describe uwsgi setup of all suported distributions 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-04 19:56:40 +01:00
Markus Heiser
3aff2c19d1 docs: complement uwsgi documentation for all distros 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-04 12:22:20 +01:00
Markus Heiser
9f1cb7a943 Merge branch 'master' of https://github.com/asciimoo/searx into filtron 2020-03-04 12:11:58 +01:00
Markus Heiser
3cb7daedb7 docs: generic doocumentation for searx-uwsgi 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-03 19:57:15 +01:00
Markus Heiser
f25da6070e docs: generic documentation get templating 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-03 16:26:02 +01:00
Markus Heiser
7af991acda LXC: add ubuntu 19.10 and 20.04 to the list of test images 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-02 19:01:08 +01:00
Markus Heiser
c3e4753ce9 docs: generic documentation from the installation scripts 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-02 19:00:19 +01:00
Markus Heiser
491cb95a1f utils/lxc.env: separate environment that is used in containers 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-01 18:28:10 +01:00
Markus Heiser
40843fe95a searx/data/engines_languages.json: sort json file (no content change) 
To get meaningfull diffs, the json file has to be sorted.  Before applying any
further content patch, the json file needs a inital sort (without changing any
content).

Sorted by::

  import sys, json

  with open('engines_languages.json') as f:
      j = json.load(f)

  with open('engines_languages.json', 'w') as f:
      json.dump(j, f, indent=2, sort_keys=True)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-01 07:56:46 +01:00
Markus Heiser
37c135f2ce LXC: improved UX when working with a bunch of containers 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-27 19:13:03 +01:00
Markus Heiser
af6acd3417 LXC: install searx-suite installs searx, filtron & morty on all containers 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-26 19:07:55 +01:00
Markus Heiser
d5917cc029 utils/lib.sh: make uWSGI installation available for all distros 
support: ubuntu, debin, fedora, archlinux

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-25 20:20:17 +01:00
Markus Heiser
f17b28743a utils/serx.sh: make uWSGI installation available for all distros (WIP) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-23 21:05:22 +01:00
Markus Heiser
5fb6d4f508 LXC: normalize package installation & user creation. 
utils/lib.sh:
- get DIST_ID & DIST_VERSION from /etc/os-release
- pkg_[install|remove|...] supports ubuntu, debian, archlinux & fedora

utils/lxc.sh
- Workaround for the "setrlimit(RLIMIT_CORE): Operation not permitted" error::
    'Set disable_coredump false' >> /etc/sudo.conf

utils/[searx.sh|filtron.sh|morty.sh]
- switched user creation from 'adduser' perl script to 'useradd' built-in
  command

utils/searx.sh
- install packages for ubuntu, debian, archlinux & fedora

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-23 12:10:45 +01:00
Markus Heiser
e36e0f80ae LXC: added archlinux and fedora31 containers 
BTW: update all packages when installing conatiner's boilerplate

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-18 18:40:34 +01:00
Markus Heiser
ccb96544f7 LXC: improved console messages & prompt 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-18 18:20:03 +01:00
Markus Heiser
59e4026762 searx.sh: install settings at /etc/searx/settings.yml 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-17 18:58:59 +01:00
Markus Heiser
de58f02f6b filtron: add missing log action to the filtron rules 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-17 15:36:10 +01:00
Markus Heiser
0bfc61dbe9 tooling box: misc fixes from lxc tests 
Tested by:

  sudo -H ./utils/lxc.sh build containers
  sudo -H ./utils/lxc.sh cmd /share/searx/utils/searx.sh    install all
  sudo -H ./utils/lxc.sh cmd /share/searx/utils/filtron.sh  install all
  sudo -H ./utils/lxc.sh cmd /share/searx/utils/morty.sh    install all

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-16 22:29:06 +01:00
Markus Heiser
a1f5f2ced8 LXC: minor fixes and renaming 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-16 22:26:03 +01:00
Markus Heiser
ad32739860 shellcheck: fix usse -n instead of ! -z (SC2236 SC2237) 
- https://www.shellcheck.net/wiki/SC2236 -- Use -n instead of ! -z.
 - https://www.shellcheck.net/wiki/SC2237 -- Use [ -n .. ] instead of ! [ -z ....

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-16 20:07:37 +01:00
Markus Heiser
e8cf225046 LXC: tooling box - add boilerplate to containers 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-16 18:18:15 +01:00
Markus Heiser
0b2b325552 LXC: tooling box - add HOST shares and commandline to containers (WIP) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-13 18:25:03 +01:00
Markus Heiser
e6bf2038c3 filtron.sh: add 'install rules' command 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-11 15:58:35 +01:00
Markus Heiser
0d6153db12 filtron.sh: updated rules from production 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-11 15:57:42 +01:00
Markus Heiser
7751b29559 LXC: add LXC tooling box (initial, WIP) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-08 19:12:28 +01:00
Markus Heiser
5c0f9c8c79 tooling box: varius fix from tests 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-08 17:12:01 +01:00
Markus Heiser
ed4cb4f160 tooling box: varius fix from tests 2020-02-08 13:24:08 +01:00
Markus Heiser
a014fa0ccd upload-pypi: twine is needed to upload to PyPi 'make upload-pypi' 
To build & deploy packages on PyPi call make target:

  make upload-pypi

- https://github.com/asciimoo/searx/issues/1829

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-05 15:26:57 +01:00
Markus Heiser
71d7550dbe tooling box ./utils/*: minor fix from production test 2020-02-04 19:47:33 +01:00
Markus Heiser
2f40f61f83 /etc/filtron/rules.json: normalize rules from docs & tooling box 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-04 17:59:58 +01:00
Markus Heiser
eedd63ccd5 docs: revision of the installation instructions 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-04 16:42:13 +01:00
Markus Heiser
79e1f837ef utils & Makefile: add .config.mk & .config.sh for searx brands 
By isolating the environment of makefiles and bash scripts into .config.mk and
.config.sh it is simple to maintain searx brands by setting some central
environments.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-04 10:39:42 +01:00
Markus Heiser
94ac560dcc doc: add descriptions of our tooling box in ./utils 
Initial started with the scripts for searx, filtron and morty installations.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-03 13:25:51 +01:00
Markus Heiser
e7401796f6 utils/morty.sh: fixed public url 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-02 18:20:20 +01:00
Markus Heiser
a4437c47ac utils/morty.sh: add script to install morty result proxy 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-02 18:14:10 +01:00
Markus Heiser
709ac51d33 utils/filtron.sh: generalize systemd, accounts and golang tasks 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-02-01 16:59:27 +01:00
Markus Heiser
56a93ee770 utils/lib.sh: colorize messages 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-31 18:24:31 +01:00
Markus Heiser
0bb8847087 utils/filtron.sh: add option to debug filtron requests 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-31 17:25:38 +01:00
Markus Heiser
91a55e159e apache: reverse proxy, set ProxyPreserveHost On 
related discussions:

- https://github.com/asciimoo/searx/issues/1822
- https://github.com/asciimoo/searx/issues/1819#issuecomment-580400259

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-31 15:54:07 +01:00
Markus Heiser
cd9b877c8c utils/searx.sh & filtron.sh: misc changes from first tests (WIP) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-31 15:53:53 +01:00
Markus Heiser
6274a54746 utils/searx.sh & filtron.sh: misc changes from first tests (WIP) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-30 19:55:51 +01:00
Markus Heiser
924bf65517 utils/searx.sh & filtron.sh: misc changes from first tests (WIP) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-29 20:00:50 +01:00
Markus Heiser
971a8264b2 utils/searx.sh: add apache site searx.conf:uwsgi (WIP) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-27 19:08:40 +01:00
Markus Heiser
d171fcd56e utils/searx.sh: add apache site searx.conf:uwsgi 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-21 18:38:57 +01:00
Markus Heiser
af2f58fc58 utils/filtron.sh: add script to install filtron middleware (apache) 
Preparation for the installation of an apache site.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-20 19:08:56 +01:00
Markus Heiser
9b5a7f7559 utils/searx.sh: add script to install isolated searx service 
First version which serves searx over uwsgi at http://127.0.0.1:8888

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-20 16:55:05 +01:00
Markus Heiser
89df9d9141 utils/searx.sh: add script to install isolated searx service (WIP) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-16 14:01:38 +01:00
Markus Heiser
3cf31528f3 utils/searx.sh: add script to install isolated searx service (WIP) 
WIP: written from scratch / linted but untested

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-14 19:26:54 +01:00
Markus Heiser
c2b27d9120 Merge branch 'filtron' of https://github.com/return42/searx into filtron 2020-01-13 18:39:46 +01:00
Markus Heiser
b5449ec47c filtron: log suspiciously frequent queries (WIP) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-13 18:37:05 +01:00
Markus Heiser
3ed82a97af utils/filtron.sh: add command 'update filtron' 
BTW: tidy up some code style and fix some issues reported by the linter.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-12 15:51:04 +01:00
Markus Heiser
39feb141bc docs(admin): add description of the utils/filtron.sh script 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-11 12:50:40 +01:00
Markus Heiser
4990b07b4b utils/filtron.sh: various fix from first installation test (WIP) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-09 16:25:05 +01:00
Markus Heiser
4139c63d23 utils/filtron.sh: add script to install filtron middleware 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-08 19:21:07 +01:00
Markus Heiser
28dacee288 utils: add lib.sh containing common shell script 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-01-08 18:13:22 +01:00
Markus Heiser
d3e4e81faf makefile.sphinx: fix gh-pages / pull before add commits 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2019-12-22 23:28:07 +01:00
Markus Heiser
121f696460 Makefile: gh-pages: keep history of gh-branch intact (don't drop) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2019-12-13 12:58:46 +01:00
Markus Heiser
ef85943e06 Makefile: add documentation build targets docs & docs-live 
BTW:
- add build & dist folder to .gitignore
- justify indentation of build messages (makefile.python)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2019-12-12 12:10:32 +01:00
Markus Heiser
bb2b9b8da4 utils: add makefile.sphinx to the boilerplate 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2019-12-12 11:48:51 +01:00
Markus Heiser
b55a8004ed boilerplate: add inital Makefile with run & install targets 
Add *Makefile* boilerplate useful for python projects.  All python tasks are
using a virtualenv from ./local/py3

$ make help
  run       - run developer instance
  install   - developer install (./local)
  uninstall - uninstall (./local)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2019-11-28 19:54:57 +01:00
Dalf
c7220b40ea Suggestions: use RawTextQuery to make the suggestions URLs. Update all themes accordingly. 2019-07-17 09:05:29 +02:00
Ivan Skytte Jørgensen
0b7f0bde62 Use a more human-friendly format in searx/data/engines_languages.json (#1399) 
Instead of a single line with 500000 characters use nicely formatted JSON.
Sort the lists in engine_languages.py so when updating it is possible to
more easily see the differences (search engines do change the order their
languages are listed in)
 2018-11-29 16:31:05 +01:00
Alex
50c836864a fetch_firefox_version.py : compatible with Python 3 and minor fixes. 2018-08-05 10:55:42 +02:00
Alexandre Flament
066bd916bf [mod] fetch firefox versions in a standalone script 2018-08-05 10:10:15 +02:00
Marc Abonce Seguin
772c048d01 refactor engine's search language handling 
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.

Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
 2018-03-27 00:08:03 -06:00
Marc Abonce Seguin
d1eae9359f fix fetch_langauges to be more accurate 
Add languages supported by either all default general engines or 10 engines.
 2018-03-20 17:58:20 -06:00
cclauss
9984c99665
print() is a function in Python 3 2017-11-14 06:48:54 +01:00
marc
44085e31d0 update engines_languages.json and languages.py 
Also, fix fetch_languages.py so it can run on python3.
 2017-10-10 16:53:28 -05:00
Thirnearez
076cfe25d7 handle input carefully in shell scripts 
- prevent whitespace-splitting of variable expansions
- prevent interpretation of values as flags/options

(mostly)
 2017-10-06 21:00:22 +02:00
NK
cda9f8593f Avoid UnicodeEncodeError when redirecting standalone output 2017-07-25 13:39:11 +01:00
marc
805fb02ed1 add language support for qwant 
closes issue #863
 2017-03-18 23:44:21 +01:00
Alexandre Flament
369ec017b7 [enh] standalone_seax.py is a command line interface to searx with JSON output. 2017-01-04 14:14:01 +01:00
Alexandre Flament
7bcdc27a2e [fix] fix standalone_search.py 2017-01-03 19:15:55 +01:00
marc
1175b3906f change language list to only include languages with a minimum of engines 
that support them.
users can still query lesser supported through the :lang_code bang.
 2016-12-29 01:55:30 -06:00
marc
4a1ff56389 minor fixes in utils/fetch_languages.py 2016-12-16 22:14:14 -06:00