Commit Graph

4252 Commits (666bf1d32b7ce53133025c5471746e494deeeade)
 

Author SHA1 Message Date
dalf 2f8a708481 Update searx.data - update_wikidata_units.py 3 years ago
dalf d9dc3376d0 Update searx.data - update_languages.py 3 years ago
dalf 2857473553 Update searx.data - update_ahmia_blacklist.py 3 years ago
Alexandre Flament 10ecc303c9
Fix integration.yml
Don't run twice the workflow on PR opened from the same repository.
3 years ago
Alexandre Flament 245f8626dd
Fix data-update.yml, run once a month 3 years ago
Alexandre Flament d9a35fc28e
Fix data-update.yml 3 years ago
Alexandre Flament 917dff3aab
Update data-update.yml
Allow to manually running the workflow

See:
* https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
* https://docs.github.com/en/actions/reference/events-that-trigger-workflows#workflow_dispatch
3 years ago
Alexandre Flament b97273df6b
Update data-update.yml
Remove cache
3 years ago
Alexandre Flament aaae9a209e
Merge pull request #2600 from dalf/searx-extra
Add searx_extra package
3 years ago
Alexandre Flament b8cd326464 Add searx_extra package
Split the utils directory into:
* searx_extra contains update scripts, standalone_searx.py
* utils contains the files to build and setup searx.
3 years ago
Alexandre Flament 1d10ae175c
Merge pull request #2618 from thezeroalpha/master
Fix security vulnerabilities in provided nginx configuration
3 years ago
Alexandre Flament 6ba37777f7
Merge pull request #2623 from return42/fix-centos7
[fix] add package `which` to CentOS-7 boilerplate
3 years ago
Alexandre Flament d26261c5ab
Merge pull request #2619 from return42/drop-ubu1604
[mod] Drop Ubuntu 16.04 (Xenial Xerus) support
3 years ago
Alexandre Flament 4c2a8aea39
Merge pull request #2620 from return42/fix-git
[fix] support git versions <v2.22
3 years ago
Alex Balgavy 8736f5bd70 Use $host in nginx morty.conf template 3 years ago
Alexandre Flament aac37f288f
Merge pull request #2593 from dalf/update-autocomplete
Update autocomplete
3 years ago
Noémi Ványi 111d38cd8f
Merge pull request #2621 from return42/fix-searx.sh
[fix] utils/serax.sh create_pyenv() - drop duplicate 'pip install .'
3 years ago
Markus Heiser c355bc3481 [fix] add package 'which' to CentOS-7 boilerplate
Newer CentOS-7 images from https://images.linuxcontainers.org do no longer
include the which command.

Issue:

    $ sudo -H ./utils/lxc.sh cmd searx-centos7 ./utils/filtron.sh install all
    INFO:  [searx-centos7] ./utils/filtron.sh install all
    ...
    Install Go in user's HOME
    -------------------------

    download and install go binary ..
    ...
    -bash: line 1: which: command not found
    -->|ERROR - Go Installation not found in PATH!?!
    -bash: line 2: which: command not found

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 2e58988191 [fix] utils/serax.sh create_pyenv() - drop duplicate 'pip install .'
The wrong and unnecessary `pip install .` is executed in /usr/local/searx and is
responsible for the error message:

    ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: /usr/local/searx

The correct pip-install comes right after changing to `cd ${SEARX_SRC}`.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 237b1cfdb3 [fix] support git versions <v2.22
LTS distros like Ubuntu 18.04 do not ship a up-to-date version of git.::

    $ sudo -H ./utils/lxc.sh cmd searx-ubu1804 git --version
    ...
    git version 2.17.1

The option `--show-current` was added in git v2.22, the alternative to this
option is::

    git rev-parse --abbrev-ref HEAD

Issue when using option `--show-current`::

    [searx-ubu1804] Clone searx sources
    [searx-ubu1804] -------------------
    [searx-ubu1804]
    [searx-ubu1804] error: unknown option `show-current'
    [searx-ubu1804] usage: git branch [<options>] [-r | -a] [--merged | --no-merged]
    ....

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser daa4b676df [mod] Drop Ubuntu 16.04 (Xenial Xerus) support
EOL of Ubuntu 16.04 (Xenial Xerus) is in April 2021 but we do not support 16.04
since a longer time.  Issues are comming from dependencies (e.g. pip)::

    [searx-ubu1604]   |searx| SyntaxError: invalid syntax
    [searx-ubu1604]   |searx| Traceback (most recent call last):
    [searx-ubu1604]   |searx|   File "/usr/local/searx/searx-pyenv/bin/pip", line 7, in <module>
    [searx-ubu1604]   |searx|     from pip._internal.cli.main import main
    [searx-ubu1604]   |searx|   File "/usr/local/searx/searx-pyenv/lib/python3.5/site-packages/pip/_internal/cli/main.py", line 60
    [searx-ubu1604]   |searx|     sys.stderr.write(f"ERROR: {exc}")
    [searx-ubu1604]   |searx|                                    ^
    [searx-ubu1604]   |searx| SyntaxError: invalid syntax

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alex Balgavy 6b59800dc6 Fix security vulnerabilities in suggested nginx configuration
The suggested configurations for nginx found in the documentation and
templates lead to vulnerabilities allowing host spoofing [1] and path
traversal [2], as reported by Gixy [3]. This commit fixes those issues.

[1] https://github.com/yandex/gixy/blob/master/docs/en/plugins/hostspoofing.md
[2] https://github.com/yandex/gixy/blob/master/docs/en/plugins/aliastraversal.md
[3] https://github.com/yandex/gixy
3 years ago
Alexandre Flament c748fc66cf
Merge pull request #2610 from searx/dependabot/pip/master/pylint-2.7.2
Bump pylint from 2.6.2 to 2.7.2
3 years ago
dependabot[bot] b54dd66ef8
Bump pylint from 2.6.2 to 2.7.2
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.6.2 to 2.7.2.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Changelog](https://github.com/PyCQA/pylint/blob/master/ChangeLog)
- [Commits](https://github.com/PyCQA/pylint/compare/pylint-2.6.2...pylint-2.7.2)

Signed-off-by: dependabot[bot] <support@github.com>
3 years ago
Alexandre Flament 87ac6df9fc
Merge pull request #2611 from return42/google-scolar
[enh] google scholar - python implementation of the engine
3 years ago
Alexandre Flament 63f17d2e4c [enh] autocomplete refactoring, autocomplete on external bangs 3 years ago
Markus Heiser d48e2e7b0b [enh] google scholar - python implementation of the engine
The old xpath configuration for google scholar did not work and is replaced by a
python implementation.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament 4fa1290c11 [fix] answers: don't crash when the query is an empty string 3 years ago
Alexandre Flament e2fb500892
Merge pull request #2608 from return42/unittest2
[py2to3] use unittest from py3, remove unittest2 from py2
3 years ago
Alexandre Flament 0c663e25fc
Merge pull request #2604 from searx/update_data_firefox_version
Update searx.data - firefox_version
3 years ago
Alexandre Flament f77983e174
Merge pull request #2602 from MarcAbonce/fix-bing-fetch-languages
Fix fetch_languages for Bing
3 years ago
GazoilKerozen 5f6ac3afa2
Add Freesound engine (#2596)
Add freesound engine with player.

Co-authored-by: Gazoil <maildeguzel@gmail.com>
3 years ago
Markus Heiser 3bae35940a [py2to3] use unittest from py3, remove unittest2 from py2
- unittest2 is a backport of the new features added to the unittest testing
  framework in Python 2.7

- unittest2 was only needed in py2 and can be dropped now

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament b05f4d0664
Merge pull request #2605 from searx/update_data_currencies
Update searx.data - currencies
3 years ago
Alexandre Flament aec5188b51
Merge pull request #2606 from searx/update_data_wikidata_units
Update searx.data - wikidata_units
3 years ago
Alexandre Flament 28ed5acde5
Merge pull request #2607 from searx/update_data_ahmia_blacklist
Update searx.data - ahmia_blacklist
3 years ago
dalf 893b6e4901 Update searx.data - ahmia_blacklist 3 years ago
dalf 7b9005df31 Update searx.data - wikidata_units 3 years ago
dalf 4c8ae5b7ed Update searx.data - firefox_version 3 years ago
dalf d2778b5efe Update searx.data - currencies 3 years ago
Marc Abonce Seguin d6681fd33b remove articles number from engines_languages.json 3 years ago
Marc Abonce Seguin 9b6ffed061 fix fetch_languages for bing
Bing has a list of regions that it supports and some of these regions
may have more than one possible language.

In some cases, like Switzerland, these languages are always shown as
options, so there is no issue. But in other cases, like Andorra, Bing
will only show one language at the time, either the region's default or
the request's language if the latter is supported by that region.

For example, if the HTTP request is in French, Andorra will appear as
fr-AD but if the same page is requested in any other language Andorra
will appear as ca-AD.

This is specially a problem when Bing assumes that the request is in
English because it overrides enough language codes to make several major
languages like Arabic dissappear from the languages.py file.

To avoid that issue, I set the Accept-Language header to a language
that's only supported in one region to hopefully avoid these overrides.
3 years ago
Alexandre Flament 728e096764
data-update.yml: on PR per file 3 years ago
Noémi Ványi bc590cbc47
Merge pull request #2592 from dalf/update-external-bangs
[mod] add utils/fetch_external_bangs.py
3 years ago
Alexandre Flament 7c1847d5f2 [mod] add utils/fetch_external_bangs.py
Based on duckduckgo bangs
Store bangs on a trie to allow autocomplete (not in this commit)
3 years ago
Alexandre Flament 606aa79e49
Merge pull request #2584 from searx/dependabot/pip/master/sphinx-3.5.1
Bump sphinx from 3.4.1 to 3.5.1
3 years ago
dependabot[bot] 048be86aaf
Bump sphinx from 3.4.1 to 3.5.1
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 3.4.1 to 3.5.1.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/3.x/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v3.4.1...v3.5.1)

Signed-off-by: dependabot[bot] <support@github.com>
3 years ago
Alexandre Flament 5f4a085fc4
Merge pull request #2595 from dalf/update-wikidata-units
[mod] update wikidata_units.json and fetch_wikidata_units.py
3 years ago
Alexandre Flament 94c9320c6f
Merge pull request #2585 from dalf/update-currencies
[mod] update currencies.json and fetch_currencies.py
3 years ago
Alexandre Flament 46ca32c3cc [mod] update currencies.json and fetch_currencies.py
use a sparql request on wikidata to get the list of currencies.

currencies.json contains the translation for all supported searx languages.

Supersede #993
3 years ago