Commit Graph

32 Commits (master)

Author SHA1 Message Date
ArtikusHG 1f8f8c1e91 Replace langdetect with fasttext 1 year ago
Alexandre Flament e473addaff User agent: don't include the patch number in the Firefox version
The Firefox version in the user agent doesn't include the patch version: 106.0 not 106.0.2

Close #1914
2 years ago
Markus Heiser 9933155a2e [fix] update_osm_keys_tags.py: sort JSON dump
To get meaningful diff, the keys in JSON dump needs to be sorted.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser ba8959ad7c [fix] typos / reported by @kianmeng in searx PR-3366
[PR-3366] https://github.com/searx/searx/pull/3366

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Alexandre Flament 578b2a8183 fix searxng_extra/update/update*.py scripts
call searx.locales.locales_initialize before using LOCALE_NAMES

Related to https://github.com/searxng/searxng/pull/1306
2 years ago
Markus Heiser ef4239c68a [doc] fix some leftovers from ad964562c
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser e8541b6006 [theme] peel out oscar from SearXNG development
This is the first step of removing oscar theme

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 62982c8812 [fix] add back missing languages & regions (followup of PR #1071)
In PR #1071 the language catalog of dailymotion has been cleaned up, before
there had been over 7000 "languages" in the catalog.

As a side effect of this clean-up the language & region catalog in SearXNG has
been reduced [1].

This patch reduce the ``min_engines_per_lang`` from 13 to 12 to get the missed
languages back in language & region catalog of SearXNG.

[1] 3bb62823ec (diff-f3f00db0f87f95b882624a192e0aac21525638af0b18c9514e765fcf1991678d)

Requested-by: @tiekoetter in a Matrix chat
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser effcde3d0e [fix] add missing territory (country) name
Related-to: https://github.com/searxng/searxng/pull/1029#issuecomment-1086824911
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Alexandre Flament 0379856712
Merge pull request #967 from return42/language-filter
[mod] add flags to the languages filter
2 years ago
Markus Heiser 34fd2021d8 [fix] pylint issue in py3.10
searxng_extra/update/update_firefox_version.py:16:0: W0402:
Uses of a deprecated module 'distutils.version' (deprecated-module)

[1] https://github.com/searxng/searxng/pull/1007

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 2e4557f3f3 [fix] languages: show country name even if there is only one country
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser a25e3767d4 [fix] don't show flags for languages without region identifier
SearXNG shows two different things:

region:
  "de-CH" is the equivalent of "Schweiz (de)" in DDG.

languages:
  "en" doesn't say anything about the location. It is up the engines to do their
  best to select English results without a region.

Suggested-by: @dalf https://github.com/searxng/searxng/pull/967#issuecomment-1072979693
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 2841abaf55 [mod] add flags to the languages filter
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Alexandre Flament dbe3eaabc9 Info: code refactoring & bug fixes 2 years ago
Alexandre Flament 1157462ff9 Various change on PR 930 2 years ago
Markus Heiser b1912607ae [mod] replace /help by /info pages and include pages in project docs
This patch implements a bolierplate to share content from info-pages of the
SearXNG instance (URL /info) with the project documentation (path /docs/user).

The info pages are using Markdown (CommonMark), to include them in the project
documentation (reST) the myst-parser [1] is used in the Sphinx-doc build chain.

If base_url is known (defined in settings.yml) links to the instance are also
inserted into the project documentation::

    searxng_extra/docs_prebuild

[1] https://www.sphinx-doc.org/en/master/usage/markdown.html

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 5ec9eb8030 [mod] switch default theme from oscar to simple
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 7cdd31440e [fix] external bangs: don't overwrite Bangs in data trie
Bangs with a `*` suffix (e.g. `!!d*`) overwrite Bangs with the same
prefix (e.g. `!!d`) [1].  This can be avoid when a non printable character is
used to tag a LEAF_KEY.

[1] https://github.com/searxng/searxng/pull/740#issuecomment-1010411888

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 295876abaa [pylint] add scripts from searxng_extra/update to pylint
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser ffea5d8ef5 [docs] add documentation for the scripts in searxng_extra/update
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser de819bb1f5 [docs] move searxng_extra/ stuff to the developer section.
Stuff in folder searxng_extra/ is not suitable for normal users and should only
be used by developers.

The script searxng_extra/standalone_searx.py must not give the impression that
it improves privacy. [1]

[1] https://github.com/searxng/searxng/pull/651#issuecomment-1001389726

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 8191e1a253 [fix] update_languages.py: generate code that passes CI
File searx/languages.py, created by update_languages.py has to pass quality
check from CI::

    make format.python

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 8a07559ab5 [fix] update_languages.py: no excption on unknown locale & language
Fix exception handling of unknown locales and languages::

    ERROR: ca_ES_valencia --> [Errno 2] No such file or directory: 'local/py3/lib/python3.8/site-packages/babel/locale-data/ca_ES_valencia.dat'
    ERROR: languages['fil-PH'] --> {'name': None, 'english_name': None}
    ERROR: languages['nb-NO'] --> {'name': None, 'english_name': None}

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 3d96a9839a [format.python] initial formatting of the python code
This patch was generated by black [1]::

    make format.python

[1] https://github.com/psf/black

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser fcdc2c2cd2 [format.python] disable py code formatting for some hunks of code
Disable the python code formatting from python-black, where the readability of
code suffers by formatting.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Alexandre Flament 9215d4cde7 [fix] python searxng_extra/standalone_searx.py --help
The scripts loads the engine and then parse the arguments.
After that the engines are initialized.

Before this commit, with "--help", the HTTP clients are closed before
the engine initialization threads. When these init threads try to send
an HTTP request, an exception is raised.
2 years ago
Alexandre Flament 811837d109 [remove] searxng_extra/google_search.py
remove deprecated script
2 years ago
Alexandre Flament 56e6d19b48
update_firefox_version.py: update user agent signature
The user agent from Windows is
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:95.0) Gecko/20100101 Firefox/95.0

See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent/Firefox#windows
2 years ago
Alexandre Flament 828088fa5a [mod] update_languages: min_engines_per_country=7
a (language,country) tuple is included if 7 engines have it, was 10 before.

close #432
3 years ago
Markus Heiser 955eab8240 [mod] searxng_extras - minor improvements
- fix docs/searxng_extra/standalone_searx.py.rst
- add SPDX tag
- pylint standalone_searx.py and update_wikidata_units.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament 1bb82a6b54 SearXNG: searxng_extra 3 years ago