Commit Graph

30 Commits (c2e4b95e8d68ac151d51fbcd3f0875f2aaa3640f)

Author SHA1 Message Date
Markus Heiser 3cdf2d6570 Update searx.data - update_languages.py
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
dalf c74b465ece Update searx.data - update_languages.py 3 years ago
Markus Heiser f63ffbb22b [fix] engine - yahoo: rewrite and fix issues
Languages are supported by mapping the language to a domain.  If domain is not
found in :py:obj:`lang2domain` URL ``<lang>.search.yahoo.com`` is used.

BTW: fix issue reported at https://github.com/searx/searx/issues/3020

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 97419a770d [upd] make data.languages
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
dalf db51a64390 Update searx.data - update_languages.py 3 years ago
dalf 4d3f2f48d2 Update searx.data - update_languages.py 3 years ago
Markus Heiser ef6e1bd6b9 [fix] Qwant engines - implement API v3 and add 'quant videos'
The implementation uses the Qwant API (https://api.qwant.com/v3). The API is
undocumented but can be reverse engineered by reading the network log of
https://www.qwant.com/ queries.

This implementation is used by different qwant engines in the settings.yml::

  - name: qwant
    categories: general
    ...
  - name: qwant news
    categories: news
    ...
  - name: qwant images
    categories: images
    ...
  - name: qwant videos
    categories: videos
    ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament cf2b549967 Update engines_languages.json 3 years ago
Marc Abonce Seguin 448bfe6005 fix Qwant's fetch_languages function 3 years ago
dalf c0668d248e Update searx.data - update_languages.py 3 years ago
dalf d9dc3376d0 Update searx.data - update_languages.py 3 years ago
Markus Heiser d48e2e7b0b [enh] google scholar - python implementation of the engine
The old xpath configuration for google scholar did not work and is replaced by a
python implementation.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser ff6804e545 [data] make engines.languages
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Marc Abonce Seguin ecf5899153 fetch google's search langs rather than ui langs 4 years ago
Marc Abonce Seguin c86504b47a fix utils/fetch_languages to work with new languages 4 years ago
Michael Ilsaas a1ce141c99
add peertube engine (#2109) 4 years ago
Markus Heiser 79412828cd update languages: engines_languages.json languages.py
build by Makefile target:

    make project

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser 40843fe95a searx/data/engines_languages.json: sort json file (no content change)
To get meaningfull diffs, the json file has to be sorted.  Before applying any
further content patch, the json file needs a inital sort (without changing any
content).

Sorted by::

  import sys, json

  with open('engines_languages.json') as f:
      j = json.load(f)

  with open('engines_languages.json', 'w') as f:
      json.dump(j, f, indent=2, sort_keys=True)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Marc Abonce Seguin 0169b63e84 [fix] fetch google's supported languages 5 years ago
Ivan Skytte Jørgensen 0b7f0bde62 Use a more human-friendly format in searx/data/engines_languages.json (#1399)
Instead of a single line with 500000 characters use nicely formatted JSON.
Sort the lists in engine_languages.py so when updating it is possible to
more easily see the differences (search engines do change the order their
languages are listed in)
6 years ago
Marc Abonce Seguin 772c048d01 refactor engine's search language handling
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.

Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
6 years ago
Marc Abonce Seguin d1eae9359f fix fetch_langauges to be more accurate
Add languages supported by either all default general engines or 10 engines.
6 years ago
marc 44085e31d0 update engines_languages.json and languages.py
Also, fix fetch_languages.py so it can run on python3.
7 years ago
marc 405e5c8f24 [fix] duckduckgo images doesn't fail with countryless language 7 years ago
marc fd65c12921 make search language handling less strict
languages.py can change, so users may query on a language that is not
on the list anymore, even if it is still recognized by a few engines.

also made no and nb the same because they seem to return the same,
though most engines will only support one or the other.
7 years ago
marc 805fb02ed1 add language support for qwant
closes issue #863
7 years ago
marc 1175b3906f change language list to only include languages with a minimum of engines
that support them.
users can still query lesser supported through the :lang_code bang.
7 years ago
marc 4a1ff56389 minor fixes in utils/fetch_languages.py 8 years ago
marc af35eee10b tests for _fetch_supported_languages in engines
and refactor method to make it testable without making requests
8 years ago
marc f62ce21f50 [mod] fetch supported languages for several engines
utils/fetch_languages.py gets languages supported by each engine and
generates engines_languages.json with each engine's supported language.
8 years ago