Commit Graph

3576 Commits (91ca5d16137acc4943e67006c215b57a915a3d84)

Author SHA1 Message Date
Léon Tiekötter 37baf46ece [fix] Rename ccengine engine to openverse
The CC engine was merged with WordPress and renamed to Openverse

Source: https://wordpress.org/news/2021/05/welcome-to-openverse/
3 years ago
Léon Tiekötter 4be6deb0a1 [fix] ccengine engine
Change domain to api.openverse.engineering
3 years ago
Markus Heiser ced656606f
Merge pull request #709 from return42/drop-etools
[fix] drop etools engine module
3 years ago
Markus Heiser 5dd3442f83 [fix] drop etools engine module
The implementation of the etools engine is poor.  No date-range support, no
language support and it is broken by a CAPTCHA.

etools is a metasearch engine, the major search engines it supports (google,
bing, wikipedia, Yahoo) are already available in SeaarXNG.

While etools does support several engines we currently don't support directly,
support for them should be added directly to SearXNG if there is demand.

In practice: in SearXNG the worse etools results will be mixed with good results
from other engines we have (as long as there is no captcha).

At best case, what we win with etools is in e.g. results from de.ask.com in a
query from a german request .. in all other cases worse results are bubble up in
SearXNG's result list.

[1] https://github.com/searxng/searxng/issues/696#issuecomment-1005855499

Closes: https://github.com/searxng/searxng/issues/696
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Martin Fischer e12525a1fa
Merge pull request #708 from not-my-profile/pref-refactor
Refactor `preferences`
3 years ago
Markus Heiser 2eb95f1532
Merge pull request #718 from searxng/dependabot/pip/master/pygments-2.11.2
Bump pygments from 2.11.0 to 2.11.2
3 years ago
Markus Heiser c956ed6c43 [build] /static 3 years ago
Markus Heiser d8db85638e make pygments.less
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
searxng-bot 651342d71a [translations] update from Weblate
7781625e - 2022-01-06 - Linerly <linerly@protonmail.com>
dcab2ef1 - 2022-01-06 - Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Léon Tiekötter 3ab826de22 Drop microsoft academic engine
Microsoft academic was discontinued on 2021-12-31.

Source: https://www.microsoft.com/en-us/research/project/academic/articles/microsoft-academic-to-expand-horizons-with-community-driven-approach/
3 years ago
Martin Fischer 180d4d068b [refactor] refactor SwitchableSetting
The previous implementation used two hash sets and a list.
... that's not necessary ... a single hash map suffices.

And it's also less error prone ... because the previous data structure
allowed a setting to be enabled and disabled at the same time.
3 years ago
Martin Fischer 56fbf22108 [refactor] stop SwitchableSetting from subclassing Setting
Previously the default_value was abused for the cookie name.
Having SwitchableSetting subclass Setting doesn't even make sense
in the first place since none of the Setting methods apply.
3 years ago
Martin Fischer 1196abcfce
Merge pull request #706 from not-my-profile/remove-broken-quest-op
[fix] remove broken ? search operator
3 years ago
Martin Fischer a0486cf827 [fix] prevent leaflet.{css,js} from polluting git grep results
Fixes #704.
3 years ago
Martin Fischer 83f8a8fc6d [refactor] remove pointless tuple 3 years ago
Martin Fischer bb06758a7b [refactor] add type hints & remove Setting._post_init
Previously the Setting classes used a horrible _post_init
hack that prevented proper type checking.
3 years ago
Martin Fischer 61935c72ef [fix] remove broken ? search operator
The ? search operator has been broken for some time and
currently only raises the question why it's still there.

## Context ##

The query "Paris !images" searches for "Paris" in the "images" category.

Once upon a time Searx supported "Paris ?images" to search for "Paris"
in the currently enabled categories and the "images" category.

The feature makes sense ... the ? syntax does not.
We will hopefully introduce a +!images syntax in the future.

Fixes #702.
3 years ago
Alexandre Flament 03189d60f4
Merge pull request #697 from searxng/remove-nl-be
Delete searx/translations/nl_BE
3 years ago
Alexandre Flament aedd6279b3
Merge pull request #634 from not-my-profile/powered-by
Introduce `categories_as_tabs` & group engines in tabs
3 years ago
Alexandre Flament 40684f6b5e
Delete searx/translations/nl_BE
See https://github.com/searxng/searxng/issues/540
3 years ago
Markus Heiser 0ebad8220f
Merge pull request #688 from dalf/settings_enable_stats
[enh] settings.yml: implement general.enable_metrics
3 years ago
Alexandre Flament 9004e84b6e
Merge pull request #596 from dalf/upgrade-httpx
Upgrade httpx
3 years ago
Alexandre Flament d3ecadd3f8
Merge pull request #679 from dalf/brand-searxng
searxng.org: update setup.py & settings.yml
3 years ago
Alexandre Flament 2134703b4b [enh] settings.yml: implement general.enable_metrics
* allow not to record metrics (response time, etc...)
* this commit doesn't change the UI. If the metrics are disabled
  /stats and /stats/errors will return empty response.
  in /preferences, the columns response time and reliability will be empty.
3 years ago
Alexandre Flament e64c3deab7 [mod] upgrade httpx 0.21.2
httpx 0.21.2 and httpcore 0.14.4 fix multiple issues:
* https://github.com/encode/httpx/releases/tag/0.21.2
* https://github.com/encode/httpcore/releases/tag/0.14.4

so most of the workarounds in searx.network have been removed.
3 years ago
Martin Fischer a4c2cfb837 [enh] change categories_as_tabs from a list to a dict
The tab icon names are currently hard coded in the templates.
This commit lets us introduce an icon property in the future, e.g:

categories_as_tabs:
  general:
    icon: search-outline
3 years ago
Martin Fischer b38036d519 [fix] stop setting superfluous cookies for "non tab categories" 3 years ago
Martin Fischer 3dd534e5c0 [refactor] make group_engines_in_tab more readable 3 years ago
Martin Fischer d01e8aa8cc [mod] introduce searx.engines.Engine for type hinting 3 years ago
Martin Fischer 3f76c9c96c [enh] enable English dictionaries by default
These dictionaries are no longer part of the general category,
so they're no longer queried by default -> we can enable them
by default without degrading general query performance.
3 years ago
Martin Fischer abe53946d4 [enh] display help text for "Other" engines tab in preferences 3 years ago
Martin Fischer 6f07d51b62 [mod] improve variable name 3 years ago
Martin Fischer 1e195f5b95 [mod] move group_engines_in_tab to searx.webutils 3 years ago
Martin Fischer 5d74bf3820 [enh] move dictionaries, Erowid & IMDb out of general category
The general category is the category that is searched by default.
From a privacy standpoint it doesn't make sense to send all general
queries to specialized search engines that cannot deal with those
queries anyway.
3 years ago
Martin Fischer ab90e2ac49 [enh] show categories not in any tab category in "Other" preferences tab
Previously we didn't have a good place to put search engines that don't
fit into any of the tab categories. This commit automatically puts
search engines that don't belong to any tab category in an "other"
category, that is only displayed in the user preferences (and not above
search results).
3 years ago
Martin Fischer 4ac6b5d32d [build] /static 3 years ago
Martin Fischer a12cb6a807 [simple] improve styling of subcategory table headers in preferences 3 years ago
Martin Fischer 20a5e43e35 [enh] display language of engines in preferences 3 years ago
Martin Fischer 31e206361f [enh] group engines in preference tabs 3 years ago
Martin Fischer b02f762687 [enh] add more categories 3 years ago
Markus Heiser d7b4306444 [fix] links in comments - https://docs.searxng.org/
fixed links from https://searxng.github.io/searxng/ to https://docs.searxng.org/
3 years ago
Markus Heiser a6cfab93fa [enh] add redis connector searx/shared/redisdb.py
Add a redis connector, the default DB connector is a socket at::

    unix:///usr/local/searxng-redis/run/redis.sock?db=0

To set up a redis instance simply use::

    $ ./manage redis.build
    $ sudo -H ./manage redis.install

A hint for developers:

To get access rights to this instance, your developer account needs to be added
to the *searxng-redis* group::

    $ sudo -H ./manage redis.addgrp "${USER}"
    # don't forget to logout & login to get member of group

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament 41ae5aea2b [fix] disabling all engines in a category makes the bang search in general
with this commit, there is no result
3 years ago
Martin Fischer 8e9ad1ccc2 [enh] introduce categories_as_tabs
Previously all categories were displayed as search engine tabs.
This commit changes that so that only the categories listed under
categories_as_tabs in settings.yml are displayed.

This lets us introduce more categories without cluttering up the UI.
Categories not displayed as tabs  can still be searched with !bangs.
3 years ago
Martin Fischer df34b1ddcf [enh] settings.yml: allow granular overwrites for about 3 years ago
Martin Fischer 8a0810cedc [fix] bang syntax for categories with spaces 3 years ago
Alexandre Flament 458b7fb72d
Merge pull request #610 from return42/img-load-error
[fix] simple theme: hide the image if img load fails
3 years ago
Alexandre Flament d83aa2b0d2
Merge pull request #613 from return42/pylint-bing-images
[pylint] Bing (Images) engine
3 years ago
Alexandre Flament 76cbfbbdda reference docs.searxng.org 3 years ago
Markus Heiser 3cdf2d6570 Update searx.data - update_languages.py
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 21d7c8b367
Merge pull request #676 from return42/fix-bing-lang
Fix issues on running update_languages.py
3 years ago
Markus Heiser 61ce0c2244 [fix] bing engines: fetch_supported_languages
The Request to and the Response from https://www.bing.com/account/general has
been changed.

[1] https://github.com/searxng/searxng/pull/672#discussion_r777104919

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament 6b49f32d3e
Merge pull request #670 from searxng/update_data_update_firefox_version.py
Update searx.data - update_firefox_version.py
3 years ago
Markus Heiser 3dcba2f67c
Merge pull request #671 from searxng/update_data_update_ahmia_blacklist.py
Update searx.data - update_ahmia_blacklist.py
3 years ago
Markus Heiser 877293c470
Merge pull request #673 from searxng/update_data_update_currencies.py
Update searx.data - update_currencies.py
3 years ago
dalf 127da09fc9 Update searx.data - update_wikidata_units.py 3 years ago
dalf 291483af53 Update searx.data - update_currencies.py 3 years ago
dalf a0016250ec Update searx.data - update_ahmia_blacklist.py 3 years ago
dalf 184242d36b Update searx.data - update_firefox_version.py 3 years ago
Markus Heiser 021b4a0a02
Merge pull request #667 from searxng/dependabot/pip/master/pygments-2.11.0
Bump pygments from 2.10.0 to 2.11.0
3 years ago
Markus Heiser efbcaaab3b [build] /static 3 years ago
Markus Heiser 391e4b57c9 make pygments.less
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
searxng-bot db4885f7d4 [translations] update from Weblate
055a6080 - 2021-12-28 - Markus Heiser <markus.heiser@darmarit.de>
d028dfd7 - 2021-12-26 - Gert-dev <gert+github@disroot.org>
d2f3b998 - 2021-12-24 - Markus Heiser <markus.heiser@darmarit.de>
1b882f56 - 2021-12-29 - VaiTon <eyadlorenzo@gmail.com>
f8e524a3 - 2021-12-27 - Markus Heiser <markus.heiser@darmarit.de>
70eafa9c - 2021-12-26 - Frederik From <frederikfrom@pm.me>
7eb20cec - 2021-12-28 - Markus Heiser <markus.heiser@darmarit.de>
3cb754a4 - 2021-12-25 - Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser f087d01e24 [build] /static 3 years ago
Markus Heiser f75199b1ec [fix] simple theme: hide the image if img load fails
Add event listener to query selector::

    '#urls img.image'

From the user point of view, I think it is better to hide the image:
img_load_error.svg is helplful in the image category because it still allows to
select the image. IMO, in the news category, the fact there is a missing image
won't help to choose the links. From a developer point of view, the place holder
is signal that may be the engine needs to be updated (at least give a look). The
browser console should show the same information too, but it requires some
additional steps. [1]

[1] https://github.com/searxng/searxng/pull/610#issuecomment-997640132

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 7966fd3bbd
Merge pull request #663 from dalf/mod_secret_key
changes about the secret_key
3 years ago
Markus Heiser 8f3a7feb47 [mod] implement is_hmac_of() in webutils / close to new_hmac()
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>, Alexandre Flament
3 years ago
Markus Heiser dc4f1f705d [pylint] Bing (Images) engine
Fix remarks from pylint and remove obsolete try/except block

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 6d7a38a912 [pylint] Bing (Videos) engine
Fix remarks from pylint and remove obsolete try/except block

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament 7d4834ac4d [mod] webutils.py: remove dead code
secret_key can't be bytes (see settings_default.py)
3 years ago
Alexandre Flament d784870209 [fix] use hmac.compare_digest instead of ==
see https://docs.python.org/3/library/hmac.html#hmac.HMAC.hexdigest
3 years ago
Markus Heiser d84226bf63 [fix] issues reported by pylint
Fix pylint issues from commit (3d96a983)

    [format.python] initial formatting of the python code

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 3d96a9839a [format.python] initial formatting of the python code
This patch was generated by black [1]::

    make format.python

[1] https://github.com/psf/black

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser fcdc2c2cd2 [format.python] disable py code formatting for some hunks of code
Disable the python code formatting from python-black, where the readability of
code suffers by formatting.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
searxng-bot 568a4df297 [translations] update from Weblate
14044898 - 2021-12-19 - Linerly <linerly@protonmail.com>
2db5cc5d - 2021-12-17 - Markus Heiser <markus.heiser@darmarit.de>
7273ced4 - 2021-12-22 - Markus Heiser <markus.heiser@darmarit.de>
bac8f580 - 2021-12-19 - Stephan Wijman <s.wijman@mailarena.net>
472c882a - 2021-12-17 - Markus Heiser <markus.heiser@darmarit.de>
bc2c60a4 - 2021-12-17 - Markus Heiser <markus.heiser@darmarit.de>
11f37c00 - 2021-12-17 - Markus Heiser <markus.heiser@darmarit.de>
10417e8a - 2021-12-22 - Mico Hautaluoma <m@mha.fi>
12d49b4b - 2021-12-22 - Markus Heiser <markus.heiser@darmarit.de>
a1bfd8f6 - 2021-12-23 - Markus Heiser <markus.heiser@darmarit.de>
fa94d205 - 2021-12-17 - Markus Heiser <markus.heiser@darmarit.de>
6a121417 - 2021-12-21 - YFdyh000 <yfdyh000@gmail.com>
bc9b6947 - 2021-12-18 - Shahed Hosseini <sajjadhusseini92@gmail.com>
3 years ago
Martin Fischer 19339067c6 [del] remove geektimes (redirects to habrahabr) 3 years ago
Martin Fischer fb037e7a83 [del] remove whaleslide (website is down)
Closes #639.
3 years ago
Martin Fischer e28c6bda35 [doc] introduce about.language and sort engines by it 3 years ago
Markus Heiser c5d604f06f [upd] engine_descriptions.json
Updated by:

    ./manage pyenv.cmd python3 \
        ./searxng_extra/update/update_engine_descriptions.py

Include documentation of new added engines:

- superuser
- stackoverflow
- askubuntu
- indb
- ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 6fb6e8c22a
Merge pull request #621 from not-my-profile/engines-table
[doc] Improve engines table
3 years ago
Martin Fischer 1fb1cca519 [doc] fix about.website for hoogle 3 years ago
Markus Heiser 7a215e07e7
Merge pull request #611 from return42/fix-bing
[fix] bing engine: fix paging support, show inital page.
3 years ago
Martin Fischer 06beffdf31 [doc] add about.website for wiby 3 years ago
Alexandre Flament b5d4cac7e1
Merge pull request #615 from return42/pylint-reddit
[pylint] Reddit engine
3 years ago
mrpaulblack c6b76aa429 [build] /static 3 years ago
mrpaulblack 414f1039d2 [fix] theme: build `img_load_error.svg` as svg 3 years ago
Markus Heiser 2af50c2588 [pylint] Reddit engine
Add Reddit engine to pylint process

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 6b85607274 [fix] bing engine: fix paging support, show inital page.
Follow up queries for the pages needed to be fixed.

- Split search-term in one for initial query and one for following queries.
- Set some headers in HTTP requests, bing needs for paging support.
- IMO //div[@class="sa_cc"] does no longer match in a bing response.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser b2177e5916 [pylint] Bing (Web) engine
Fix remarks from pylint and improved code-style.  In preparation for a bug-fix
of the Bing (Web) engine I add this engine to the pylint-list.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament 4da1e0026c
Merge pull request #609 from return42/fix-bing-news
[fix] engine bing-news: replace the http:// by https://
3 years ago
Markus Heiser f41734a543 [fix] engine bing-news: replace the http:// by https://
BTW: add bing_news to the pylint process

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Alexandre Flament f6b1264b5f
Merge pull request #584 from return42/drop-css-vendor
[mod] simple-theme: remove vendor CSS
3 years ago
searxng-bot 3cf9065478 [translations] update
1a891c16 - 2021-12-15 - Markus Heiser <markus.heiser@darmarit.de>
33bd07e7 - 2021-12-14 - Alexandre Flament <alex@al-f.net>
7ba24343 - 2021-12-12 - Markus Heiser <markus.heiser@darmarit.de>
80d54b89 - 2021-12-15 - FilGiov <quota679@gmail.com>
af9e007c - 2021-12-15 - Markus Heiser <markus.heiser@darmarit.de>
9d4e645c - 2021-12-15 - Markus Heiser <markus.heiser@darmarit.de>
21dec08c - 2021-12-15 - Shahed Hosseini <sajjadhusseini92@gmail.com>
3 years ago
Alexandre Flament 6bb6f7ffc1
Merge pull request #602 from return42/fix-600
[fix] move engines gpodder and AZlyrics into category music
3 years ago
Markus Heiser 07f2a9ee3c [fix] engine: MediathekViewWeb (mvw) disable by default
The MediathekViewWeb delivers only content for the german speaking area.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 84836a5000 [build] /static 3 years ago
Markus Heiser c020b90056 [mod] simple theme: remove vendor prefix of `width: max-content`
Remove no longer needed workarounds like `width: 1000px;`
and vendor prefix of max-content [1].

[1] https://developer.mozilla.org/en-US/docs/Web/CSS/max-content#browser_compatibility

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 459f33a18e [mod] simple-theme: remove vendor CSS
flexbox
  Browser support for flexbox is excellent, and the majority of browsers do not
  need a prefix at this point. Safari was the last of the major browsers to
  remove prefixes, with the release of Safari 9 in 2015. [1]

user-select:
  Vendor prefix of user-select is not needed, see 'Browser compatibility' [2].

[1] https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Flexible_Box_Layout/Backwards_Compatibility_of_Flexbox#status_in_browsers
[2] https://developer.mozilla.org/en-US/docs/Web/CSS/user-select#browser_compatibility

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago
Markus Heiser 715f578cda [build] /static 3 years ago
Markus Heiser c416464949 [fix] lazy loading of <img> tags
Images should include dimension attributes.  Without `width` and `height`
specified, image dimensions are 0×0 pixels at first.  ...  In this case the
browser determines that all of them are visible to the user and decides to load
everything [1].

In CSS the `width` is set to a value and the `height` is unsed to scale the image
proportional in both dimensions.

[1] https://web.dev/browser-level-image-lazy-loading/#images-should-include-dimension-attributes
[2] https://caniuse.com/loading-lazy-attr

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
3 years ago