Commit Graph

8140 Commits

Author SHA1 Message Date
Markus Heiser
7ab577a1fb [mod] Revision of the favicon solution
All favicons implementations have been documented and moved to the Python
package:

    searx.favicons

There is a configuration (based on Pydantic) for the favicons and all its
components:

    searx.favicons.config

A solution for caching favicons has been implemented:

    searx.favicon.cache

If the favicon is already in the cache, the returned URL is a data URL [1]
(something like `data:image/png;base64,...`).  By generating a data url from
the FaviconCache, additional HTTP roundtripps via the favicon_proxy are saved:

    favicons.proxy.favicon_url

The favicon proxy service now sets a HTTP header "Cache-Control: max-age=...":

    favicons.proxy.favicon_proxy

The resolvers now also provide the mime type (data, mime):

    searx.favicon.resolvers

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-10-05 08:18:28 +02:00
Markus Heiser
c49a2707c1 [mod] sqlitedb: access to SQLite databases a little more convenient.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-10-05 08:18:28 +02:00
Brock Vojkovic
e17d7632d0 [feat] add favicons to result urls 2024-10-05 08:18:28 +02:00
searxng-bot
3e747d0491 [l10n] update translations from Weblate
b59b0d937 - 2024-10-04 - ghose <ghose@users.noreply.translate.codeberg.org>
770781548 - 2024-10-04 - return42 <return42@users.noreply.translate.codeberg.org>
0d5f72f8c - 2024-10-04 - return42 <return42@users.noreply.translate.codeberg.org>
b14fd01b1 - 2024-10-04 - return42 <return42@users.noreply.translate.codeberg.org>
e457a6b1c - 2024-10-03 - SomeTr <SomeTr@users.noreply.translate.codeberg.org>
328b93af4 - 2024-10-03 - Fjuro <fjuro@alius.cz>
2024-10-04 09:18:26 +02:00
Grant Lanham
44a06190bb [refactor] unit tests to utilize paramaterized and break down monolithic tests
- for tests which perform the same arrange/act/assert pattern but with different
  data, the data portion has been moved to the ``paramaterized.expand`` fields

- for monolithic tests which performed multiple arrange/act/asserts,
  they have been broken up into different unit tests.

- when possible, change generic assert statements to more concise
  asserts (i.e. ``assertIsNone``)

This work ultimately is focused on creating smaller and more concise tests.
While paramaterized may make adding new configurations for existing tests
easier, that is just a beneficial side effect.  The main benefit is that smaller
tests are easier to reason about, meaning they are easier to debug when they
start failing.  This improves the developer experience in debugging what went
wrong when refactoring the project.

Total number of tests went from 192 -> 259; or, broke apart larger tests into 69
more concise ones.
2024-10-03 13:20:32 +02:00
dependabot[bot]
042c7190e6 [upd] pypi: Bump sphinx-autobuild from 2021.3.14 to 2024.10.3
Bumps [sphinx-autobuild](https://github.com/sphinx-doc/sphinx-autobuild) from 2021.3.14 to 2024.10.3.
- [Release notes](https://github.com/sphinx-doc/sphinx-autobuild/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx-autobuild/blob/main/NEWS.rst)
- [Commits](https://github.com/sphinx-doc/sphinx-autobuild/compare/2021.03.14...2024.10.03)

---
updated-dependencies:
- dependency-name: sphinx-autobuild
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-03 13:18:03 +02:00
Markus Heiser
2fd6730d4f [mod] py3.8 EOL / upgrade to actions/setup-python@v5
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-10-03 13:05:43 +02:00
Markus Heiser
e7a4d7d7c3 [doc] slightly improve documentation of SQL engines
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-10-03 13:04:06 +02:00
Grant Lanham
2a29e16d25 [feat] implement mariadb engine 2024-10-03 13:04:06 +02:00
Markus Heiser
d48f04e809 [build] /static 2024-10-03 11:05:46 +02:00
Bnyro
421c131707 [refactor] simple theme: *.less - replace hardcoded colors
Closes: https://github.com/searxng/searxng/issues/3842
2024-10-03 11:05:46 +02:00
Bnyro
b42ce34ca8 [build] /static 2024-10-03 10:53:17 +02:00
Bnyro
e4b2823abd [feat] simple theme: pure black theme style 2024-10-03 10:53:17 +02:00
Austin-Olacsi
cbf1e90979 add get_embeded_stream_url to searx.utils 2024-10-03 07:10:53 +02:00
Markus Heiser
f07ab6deb0 [data] update searx.data - update_engine_traits.py
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-09-29 14:29:22 +02:00
0xhtml
0a0fb450b5 [enh] engine: stract - add language/region support 2024-09-29 14:29:22 +02:00
return42
b5009b8610 [data] update searx.data - update_engine_descriptions.py 2024-09-29 13:17:43 +02:00
return42
d6b04d3ba1 [data] update searx.data - update_engine_traits.py 2024-09-29 13:17:06 +02:00
return42
bc6ee05422 [data] update searx.data - update_currencies.py 2024-09-29 13:16:45 +02:00
return42
a4558dda47 [data] update searx.data - update_firefox_version.py 2024-09-29 13:16:25 +02:00
return42
eb31eaaba0 [data] update searx.data - update_ahmia_blacklist.py 2024-09-29 13:15:57 +02:00
return42
609ead9ffe [data] update searx.data - update_wikidata_units.py 2024-09-29 13:14:53 +02:00
dependabot[bot]
f95a5effcc [upd] pypi: Bump selenium from 4.24.0 to 4.25.0
Bumps [selenium](https://github.com/SeleniumHQ/Selenium) from 4.24.0 to 4.25.0.
- [Release notes](https://github.com/SeleniumHQ/Selenium/releases)
- [Commits](https://github.com/SeleniumHQ/Selenium/compare/selenium-4.24.0...selenium-4.25.0)

---
updated-dependencies:
- dependency-name: selenium
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-28 08:50:35 +02:00
searxng-bot
9bae26a106 [l10n] update translations from Weblate
7f8a5a94e - 2024-09-24 - abhabongse <abhabongse@users.noreply.translate.codeberg.org>
d69f2d929 - 2024-09-23 - kratos <kratos@users.noreply.translate.codeberg.org>
f7f0c0f94 - 2024-09-23 - kratos <kratos@users.noreply.translate.codeberg.org>
52bea48dc - 2024-09-23 - AndersNordh <AndersNordh@users.noreply.translate.codeberg.org>
fe6659955 - 2024-09-22 - tentsbet <tentsbet@users.noreply.translate.codeberg.org>
69a05cbae - 2024-09-22 - Linerly <Linerly@users.noreply.translate.codeberg.org>
2024-09-28 08:49:10 +02:00
dependabot[bot]
940da05f03 [upd] pypi: Bump linuxdoc from 20240509 to 20240924
Bumps [linuxdoc](https://github.com/return42/linuxdoc) from 20240509 to 20240924.
- [Changelog](https://github.com/return42/linuxdoc/blob/master/CHANGELOG)
- [Commits](https://github.com/return42/linuxdoc/commits)

---
updated-dependencies:
- dependency-name: linuxdoc
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-28 08:38:22 +02:00
Grant Lanham
6a3375be37 [fix] use get accessor to pull desc from bing_images 2024-09-26 07:26:51 +02:00
Zhijie He
6be56aee11 add Cloudflare AI Gateway engine
add Cloudflare AI Gateway engine

add settings for Cloudflare AI Gateway engine

set utf8 encode for data, fix non english char cause 500 error

format json data

fixed indentation and config format error

fix line-length limitation in CI

reformatted code for CI

reformatted code for CI

limit system prompts to less 120 chars

cleanup unused variable & format code
2024-09-23 07:02:10 +02:00
Grant Lanham
14241e7dac Add paramaterized with example of refactor
reduce test name size

fix imports
2024-09-22 08:03:02 +02:00
searxng-bot
ea16c82d78 [l10n] update translations from Weblate
7174f090f - 2024-09-15 - MPBDev <MPBDev@users.noreply.translate.codeberg.org>
c7c76552e - 2024-09-14 - return42 <return42@users.noreply.translate.codeberg.org>
9b9ec6361 - 2024-09-14 - kny5 <kny5@users.noreply.translate.codeberg.org>
af19df705 - 2024-09-14 - return42 <return42@users.noreply.translate.codeberg.org>
cc658f945 - 2024-09-15 - Fjuro <fjuro@alius.cz>
15a59fda5 - 2024-09-14 - return42 <return42@users.noreply.translate.codeberg.org>
2024-09-22 07:32:01 +02:00
dwitterer
915cf9b7af Update own-instance.rst
Implemented the suggested changes.
2024-09-22 07:28:32 +02:00
dwitterer
ba4942ea51 Update own-instance.rst
Improved English grammar and punctuation for professionalism.
2024-09-22 07:28:32 +02:00
Markus
0b3724651e [fix] simple template: drop useless `aria-labelledby` attributes
This patch removes the ``aria-labelledby`` attributes for which there is no tag
with the corresponding ID.

Reported-by: @glanham-jr https://github.com/searxng/searxng/issues/3793#issuecomment-2351689483
Signed-off-by: Markus <markus@venom.fritz.box>
2024-09-21 15:23:21 +02:00
Markus
5ad0214bd4 [fix] simple template: macro checkbox_onoff_reversed
In its previous implementation, the macro ``checkbox_onoff_reversed`` always
created an ``aria-labelledby`` attribute, even if there was no descriptive tag
with the generated ID (used as the value of the ``aria-labelledby``).

Before this patch, the Nu-HTML-Checker [1] reported 255 issues of this type::

    The aria-labelledby attribute must point to an element in the same document. (255)

[1] https://validator.w3.org/nu/

Signed-off-by: Markus <markus@venom.fritz.box>
2024-09-21 15:23:21 +02:00
Markus
8b8d830fd3 [fix] simple template: add ID to elements used for aria-labelledby
The ``aria-labelledby`` [1] attribute identifies the element that labels the
element it is applied to.  The templates ``infinite_scroll.html`` and
``search_on_category_select.html`` define a ``aria-labelledby`` at the <input>
tag but miss the id in the <div> with the description.

[1] https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Attributes/aria-labelledby

Signed-off-by: Markus <markus@venom.fritz.box>
2024-09-21 15:23:21 +02:00
Markus
67fcf3cc67 [fix] Removes `/>` ending tags for void HTML elements
continuation of commit: 0b832f19b

Signed-off-by: Markus <markus@venom.fritz.box>
2024-09-21 15:23:21 +02:00
Markus Heiser
d026486ce3 [fix] scripts: elimination of limitations on dedicated distributions
The restriction of shell scripts to certain distributions is only required for
certain actions such as the installation of a SearXNG instance. The maintenance
scripts and build processes were previously also restricted to these specific
distributions.  With this patch, the build processes (such as the build of
online documentation) can now also be executed on all Linux distributions.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-09-19 09:35:54 +02:00
Grant Lanham
0b832f19bf [fix] Removes `/>` ending tags for void HTML elements
Removes ``/>`` ending tags for void elements [1] and replaces them with ``>``.
Part of the larger cleanup to cleanup invalid HTML throughout the codebase [2].

[1] https://html.spec.whatwg.org/multipage/syntax.html#void-elements
[2] https://github.com/searxng/searxng/issues/3793
2024-09-15 15:19:51 +02:00
Markus
28dc623785 [fix] drop engine alexandria.org
The origin alexandria.org is broken:

  https://www.alexandria.org/?c=&r=&a=0&q=foo

returns "504 Gateway Time-out"

- Closes: https://github.com/searxng/searxng/issues/3786

Signed-off-by: Markus <markus@venom.fritz.box>
2024-09-15 14:45:48 +02:00
Markus
3630e464b3 [fix] drop engine gpodder
gpodder is ultra slow on search terms like foo

  https://gpodder.net/search.json?q=foo

takes up to a minute to return an empty json response.

- Closes: https://github.com/searxng/searxng/issues/3785
Signed-off-by: Markus <markus@venom.fritz.box>
2024-09-15 14:45:38 +02:00
Markus
d3a795c7e7 [fix] engine: qwant - detect captchaUrl and raise SearxEngineCaptchaException
So far a CAPTCHA was not recognized in the response of the qwant engine and a
SearxEngineAPIException was raised by mistake.  With this patch a CAPTCHA
redirect is recognized and the correct SearxEngineCaptchaException is raised.

Closes: https://github.com/searxng/searxng/issues/3806
Signed-off-by: Markus <markus@venom.fritz.box>
2024-09-15 14:45:23 +02:00
Markus
55e2f4a97f [data] update searx.data - update_engine_traits.py 2024-09-15 12:48:35 +02:00
Markus
cdb4927b8b [fix] fetch_traits: brave, google, annas_archive & radio_browser
This patch fixes a bug reported by CI "Fetch traits" [1] (brave) and improves
other fetch traits functions (google, annas_archive & radio_browser).

brave:

    File "/home/runner/work/searxng/searxng/searx/engines/brave.py", line 434, in fetch_traits
      sxng_tag = region_tag(babel.Locale.parse(ui_lang, sep='-'))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/runner/work/searxng/searxng/searx/locales.py", line 155, in region_tag
    Error:     raise ValueError('%s missed a territory')

google:

  change ERROR message about unknow UI language to INFO message

radio_browser:

  country_list contains duplicates that differ only in upper/lower case

annas_archive:

  for better diff; sort the persistence of the traits

[1] https://github.com/searxng/searxng/actions/runs/10606312371/job/29433352518#step:6:41

Signed-off-by: Markus <markus@venom.fritz.box>
2024-09-15 12:48:35 +02:00
Bnyro
84e2f9d46a [feat] gitlab: implement dedicated module
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2024-09-15 08:04:21 +02:00
searxng-bot
231e55f38d [l10n] update translations from Weblate
6bea94d18 - 2024-09-11 - sushen23 <sushen23@users.noreply.translate.codeberg.org>
771eedb39 - 2024-09-08 - 0ko <0ko@users.noreply.translate.codeberg.org>
b75ec6466 - 2024-09-08 - alexgabi <alexgabi@users.noreply.translate.codeberg.org>
2024-09-14 15:23:49 +02:00
Bnyro
33c1236923 [fix] search: crash when no engines are used to search 2024-09-14 11:13:53 +02:00
Finn Steffens
9e2bfe14db
[feat] engine: add right dao
* [feat] engine: add right dao

* [enh] right dao engine: allow additional classes

Allow additional classes while parsing to prevent the engine from breaking in the future if additional classes are added to the elements.

Co-authored-by: Bnyro <bnyro@tutanota.com>

---------

Co-authored-by: Bnyro <bnyro@tutanota.com>
2024-09-12 17:51:47 +02:00
Lucas Schwiderski
f05566d925 [fix] json_engine: Fix result fields being mixed up
Fixes #3810.
2024-09-12 10:47:08 +02:00
0xhtml
c45870dd71 [fix] yep engine: remove links to other engines
Yep includes links to search for the same query on Google and other
search engines as a result in the search result. This fix skips these
results.
2024-09-12 00:04:04 +02:00
searxng-bot
5cca3f6ef2 [l10n] update translations from Weblate
cab91b92c - 2024-09-05 - xtex <xtexchooser@duck.com>
bbc77a9c4 - 2024-09-05 - Priit Jõerüüt <jrtcdbrg@users.noreply.translate.codeberg.org>
16ab61e99 - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
34b1487b6 - 2024-09-04 - Erico <Erico@users.noreply.translate.codeberg.org>
811132898 - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
099cb381e - 2024-09-04 - return42 <return42@users.noreply.translate.codeberg.org>
eebb940df - 2024-09-04 - 0ko <0ko@users.noreply.translate.codeberg.org>
50c845d17 - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
f12addf27 - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
c0879d377 - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
7732d1356 - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
dbe33cfb1 - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
759dacddd - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
7607b9ac6 - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
aab0df3e7 - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
f9232fbd6 - 2024-09-04 - return42 <return42@users.noreply.translate.codeberg.org>
f3a339dac - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
4f3bdb5c9 - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
fb92cc42d - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
d6412e0c6 - 2024-09-05 - return42 <return42@users.noreply.translate.codeberg.org>
e1f6103fd - 2024-09-04 - Fjuro <fjuro@alius.cz>
4ee95b7f2 - 2024-09-03 - SomeTr <SomeTr@users.noreply.translate.codeberg.org>
84e4641d5 - 2024-09-04 - ghose <ghose@users.noreply.translate.codeberg.org>
2024-09-06 11:04:20 +02:00
dependabot[bot]
d2f36cacb3 [upd] pypi: Bump pylint from 3.2.6 to 3.2.7
Bumps [pylint](https://github.com/pylint-dev/pylint) from 3.2.6 to 3.2.7.
- [Release notes](https://github.com/pylint-dev/pylint/releases)
- [Commits](https://github.com/pylint-dev/pylint/compare/v3.2.6...v3.2.7)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-06 11:03:27 +02:00