Commit Graph

1414 Commits (master)

Author SHA1 Message Date
Ahmad Alkadri 7fc8d72889 [fix] bing: parsing result; check to see if the element contains links
This patch is to hardening the parsing of the bing response:

1. To fix [2087] check if the selected result item contains a link, otherwise
   skip result item and continue in the result loop.  Increment the result
   pointer when a result has been added / the enumerate that counts for skipped
   items is no longer valid when result items are skipped.

   To test the bugfix use:   ``!bi :all cerbot``

2. Limit the XPath selection of result items to direct children nodes (list
   items ``li``) of the ordered list (``ol``).

   To test the selector use: ``!bi :en pontiac aztek wiki``

   .. in the result list you should find the wikipedia entry on top,
   compare [2068]

[2087] https://github.com/searxng/searxng/issues/2087
[2068] https://github.com/searxng/searxng/issues/2068
1 year ago
ahmad-alkadri 9ee99423fe [fix] Bing-Web engine: XPath to get the wikipedia result
Modify the XPath selector to get the wikipedia result plus small fixes.

About result content: especially with the Wikipedia result, we'd get several
paragraph elements, only the first paragraph would be taken and displayed on the
search result
1 year ago
Rudis Muiznieks 128b8c7f0a
Add HTTP Referer header to DuckDuckGo requests
closes #2080
1 year ago
Rudis Muiznieks 6804ff048d
Fix: add trailing slash to duckduckgo url
Close #1854
1 year ago
Alexandre Flament 269326063a Fix: don't crash when engine or name is missing in settings.yml
SearXNG crashes if the engine or name fields are missing.
With this commit, the app displays an error in the log and keeps loading.

Close #1951
1 year ago
Émilien Devos 46ad32343a Switch back to protobuf for raw HTML 2 years ago
ngosang 78be4b4c70 Fix Google search engine.
- Fix broken links. Resolves #1794
- Fix missing results. Resolves #1829
2 years ago
Alexandre Flament 8f19bdaf17
Merge pull request #1882 from fehho/metacpan
Add MetaCPAN engine
2 years ago
fehho fe351c2802 Add MetaCPAN engine 2 years ago
Vasilis Gerakaris 947b62c9d5
Fix floating point format in DDG weather humidity
Fixes #1836
2 years ago
Alexandre FLAMENT 035bc507ec [fix] startpage engine 2 years ago
Alexandre Flament a3148e5115
Merge pull request #1814 from return42/fix-typos
[fix] typos / reported by @kianmeng in searx PR-3366
2 years ago
Alexandre Flament 0e00af9c26
Merge pull request #1810 from return42/fix-1809
[fix] springer: unsupported operand type(s) for +: 'NoneType' and 'str'
2 years ago
Markus Heiser ba8959ad7c [fix] typos / reported by @kianmeng in searx PR-3366
[PR-3366] https://github.com/searx/searx/pull/3366

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 52023e3d6e [fix] doc of the paper.html template (isbn, issn)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 0052887929 [fix] springer: unsupported operand type(s) for +: 'NoneType' and 'str'
- fix issue reported #1809
- filter out `None` value from issn and isbn list
- add comments (from publicationName)
- add publisher

Closes: https://github.com/searxng/searxng/issues/1809
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser e36b023508 [mod] core.ac.uk: add cetgory 'scientific publications'
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Alexandre Flament 16443d4f4a [mod] core.ac.uk: try multiple ways to get url
If the url is not found, using:
* the DOI
* the downloadUrl
* the ARK id
2 years ago
Markus Heiser c76830d8a8 [mod] core.ac.uk: use paper.html template
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 3ff2ad939d [fix] ERROR searx.engines.core.ac.uk: list index out of range
Some result items from core.ac.uk do not have an URL::

  Traceback (most recent call last):
  File "searx/search/processors/online.py", line 154, in search
    search_results = self._search_basic(query, params)
  File "searx/search/processors/online.py", line 142, in _search_basic
    return self.engine.response(response)
  File "SearXNG/searx/engines/core.py", line 73, in response
    'url': source['urls'][0].replace('http://', 'https://', 1),

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Alexandre Flament d6446be38f [mod] science category: various update of about PR 1705 2 years ago
Alexandre FLAMENT e36f85b836 Science category: update the engines
* use the paper.html template
* fetch more data from the engines
* add crossref.py
2 years ago
Alexandre Flament bef3984d03
Merge pull request #1728 from liimee/eng-ddw
add duckduckgo weather engine
2 years ago
Alexandre Flament d3fec1388c
Merge pull request #1624 from liimee/eng-wttr
Add wttr.in engine
2 years ago
Alexandre Flament 1a7b6872b5
Merge pull request #1792 from unixfox/google-images-internal-api
use the internal API for google images
2 years ago
Markus Heiser cf7ee67f71 [mod] google-images: slightly improvements of the engine
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Emilien Devos df5f8d0e8e use the internal API for google images 2 years ago
Markus Heiser dcf1d408a5 [fix] google-news: origin result does not have a content area
The google news are in a rework, the content area of a news item has been
removed.

Closes: https://github.com/searxng/searxng/issues/1790
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser fbf07237ff [fix] and improve docs generated from source code.
Fix::

    searx/locales.py:docstring of searx.locales.get_engine_locale:17: \
      WARNING: Definition list ends without a blank line; unexpected unindent.

Improvement: don't show default values in the generated documentation whe it is
more a mess than a usefull information (`:meta hide-value:`).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Alexandre FLAMENT dd0887be18 xpath engine: change raise_for_httperror to no_result_for_http_status
no_result_for_http_status contains a list of HTTP status.
These HTTP status are seen an empty result list.
In other cases an exception is thrown as usual.

Previously raise_for_httperror were ignoring all HTTP error,
which make defective engines invisible in the stats.
2 years ago
Markus Heiser a15dfa5ee1 [fix] engine woxikon.de - don't raise exception on empty result list
Woxikon expects a word in German, so with query "foo" the site finds nothing and
respons a 404:

    httpx.HTTPStatusError: Client error '404 Not Found' \
      for url 'https://synonyme.woxikon.de/synonyme/foo.php'

[1] https://github.com/searxng/searxng/issues/1543#issuecomment-1193317054

Closes: https://github.com/searxng/searxng/issues/1543
Suggested-by: @allendema [1]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2 years ago
Markus Heiser 8e9fb0b435
Merge pull request #1647 from return42/deepl-engine
[mod] add deepl translation engine
2 years ago
ta 85b5293e40 simplify infobox result 2 years ago
ta 12f7d4a46b add duckduckgo weather engine 2 years ago
Alexandre Flament 56000d5162
Merge pull request #1699 from liimee/eng-app-store
add apple app store engine
2 years ago
Alexandre Flament 44bc94c36e
Merge pull request #1700 from liimee/eng-ddm
add apple maps engine
2 years ago
ta 5057007270 remove thumbnail from results 2 years ago
ta 525946d7dd add poi's website and phone number, doesn't crash when there is no `displayMapRegion`, query the token on the first request 2 years ago
ta 5dce299b22 add apple maps engine 2 years ago
ta cef7bbab22 get the not cropped version of the thumbnail when the image height is not too important 2 years ago
ta 78bff4618c add safesearch support 2 years ago
ta bcae7ae4e3 add developer info as author 2 years ago
ta e5c1b64b1d add the apple app store engine
The Apple App Store is the digital app distribution platform for iOS & iPadOS.
2 years ago
ta 040e24f9ad support playing videos directly 2 years ago
ta 79d06509c1 add tags as suggestions 2 years ago
ta d22f469010 use `invalid-name` instead of `C0103` for pylint 2 years ago
ta dd9127492f add 9gag engine
9GAG is a social media website where users upload and share user-generated images and videos
2 years ago
ta e64cca8c3f don't raise error when nothing was found 2 years ago
M Asenov faa32d5773 fixed xpath selector for appropriate results 2 years ago
Alexandre Flament 5ed40af3ba
Merge pull request #1661 from liimee/eng-tw
Add twitter engine
2 years ago