Markus Heiser
16f8ec894a
[fix] revise google images engine
...
this commit is picked from #1985
4 years ago
Markus Heiser
410c2f903d
[fix] revise google engine
...
this commit is picked from #1985
4 years ago
Markus Heiser
8d318ee142
Merge branch 'master' into gigablast
4 years ago
Sophie Tauchert
71db7b1238
Fix YaCy text results returned as images
4 years ago
Noémi Ványi
93cbd85b8a
Merge branch 'master' into duckduckgo_correction
4 years ago
Markus Heiser
5fac6cffa2
Merge branch 'master' into gigablast
4 years ago
Markus Heiser
5293e58032
[fix] yahoo engine - changed content_xpath
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser
223430ff30
Merge branch 'master' into gigablast
4 years ago
Adam Tauber
32f7877235
[fix] resolve flickr_noapi encoding issues
4 years ago
Gordon Quad
385e9b5c9e
add correction support for duckduckgo
4 years ago
Markus Heiser
ee0da61cbb
Merge branch 'master' into gigablast
4 years ago
Adam Tauber
aa7c043ff4
[fix] resolve pep8 errors
4 years ago
Adam Tauber
29960aa1d9
[enh] add official site link to the top of the infobox - closes #1644
4 years ago
Adam Tauber
6c06286251
[enh] add length and author details to youtube videos
...
closes #775
4 years ago
Adam Tauber
2c6531b233
[enh] add routing directions to osm search - closes #254
4 years ago
Markus Heiser
74135007eb
Merge branch 'master' into gigablast
4 years ago
Noémi Ványi
e3282748d0
add display_error_messages option to engine settings
...
A new option is added to engines to hide error messages from users. It
is called `display_error_messages` and by default it is set to `True`.
If it is set to `False` error messages do not show up on the UI.
Keep in mind that engines are still suspended if needed regardless of
this setting.
Closes #1828
4 years ago
Markus Heiser
ee5d2b319b
[fix] gigablast requires a random extra parameter
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser
a18760b322
[fix] revise of the gigablast engine (WIP)
...
The gigablast API has changed and seems to have some quirks, this is the first
revise. More work (hacks) are needed.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser
57c7b90edd
[fix] gigablast does no longer support *supported_languages_url*
...
Since there are zero results, we can remove it:
$ make engines.languages
fetch languages ..
...
fetched 0 languages from engine gigablast
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser
de179ecc5b
[fix] remove debug print from commit e5305f8
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser
9302d1fc17
Merge branch 'master' into master
4 years ago
Noémi Ványi
fcb44c6542
Merge branch 'master' into fix_startpage_ValueError_on_spanish_datetime
4 years ago
HLFH
3a26093c46
Remove discontinued faroo engine
4 years ago
Spühler Stefan
4f90fb6a92
[Fix] Startpage ValueError on Spanish date format
...
datetime.parser.parse() does not know the Spanish date format which
leads to a ValueError. Fixes #1870
Traceback (most recent call last):
File "/usr/local/searx/searx/search.py", line 160, in search_one_http_request_safe
search_results = search_one_http_request(engine, query, request_params)
File "/usr/local/searx/searx/search.py", line 97, in search_one_http_request
return engine.response(response)
File "/usr/local/searx/searx/engines/startpage.py", line 102, in response
published_date = parser.parse(date_string, dayfirst=True)
File "/usr/local/searx/searx-ve/lib/python3.6/site-packages/dateutil/parser/_parser.py", line 1358, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "/usr/local/searx/searx-ve/lib/python3.6/site-packages/dateutil/parser/_parser.py", line 649, in parse
raise ValueError("Unknown string format:", timestr)
ValueError: ('Unknown string format:', '24 Ene 2013')
5 years ago
Markus Heiser
ad7a6e6e10
bugfix(!biv) : bing-video do not like "older" User-Agents
...
When selecting other languages than 'en', bing-video did not handle the language
correct and gave very bad results. Since User-Agent is normaly rotated in
searx, the behavior of a !biv search was unpredictable and paging was broken.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
5 years ago
Markus Heiser
1c853f9573
bing_news: parital rollback of c89c05bc
...
The bing_news bug (discussed in #1838 ) was caused by wrong language tags, which
was fixed e0c99d9d
/ no need to change the bing_news search string.
closes: https://github.com/asciimoo/searx/issues/1838
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
5 years ago
Markus Heiser
e0c99d9dcb
bugfix: fetch_supported_languages bing, -news, -videos, -images
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
5 years ago
Markus Heiser
c89c05bceb
bugfix: google-news and bing-news has changed the language parameter
...
closes: https://github.com/asciimoo/searx/issues/1838
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
5 years ago
piplongrun
f0684a5bb5
Add eTools engine
5 years ago
Noémi Ványi
99435381a8
[enh] introduce private engines
...
This PR adds a new setting to engines named `tokens`.
It expects a list of tokens which lets searx validate
if the request should be accepted or not.
5 years ago
frankdelange
db9d7d47bd
Fix double-encode error ( fixes #1799 )
5 years ago
Adam Tauber
17b6faa4c3
[fix] pep8
5 years ago
Adam Tauber
ad5bb994b1
[fix] add py3 compatibility
5 years ago
Adam Tauber
1e6253ce16
[fix] handle empty response
5 years ago
Adam Tauber
86a378bd01
[fix] handle missing thumbnail
5 years ago
Adam Tauber
2dc2e1e8f9
[fix] skip invalid encoded attributes
5 years ago
Adam Tauber
2292e6e130
[fix] handle missing result size
5 years ago
Markus Heiser
36e72a4619
Merge branch 'master' into fix-engine-spotify
5 years ago
Marc Abonce Seguin
5706c12fba
remove empty parenthesis in wikipedia's summary
...
They're usually IPA pronunciations which are removed
by the API.
5 years ago
Marc Abonce Seguin
c18048e045
exclude disambiguation pages from wikipedia infobox
5 years ago
Adam Tauber
34ad3d6b34
[enh] display error message if gigablast extra param expired
5 years ago
Adam Tauber
fc457569f7
[fix] pep8
5 years ago
Adam Tauber
00512e36c1
[fix] handle empty response from wikipedia engine - closes #1114
5 years ago
Adam Tauber
f8713512be
[fix] convert byte query to string in osm engine - fixes #1220
5 years ago
Adam Tauber
e5305f886c
[fix] fetch extra search param of gigablast - fixes #1293
5 years ago
Adam Tauber
8850036ded
[fix] add explicit useragent header to requests - closes #1459
5 years ago
Marc Abonce Seguin
ccaf6ca02c
[fix] update xpaths for new google results page
5 years ago
Adam Tauber
731e34299d
Merge pull request #1744 from dalf/optimizations
...
[mod] speed optimization
5 years ago
Adam Tauber
574cb25a16
Merge pull request #1758 from return42/ddd-fix
...
[fix] duckduckgo_definitions
5 years ago
Markus Heiser
30ad0c666d
duckduckgo_definitions: remove the debug message
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
5 years ago
Adam Tauber
20da8f2cbf
Merge pull request #1754 from MarcAbonce/seedpeer
...
Add Seedpeer again
5 years ago
Markus Heiser
b6d9f5aa71
[fix] duckduckgo_definition issues reported by 'manage.sh test'
...
Fix this error while travis build::
/home/travis/build/asciimoo/searx/searx/engines/duckduckgo_definitions.py:21:44: E225 missing whitespace around operator
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
5 years ago
Markus Heiser
4998e9ec85
[fix] duckduckgo_definitions - where 'AnswerType' is 'calc'
...
Do not try to get text when 'AnswerType' is 'calc'.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
5 years ago
Markus Heiser
2aa95c16e3
[fix] soundcloud: URLs of JS sources has been moved
...
The client_id is found under (new) URL:
https://a-v2.sndcdn.com/assets/49-a0c01933-3.js
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
5 years ago
Adam Tauber
789d71350d
Merge pull request #1745 from lorddavidiii/python3.8-fix
...
Fix python 3.8 compatibility
5 years ago
Adam Tauber
05033ea8d8
Merge pull request #1689 from MarcAbonce/images_fixes
...
[fix] Google Images
5 years ago
Marc Abonce Seguin
9299355570
add seedpeer again
5 years ago
Emilien Devos
8f51430f5c
[fix] Force Google old UI with a new user agent
5 years ago
lorddavidiii
5e5ff0cbf8
webapp.py: use html.escape if cgi.escape is not available
...
- cgi.escape was removed in python 3.8
- also use html.escape in framalibre.py
5 years ago
Dalf
85b3723345
[mod] speed optimization
...
compile XPath only once
avoid redundant call to urlparse
get_locale(webapp.py): avoid useless call to request.accept_languages.best_match
5 years ago
Noémi Ványi
5796dc60c9
fix pep 8 check
5 years ago
Noémi Ványi
a6f20caf32
add initial support for offline engines && command engine
5 years ago
Adam Tauber
7d8fd4b95e
[fix] pep8
5 years ago
Adam Tauber
bbe4442a86
[fix] update gigablast engine
5 years ago
Adam Tauber
1057e42cfd
[fix] update digg engine
5 years ago
Adam Tauber
7177c9e12f
[fix] update devianart engine
5 years ago
Adam Tauber
6ca1622378
[fix] update 1x engine
5 years ago
Adam Tauber
c98a2df36d
[fix] enable paging support for arxiv engine
5 years ago
Adam Tauber
ed1c1bdb04
[fix] pep8
5 years ago
Adam Tauber
77a70fe541
[fix] update startpage engine - closes #1601
5 years ago
Adam Tauber
94ea9d6622
[fix] duckduckgo paging - closes #1677
5 years ago
Marc Abonce Seguin
bb4d223770
[fix] google images
5 years ago
Léo Bourrel
88261e111c
Fix bing engine results count ( #1387 )
...
This PR fixes the result count from bing which was throwing an (hidden) error and add a validation to avoid reading more results than avalaible.
For example :
If there is 100 results from some search and we try to get results from 120 to 130, Bing will send back the results from 0 to 10 and no error. If we compare results count with the first parameter of the request we can avoid this "invalid" results.
5 years ago
Dalf
1cee2c1796
[fix] bing engine
...
before this commit, sometimes there are no results
use a generic user-agent instead of one with the OS "Windows NT 6.3; WOW64"
5 years ago
Dalf
fcc9587ee9
[fix] fdroid engine
5 years ago
Dalf
fbf6b689dd
[fix] dictzone engine
5 years ago
Dalf
9ff5001816
[fix] arxiv engine
5 years ago
Alexandre Flament
2179079a91
[fix] fix flickr_noapi decoding ( #1655 )
...
Characters that were not ASCII were incorrectly decoded.
Add an helper function: searx.utils.ecma_unescape (Python implementation of unescape Javascript function).
5 years ago
cy8aer
4dc792e1e2
[enh] add invidious engine. ( #1657 )
...
closes #1372
5 years ago
0xhtml
b2e1ee8d35
Fix some more errors with none/wrong credentials
5 years ago
0xhtml
275b37cc7c
Fix error if the user hasn't set api credentials
5 years ago
0xhtml
c329ea135e
Fix spotify engine
5 years ago
Dalf
0c032c8429
[fix] youtube_noapi engine: fix the title
5 years ago
Dalf
8b7ac56669
[fix] google_videos engine: some results don't a thumbnail
5 years ago
Dalf
d44677e226
[fix] dailymotion engine: remove HTML tags from the description
5 years ago
Dalf
6e0285b2db
[fix] wikidata engine: faster processing, remove one HTTP redirection.
...
* Search URL is https://www.wikidata.org/w/index.php?{query}&ns0=1 (with ns0=1 at the end to avoid an HTTP redirection)
* url_detail: remove the disabletidy=1 deprecated parameter
* Add eval_xpath function: compile once for all xpath.
* Add get_id_cache: retrieve all HTML with an id, avoid the slow to procress dynamic xpath '//div[@id="{propertyid}"]'.replace('{propertyid}')
* Create an etree.HTMLParser() instead of using the global one (see #1575 )
5 years ago
Frank de Lange
cbc5e13275
[enh] flickr_noapi: use complete JSON data block, add 'content', 'img_format', 'source', etc. ( #1571 )
...
Fetch complete JSON data block, use legend to extract images.
Unquote urlencoded strings.
Add image description as 'content'.
Add 'img_format' and 'source' data (needs PR #1567 to enable this data to be displayed).
Show images which lack ownerid instead of discarding them.
5 years ago
Frank de Lange
204a2cbbf0
[fix] bing_videos ( #1579 )
...
use JSON where possible, compose 'content' using all available data, use correct 'url' (direct to source instead of redirect through bing)
5 years ago
Dalf
23611897ec
[fix] make sure then engine name is lower case
...
Minor fix: "%s engine initialized" display the right engine name
5 years ago
Frank de Lange
11fc9913e9
[enh] bing_images: use data from embedded JSON to improve results (e.g. real page title) ( #1568 )
...
use data from embedded JSON to improve results (e.g. real page title), add image format and source info (see PR #1567 ), improve paging logic (it now works)
5 years ago
Alexandre Flament
f34b5cedb1
[fix] fixes google play engines ( #1651 )
...
update commit 87baa74a86
5 years ago
volth
eb182df132
[mod] restore btdigg engine as btdig.com ( #1515 )
5 years ago
rachmadani haryono
3b1122c5fa
[fix] fix duden engine ( #1594 )
5 years ago
Venca24
87baa74a86
[fix] fixes google play engines and adds thumbnails to their results ( #1612 )
...
fix google play apps, google play apps, google play music engines
xpath engine: thumbnail_xpath can define an optional thumbnail
5 years ago
Dalf
da0ce5880f
[fix] fix soundcloud engine, speed up searx start time
5 years ago
Dalf
45702b77ca
embedded iframe (youtube, dailymotion, vimeo): use https
5 years ago
Emilien Devos
cbd1ebdce8
[fix] Force Google old UI ( #1597 )
5 years ago
Frank de Lange
4b7332286a
Use string formatter to create source and img_format labels ( #1566 )
...
google_images : use JSON embedded in HTML (engine expected pure JSON)
5 years ago
Dalf
ffe0972f91
Remove some engines : subtitleseeker, seedpeer, swisscows
...
http://www.subtitleseeker.com and http://www.seedpeer.eu don't exist anymore.
https://swisscows.ch/ has change : the engine needs to be updated
5 years ago
Alexandre Flament
df2b9a76f7
Merge branch 'master' into ne/fix-google-image-search
5 years ago
Nick Espig
1c6ab79b9f
Fix google image search
...
- Because there is not full image url in the dom, we replace "image_url" with the same url as the "url" (url of source).
See example HTML https://gist.github.com/Nachtalb/2dea8a4d2c723c49226ad9645838121f
- Remove unused import
- Fix google image search title
- Keep google image safe value up to date
5 years ago
Marc Abonce Seguin
3e1c2153f7
[fix] duckduckgo images requests
5 years ago
Marc Abonce Seguin
f2d49a6971
[fix] get youtube results from js object
...
Results are not appearing in the html document anymore,
instead they are found inside an object embedded in a script.
6 years ago
Jonas Zohren
f7bdd827c4
[enh] adds apkmirror search engine
6 years ago
Léo Bourrel
bf4a38ad66
Remove asksteem
6 years ago
d-tux
f1814079f0
Merge branch 'master' into engines/unsplash
6 years ago
Marc Abonce Seguin
626a8e9ac9
[fix] unicode error with WolframAlpha API engine
6 years ago
d-tux
329172f66e
Merge branch 'master' into engines/unsplash
6 years ago
Noémi Ványi
97351a2c72
fix after rebase
6 years ago
Noémi Ványi
b63d645a52
Revert "remove 'all' option from search languages"
...
This reverts commit 4d1770398a
.
6 years ago
Marc Abonce Seguin
0169b63e84
[fix] fetch google's supported languages
6 years ago
Marc Abonce Seguin
51111c2594
[fix] always set langauge_aliases even if it's empty
6 years ago
Marc Abonce Seguin
5568f24d6c
[fix] check language aliases when setting search language
6 years ago
Venca24
2456b8f571
[mod] google videos
6 years ago
Noémi Ványi
1ea56576dc
Merge branch 'master' into devel_google_videos
6 years ago
Noémi Ványi
aeb6dab187
Merge branch 'master' into master
6 years ago
Noémi Ványi
5b81f7c2cc
Merge branch 'master' into bugfix_startpage
6 years ago
Venca24
cf26aba93b
[FIX] google videos thumbnails
6 years ago
Léo Bourrel
73ec2ba74e
Update bing image engine with new image json format
6 years ago
Michael Pfitzner
44ce51f0c5
restore startpage search results
6 years ago
d-tux
4a127b19de
Merge branch 'master' into engines/unsplash
6 years ago
dimqua
0d86ed9c7e
update startpage.py
6 years ago
dimqua
442772a6c5
remove findx engine ( #1452 )
6 years ago
Venca24
0bdf090ed7
[fix] google videos engine
6 years ago
Denis Wernert
b9ada93b3a
Removes what looks like tracking parameters
6 years ago
Denis Wernert
72d063d27d
Uses the raw url for the image result, rather than the full
...
size result.
6 years ago
Denis Wernert
cd7849ffc8
Adds the Unsplash image engine
6 years ago
Adam Tauber
052a71d05a
Merge pull request #1368 from aliceinwire/gentoo_engine
...
Add Gentoo wiki search engine
6 years ago
Alice Ferrazzi
d24c58cf4e
Fix pep8 errors
6 years ago
Alice Ferrazzi
ff2eac9f6b
Fix choosing languges function
...
Searching for english is now giving all pages results.
Instead searching for a specific language different from english
is correctly returning only translated pages for the selected language.
6 years ago
Adam Tauber
baacfac32b
[fix] remove publicly unavailable 500px engine - #1338
6 years ago
dadosch
b575f898c0
duden.de engine
6 years ago
Adam Tauber
845ba79e75
[fix] update qwant search url
...
thx to @dalf for the fix (#1365 )
6 years ago
Alice Ferrazzi
5f0888b8f4
added languages to Gentoo engine
6 years ago
Alice Ferrazzi
b7886bb15c
gentoo engine test
6 years ago
Adam Tauber
d51732c0e5
Merge pull request #1303 from MarcAbonce/bing
...
Fix bing "garbage" results
6 years ago
Léo Bourrel
7a474db61b
Fix formatting
6 years ago
Léo Bourrel
acaef6600e
Update path to wikidata image
6 years ago
Adam Tauber
f82ead3e30
[fix] add py3 compatibility to base engine - closes #1319
6 years ago
Adam Tauber
57e7e9da98
[fix] use html result page in google images (previous endpoint stopped working)
6 years ago
Noémi Ványi
16915116e0
fix piratebay url
...
Closes #1229
6 years ago
Marc Abonce Seguin
75b276f408
fix bing "garbage" results (issue #1275 )
6 years ago
Noémi Ványi
b843c1351a
add missing parameter to gigablast search url
...
Closes #1293
6 years ago
rinpatch
059c2ccb95
Fix acgsou encoding error on python2 and add unicode characters to unit testt
6 years ago
rinpatch
3b89ce9bfa
Switch acgsou to http
6 years ago
Adam Tauber
1a1f9852f1
Merge pull request #1283 from rinpatch/acgsou-engine
...
[Feature] Acgsou as a searchable engine
6 years ago
rinpatch
f6695cea59
Fix nyaa to use https
6 years ago
rinpatch
f17e54a396
Forgot to remove print command used for debugging
6 years ago
rinpatch
dcc9fdb47f
Added unit test
6 years ago