Adam Tauber
8e727ac77f
[fix] update csv unit test
5 years ago
Noémi Ványi
99435381a8
[enh] introduce private engines
...
This PR adds a new setting to engines named `tokens`.
It expects a list of tokens which lets searx validate
if the request should be accepted or not.
5 years ago
Adam Tauber
52ccaa7acc
[mod] remove useless engine unit tests
...
These tests are not able to detect engine errors if the upstream
site changes.
5 years ago
Marc Abonce Seguin
ccaf6ca02c
[fix] update xpaths for new google results page
5 years ago
Marc Abonce Seguin
9299355570
add seedpeer again
5 years ago
Adam Tauber
3c425f09c1
[fix] remove useless engine tests
5 years ago
Adam Tauber
e74bdf8429
[fix] engine test
5 years ago
Léo Bourrel
88261e111c
Fix bing engine results count ( #1387 )
...
This PR fixes the result count from bing which was throwing an (hidden) error and add a validation to avoid reading more results than avalaible.
For example :
If there is 100 results from some search and we try to get results from 120 to 130, Bing will send back the results from 0 to 10 and no error. If we compare results count with the first parameter of the request we can avoid this "invalid" results.
5 years ago
Dalf
fcc9587ee9
[fix] fdroid engine
5 years ago
Dalf
9ff5001816
[fix] arxiv engine
5 years ago
Alexandre Flament
333e54943d
[fix] fix monkey patch in test_webapp.py ( #1667 )
...
at the end of test_webapp.py, the monkey patch of searx.search.Search was not revert which lead to side effects on other tests
close #1663
5 years ago
Alexandre Flament
72029d27de
[enh] Add timeout limit per request ( #1640 )
...
The new url parameter "timeout_limit" set timeout limit defined in second.
Example "timeout_limit=1.5" means the timeout limit is 1.5 seconds.
In addition, the query can start with <[number] to set the timeout limit.
For number between 0 and 99, the unit is the second :
Example: "<30 searx" means the timeout limit is 3 seconds
For number above 100, the unit is the millisecond:
Example: "<850 searx" means the timeout is 850 milliseconds.
In addition, there is a new optional setting: outgoing.max_request_timeout.
If not set, the user timeout can't go above searx configuration (as before: the max timeout of selected engine for a query).
If the value is set, the user can set a timeout between 0 and max_request_timeout using
<[number] or timeout_limit query parameter.
Related to #1077
Updated version of PR #1413 from @isj-privacore
5 years ago
Alexandre Flament
2179079a91
[fix] fix flickr_noapi decoding ( #1655 )
...
Characters that were not ASCII were incorrectly decoded.
Add an helper function: searx.utils.ecma_unescape (Python implementation of unescape Javascript function).
5 years ago
Dalf
6e0285b2db
[fix] wikidata engine: faster processing, remove one HTTP redirection.
...
* Search URL is https://www.wikidata.org/w/index.php?{query}&ns0=1 (with ns0=1 at the end to avoid an HTTP redirection)
* url_detail: remove the disabletidy=1 deprecated parameter
* Add eval_xpath function: compile once for all xpath.
* Add get_id_cache: retrieve all HTML with an id, avoid the slow to procress dynamic xpath '//div[@id="{propertyid}"]'.replace('{propertyid}')
* Create an etree.HTMLParser() instead of using the global one (see #1575 )
5 years ago
Frank de Lange
cbc5e13275
[enh] flickr_noapi: use complete JSON data block, add 'content', 'img_format', 'source', etc. ( #1571 )
...
Fetch complete JSON data block, use legend to extract images.
Unquote urlencoded strings.
Add image description as 'content'.
Add 'img_format' and 'source' data (needs PR #1567 to enable this data to be displayed).
Show images which lack ownerid instead of discarding them.
5 years ago
Frank de Lange
204a2cbbf0
[fix] bing_videos ( #1579 )
...
use JSON where possible, compose 'content' using all available data, use correct 'url' (direct to source instead of redirect through bing)
5 years ago
Frank de Lange
11fc9913e9
[enh] bing_images: use data from embedded JSON to improve results (e.g. real page title) ( #1568 )
...
use data from embedded JSON to improve results (e.g. real page title), add image format and source info (see PR #1567 ), improve paging logic (it now works)
5 years ago
volth
eb182df132
[mod] restore btdigg engine as btdig.com ( #1515 )
5 years ago
rachmadani haryono
3b1122c5fa
[fix] fix duden engine ( #1594 )
5 years ago
Alexandre Flament
554a21e1d0
[enh] Add Server-Timing header ( #1637 )
...
Server Timing specification: https://www.w3.org/TR/server-timing/
In the browser Dev Tools, focus on the main request, there are the responses per engine in the Timing tab.
5 years ago
rachmadani haryono
ec88fb8a0f
[fix] secret_key can be bytes instead of a string ( #1602 )
...
Fix #1600
In settings.yml, the secret_key can be written as string or as base64 encoded data using !!binary notation.
5 years ago
rachmadani haryono
8f44014627
[fix] preference query parameter decoding ( #1599 )
...
Fix issue #1598
5 years ago
rachmadani haryono
ac357b12e3
Merge branch 'master' into feature/fix-config
5 years ago
Dalf
ffe0972f91
Remove some engines : subtitleseeker, seedpeer, swisscows
...
http://www.subtitleseeker.com and http://www.seedpeer.eu don't exist anymore.
https://swisscows.ch/ has change : the engine needs to be updated
5 years ago
rachmadaniHaryono
9afc1b1e83
new: dev: test for config api
5 years ago
Marc Abonce Seguin
3e1c2153f7
[fix] duckduckgo images requests
6 years ago
Marc Abonce Seguin
f2d49a6971
[fix] get youtube results from js object
...
Results are not appearing in the html document anymore,
instead they are found inside an object embedded in a script.
6 years ago
d-tux
f1814079f0
Merge branch 'master' into engines/unsplash
6 years ago
Marc Abonce Seguin
626a8e9ac9
[fix] unicode error with WolframAlpha API engine
6 years ago
d-tux
329172f66e
Merge branch 'master' into engines/unsplash
6 years ago
Noémi Ványi
97351a2c72
fix after rebase
6 years ago
Noémi Ványi
b63d645a52
Revert "remove 'all' option from search languages"
...
This reverts commit 4d1770398a
.
6 years ago
Marc Abonce Seguin
0169b63e84
[fix] fetch google's supported languages
6 years ago
Marc Abonce Seguin
5568f24d6c
[fix] check language aliases when setting search language
6 years ago
Noémi Ványi
1ea56576dc
Merge branch 'master' into devel_google_videos
6 years ago
Noémi Ványi
5b81f7c2cc
Merge branch 'master' into bugfix_startpage
6 years ago
Venca24
0e493db2fb
[fix] google videos test
6 years ago
Léo Bourrel
73ec2ba74e
Update bing image engine with new image json format
6 years ago
Michael Pfitzner
83a06d276d
change rest of test_startpage.py
6 years ago
Michael Pfitzner
5a20d9ecd1
change the html in test_startpage.py to look more like the actual startpage site
6 years ago
Venca24
cee15f0375
[fix] google videos test
6 years ago
Denis Wernert
b9ada93b3a
Removes what looks like tracking parameters
6 years ago
Denis Wernert
ee07a5e750
Adds a unit test for the unsplash engine
6 years ago
Adam Tauber
baacfac32b
[fix] remove publicly unavailable 500px engine - #1338
6 years ago
dadosch
b575f898c0
duden.de engine
6 years ago
Léo Bourrel
0a37f90990
Fix wikidata tests with updated path to media
6 years ago
Adam Tauber
a3b83b73cb
[fix] remove obsolete test
6 years ago
Noémi Ványi
7fd7bd75ff
follow up tests
6 years ago
rinpatch
059c2ccb95
Fix acgsou encoding error on python2 and add unicode characters to unit testt
7 years ago
rinpatch
b7eb05dac6
Fix unit test url
7 years ago
rinpatch
a79c676f27
Remove Japanese characters
7 years ago
rinpatch
1ac0c90037
Fix unit test
7 years ago
rinpatch
dcc9fdb47f
Added unit test
7 years ago
rinpatch
06e070aee2
Fix string length
7 years ago
rinpatch
c40d8e1d15
Unit test
7 years ago
Marc Abonce Seguin
f7f9c50393
[fix] force English results in Google when using en-US
7 years ago
Marc Abonce Seguin
9687786226
update unit tests for google news and wikidata
7 years ago
Marc Abonce Seguin
772c048d01
refactor engine's search language handling
...
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.
Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
7 years ago
Noémi Ványi
3ef8533f4d
fix unit tests
7 years ago
marc
4d1770398a
remove 'all' option from search languages
7 years ago
Adam Tauber
9e79ad4bde
[fix] pdbe test
7 years ago
Adam Tauber
060c797739
[fix] change domain name in google engine test
7 years ago
pyrrh0n1c
a1d1a50149
Fixed unittests.
7 years ago
pyrrh0n1c
4340c0b16c
Fixed typos.
7 years ago
Adam Tauber
00a7041daa
Merge pull request #1068 from Apply55gx/genius
...
Continuation of PR #1004 (seems abandoned)
7 years ago
Noémi Ványi
5954a8e16a
minor fix of BASE engine
7 years ago
jibe-b
df0d915806
[add] pubmed engine
7 years ago
Noémi Ványi
9c2b7a82f0
minor fixes of arxiv
...
Closes #1050
7 years ago
jibe-b
3e3672e079
[add] arxiv engine
7 years ago
Apply55gx
d800e3fcfa
Merge pull request #1 from asciimoo/master
...
-
7 years ago
marc
54f0ab1f3b
test fetch langauges function on bing images
7 years ago
marc
44085e31d0
update engines_languages.json and languages.py
...
Also, fix fetch_languages.py so it can run on python3.
7 years ago
marc
a524dbb823
[fix] language support for bing images and videos
7 years ago
jibe-b
d081eee3f8
[add] unit_test for base engine
7 years ago
misnyo
c3232b0e1a
Merge branch 'master' into nyaa
7 years ago
misnyo
01330f71cd
[fix] nyaa.si fixed
7 years ago
misnyo
3182ba7069
[fix] google news dom xpath fix
7 years ago
Adam Tauber
e74aaa781e
Merge pull request #1016 from misnyo/blekko
...
[mod]blekko images removed
7 years ago
misnyo
0607b167f8
[fix] faroo json api and image layout fixed
7 years ago
misnyo
f139f9cfdb
[mod]blekko images removed
7 years ago
misnyo
bf3f9a91fa
[fix]torrentz unit_tests timezone fix
7 years ago
misnyo
2098e1b7ff
[fix]torrentz unit_tests timezone fix
7 years ago
misnyo
d14e2781b2
[fix]torrentz search engine fixed for new version
7 years ago
woorst
636b760196
remove unicode characters
7 years ago
woorst
62b2f79ce7
fix line lengths for pep8 standards
7 years ago
woorst
2434c29dc5
New engine: Genius (lyrics)
7 years ago
marc
3ca9cad927
add bing videos engine
7 years ago
marc
856dfc3018
add google videos
7 years ago
Noémi Ványi
2242000bd4
change unresponsive_engines to a set to eliminate duplication of errors
7 years ago
Noémi Ványi
243d3e4298
show engine errors in infobox && add new error alert
7 years ago
marc
405e5c8f24
[fix] duckduckgo images doesn't fail with countryless language
7 years ago
marc
3b950929b1
[fix] fix duckduckgo's offset
...
First page now starts with 0 offset,
rather than starting on the 30th result.
DuckDuckGo returns 30 results on each page.
7 years ago
marc
c65a409f0d
add duckduckgo images engine
7 years ago
Adam Tauber
8db527c1d2
[fix] use raw response with etree.parsefromstring - Unicode strings with encoding declaration are not supported
7 years ago
marc
9ee8e552da
[fix] bing images
7 years ago
Alexandre Flament
f5128c7cb9
[mod] add/modify image fetching for bing_news, qwant and twitter engines
7 years ago
Adam Tauber
52e615dede
[enh] py3 compatibility
7 years ago
François Revol
45d15bd6f0
Add framalibre engine
...
framalibre.org is a catalogue of Free Software, edited by Framasoft.
For now we pass the thumbnail as img_src as it doesn't seem to be used
for IT...
8 years ago
marc
805fb02ed1
add language support for qwant
...
closes issue #863
8 years ago
Alexandre Flament
e67dfaaac7
Merge branch 'master' into flask_perimeter
8 years ago