Markus Heiser
eae3481688
[fix] commit 2c6531b2
breaks the unit test, this is a hotfix
...
commit 2c6531b2
does not only break the unit test, it is a significant change of
the data model and the searx search-syntax model (UI) without any discussion nor
documentation.
At the end, adding routes to instant answers is a nice feature but commit
2c6531b2
leaf some questions open.
In that sense, this patch is only a hotfix not a assessment.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Adam Tauber
8e727ac77f
[fix] update csv unit test
5 years ago
Noémi Ványi
99435381a8
[enh] introduce private engines
...
This PR adds a new setting to engines named `tokens`.
It expects a list of tokens which lets searx validate
if the request should be accepted or not.
5 years ago
Adam Tauber
52ccaa7acc
[mod] remove useless engine unit tests
...
These tests are not able to detect engine errors if the upstream
site changes.
5 years ago
Marc Abonce Seguin
ccaf6ca02c
[fix] update xpaths for new google results page
5 years ago
Marc Abonce Seguin
9299355570
add seedpeer again
5 years ago
Adam Tauber
3c425f09c1
[fix] remove useless engine tests
5 years ago
Adam Tauber
e74bdf8429
[fix] engine test
5 years ago
Léo Bourrel
88261e111c
Fix bing engine results count ( #1387 )
...
This PR fixes the result count from bing which was throwing an (hidden) error and add a validation to avoid reading more results than avalaible.
For example :
If there is 100 results from some search and we try to get results from 120 to 130, Bing will send back the results from 0 to 10 and no error. If we compare results count with the first parameter of the request we can avoid this "invalid" results.
5 years ago
Dalf
fcc9587ee9
[fix] fdroid engine
5 years ago
Dalf
9ff5001816
[fix] arxiv engine
5 years ago
Alexandre Flament
333e54943d
[fix] fix monkey patch in test_webapp.py ( #1667 )
...
at the end of test_webapp.py, the monkey patch of searx.search.Search was not revert which lead to side effects on other tests
close #1663
5 years ago
Alexandre Flament
72029d27de
[enh] Add timeout limit per request ( #1640 )
...
The new url parameter "timeout_limit" set timeout limit defined in second.
Example "timeout_limit=1.5" means the timeout limit is 1.5 seconds.
In addition, the query can start with <[number] to set the timeout limit.
For number between 0 and 99, the unit is the second :
Example: "<30 searx" means the timeout limit is 3 seconds
For number above 100, the unit is the millisecond:
Example: "<850 searx" means the timeout is 850 milliseconds.
In addition, there is a new optional setting: outgoing.max_request_timeout.
If not set, the user timeout can't go above searx configuration (as before: the max timeout of selected engine for a query).
If the value is set, the user can set a timeout between 0 and max_request_timeout using
<[number] or timeout_limit query parameter.
Related to #1077
Updated version of PR #1413 from @isj-privacore
5 years ago
Alexandre Flament
2179079a91
[fix] fix flickr_noapi decoding ( #1655 )
...
Characters that were not ASCII were incorrectly decoded.
Add an helper function: searx.utils.ecma_unescape (Python implementation of unescape Javascript function).
5 years ago
Dalf
6e0285b2db
[fix] wikidata engine: faster processing, remove one HTTP redirection.
...
* Search URL is https://www.wikidata.org/w/index.php?{query}&ns0=1 (with ns0=1 at the end to avoid an HTTP redirection)
* url_detail: remove the disabletidy=1 deprecated parameter
* Add eval_xpath function: compile once for all xpath.
* Add get_id_cache: retrieve all HTML with an id, avoid the slow to procress dynamic xpath '//div[@id="{propertyid}"]'.replace('{propertyid}')
* Create an etree.HTMLParser() instead of using the global one (see #1575 )
5 years ago
Frank de Lange
cbc5e13275
[enh] flickr_noapi: use complete JSON data block, add 'content', 'img_format', 'source', etc. ( #1571 )
...
Fetch complete JSON data block, use legend to extract images.
Unquote urlencoded strings.
Add image description as 'content'.
Add 'img_format' and 'source' data (needs PR #1567 to enable this data to be displayed).
Show images which lack ownerid instead of discarding them.
5 years ago
Frank de Lange
204a2cbbf0
[fix] bing_videos ( #1579 )
...
use JSON where possible, compose 'content' using all available data, use correct 'url' (direct to source instead of redirect through bing)
5 years ago
Frank de Lange
11fc9913e9
[enh] bing_images: use data from embedded JSON to improve results (e.g. real page title) ( #1568 )
...
use data from embedded JSON to improve results (e.g. real page title), add image format and source info (see PR #1567 ), improve paging logic (it now works)
5 years ago
volth
eb182df132
[mod] restore btdigg engine as btdig.com ( #1515 )
5 years ago
rachmadani haryono
3b1122c5fa
[fix] fix duden engine ( #1594 )
5 years ago
Alexandre Flament
554a21e1d0
[enh] Add Server-Timing header ( #1637 )
...
Server Timing specification: https://www.w3.org/TR/server-timing/
In the browser Dev Tools, focus on the main request, there are the responses per engine in the Timing tab.
5 years ago
rachmadani haryono
ec88fb8a0f
[fix] secret_key can be bytes instead of a string ( #1602 )
...
Fix #1600
In settings.yml, the secret_key can be written as string or as base64 encoded data using !!binary notation.
5 years ago
rachmadani haryono
8f44014627
[fix] preference query parameter decoding ( #1599 )
...
Fix issue #1598
5 years ago
rachmadani haryono
ac357b12e3
Merge branch 'master' into feature/fix-config
5 years ago
Dalf
ffe0972f91
Remove some engines : subtitleseeker, seedpeer, swisscows
...
http://www.subtitleseeker.com and http://www.seedpeer.eu don't exist anymore.
https://swisscows.ch/ has change : the engine needs to be updated
5 years ago
rachmadaniHaryono
9afc1b1e83
new: dev: test for config api
6 years ago
Marc Abonce Seguin
3e1c2153f7
[fix] duckduckgo images requests
6 years ago
Marc Abonce Seguin
f2d49a6971
[fix] get youtube results from js object
...
Results are not appearing in the html document anymore,
instead they are found inside an object embedded in a script.
6 years ago
d-tux
f1814079f0
Merge branch 'master' into engines/unsplash
6 years ago
Marc Abonce Seguin
626a8e9ac9
[fix] unicode error with WolframAlpha API engine
6 years ago
d-tux
329172f66e
Merge branch 'master' into engines/unsplash
6 years ago
Noémi Ványi
97351a2c72
fix after rebase
6 years ago
Noémi Ványi
b63d645a52
Revert "remove 'all' option from search languages"
...
This reverts commit 4d1770398a
.
6 years ago
Marc Abonce Seguin
0169b63e84
[fix] fetch google's supported languages
6 years ago
Marc Abonce Seguin
5568f24d6c
[fix] check language aliases when setting search language
6 years ago
Noémi Ványi
1ea56576dc
Merge branch 'master' into devel_google_videos
6 years ago
Noémi Ványi
5b81f7c2cc
Merge branch 'master' into bugfix_startpage
6 years ago
Venca24
0e493db2fb
[fix] google videos test
6 years ago
Léo Bourrel
73ec2ba74e
Update bing image engine with new image json format
6 years ago
Michael Pfitzner
83a06d276d
change rest of test_startpage.py
6 years ago
Michael Pfitzner
5a20d9ecd1
change the html in test_startpage.py to look more like the actual startpage site
6 years ago
Venca24
cee15f0375
[fix] google videos test
6 years ago
Denis Wernert
b9ada93b3a
Removes what looks like tracking parameters
6 years ago
Denis Wernert
ee07a5e750
Adds a unit test for the unsplash engine
6 years ago
Adam Tauber
baacfac32b
[fix] remove publicly unavailable 500px engine - #1338
6 years ago
dadosch
b575f898c0
duden.de engine
6 years ago
Léo Bourrel
0a37f90990
Fix wikidata tests with updated path to media
6 years ago
Adam Tauber
a3b83b73cb
[fix] remove obsolete test
6 years ago
Noémi Ványi
7fd7bd75ff
follow up tests
7 years ago
rinpatch
059c2ccb95
Fix acgsou encoding error on python2 and add unicode characters to unit testt
7 years ago
rinpatch
b7eb05dac6
Fix unit test url
7 years ago
rinpatch
a79c676f27
Remove Japanese characters
7 years ago
rinpatch
1ac0c90037
Fix unit test
7 years ago
rinpatch
dcc9fdb47f
Added unit test
7 years ago
rinpatch
06e070aee2
Fix string length
7 years ago
rinpatch
c40d8e1d15
Unit test
7 years ago
Marc Abonce Seguin
f7f9c50393
[fix] force English results in Google when using en-US
7 years ago
Marc Abonce Seguin
9687786226
update unit tests for google news and wikidata
7 years ago
Marc Abonce Seguin
772c048d01
refactor engine's search language handling
...
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.
Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
7 years ago
Noémi Ványi
3ef8533f4d
fix unit tests
7 years ago
marc
4d1770398a
remove 'all' option from search languages
7 years ago
Adam Tauber
9e79ad4bde
[fix] pdbe test
7 years ago
Adam Tauber
060c797739
[fix] change domain name in google engine test
7 years ago
pyrrh0n1c
a1d1a50149
Fixed unittests.
7 years ago
pyrrh0n1c
4340c0b16c
Fixed typos.
7 years ago
Adam Tauber
00a7041daa
Merge pull request #1068 from Apply55gx/genius
...
Continuation of PR #1004 (seems abandoned)
7 years ago
Noémi Ványi
5954a8e16a
minor fix of BASE engine
7 years ago
jibe-b
df0d915806
[add] pubmed engine
7 years ago
Noémi Ványi
9c2b7a82f0
minor fixes of arxiv
...
Closes #1050
7 years ago
jibe-b
3e3672e079
[add] arxiv engine
7 years ago
Apply55gx
d800e3fcfa
Merge pull request #1 from asciimoo/master
...
-
7 years ago
marc
54f0ab1f3b
test fetch langauges function on bing images
7 years ago
marc
44085e31d0
update engines_languages.json and languages.py
...
Also, fix fetch_languages.py so it can run on python3.
7 years ago
marc
a524dbb823
[fix] language support for bing images and videos
7 years ago
jibe-b
d081eee3f8
[add] unit_test for base engine
7 years ago
misnyo
c3232b0e1a
Merge branch 'master' into nyaa
7 years ago
misnyo
01330f71cd
[fix] nyaa.si fixed
7 years ago
misnyo
3182ba7069
[fix] google news dom xpath fix
7 years ago
Adam Tauber
e74aaa781e
Merge pull request #1016 from misnyo/blekko
...
[mod]blekko images removed
7 years ago
misnyo
0607b167f8
[fix] faroo json api and image layout fixed
7 years ago
misnyo
f139f9cfdb
[mod]blekko images removed
7 years ago
misnyo
bf3f9a91fa
[fix]torrentz unit_tests timezone fix
7 years ago
misnyo
2098e1b7ff
[fix]torrentz unit_tests timezone fix
7 years ago
misnyo
d14e2781b2
[fix]torrentz search engine fixed for new version
7 years ago
woorst
636b760196
remove unicode characters
7 years ago
woorst
62b2f79ce7
fix line lengths for pep8 standards
7 years ago
woorst
2434c29dc5
New engine: Genius (lyrics)
7 years ago
marc
3ca9cad927
add bing videos engine
7 years ago
marc
856dfc3018
add google videos
7 years ago
Noémi Ványi
2242000bd4
change unresponsive_engines to a set to eliminate duplication of errors
7 years ago
Noémi Ványi
243d3e4298
show engine errors in infobox && add new error alert
7 years ago
marc
405e5c8f24
[fix] duckduckgo images doesn't fail with countryless language
7 years ago
marc
3b950929b1
[fix] fix duckduckgo's offset
...
First page now starts with 0 offset,
rather than starting on the 30th result.
DuckDuckGo returns 30 results on each page.
8 years ago
marc
c65a409f0d
add duckduckgo images engine
8 years ago
Adam Tauber
8db527c1d2
[fix] use raw response with etree.parsefromstring - Unicode strings with encoding declaration are not supported
8 years ago
marc
9ee8e552da
[fix] bing images
8 years ago
Alexandre Flament
f5128c7cb9
[mod] add/modify image fetching for bing_news, qwant and twitter engines
8 years ago
Adam Tauber
52e615dede
[enh] py3 compatibility
8 years ago
François Revol
45d15bd6f0
Add framalibre engine
...
framalibre.org is a catalogue of Free Software, edited by Framasoft.
For now we pass the thumbnail as img_src as it doesn't seem to be used
for IT...
8 years ago
marc
805fb02ed1
add language support for qwant
...
closes issue #863
8 years ago
Alexandre Flament
e67dfaaac7
Merge branch 'master' into flask_perimeter
8 years ago
Alexandre Flament
7fdfeca3a4
[mod] add a __common__ template that can't be selected but that provides a common place for shared templates.
...
What has been moved into this template :
* opensearch*.xml is always the same whatever the themes.
* the text inside */about.html
8 years ago
David A Roberts
1d30141c20
[enh] show spelling corrections
8 years ago
Alexandre Flament
84a2c97a65
[mod] searx uses flask framework only in webapp.py. Make migration to another framework easier.
8 years ago
Adam Tauber
6bf9c398a7
[fix] use english as default language in bing
...
If no language is specified, bing returns results with multiple languages
for one query which isn't really useful. Setting english as default
insted if nothing.
8 years ago
marc
1175b3906f
change language list to only include languages with a minimum of engines
...
that support them.
users can still query lesser supported through the :lang_code bang.
8 years ago
marc
af35eee10b
tests for _fetch_supported_languages in engines
...
and refactor method to make it testable without making requests
8 years ago
marc
e0c270bd72
tests for language support in engines
8 years ago
marc
f62ce21f50
[mod] fetch supported languages for several engines
...
utils/fetch_languages.py gets languages supported by each engine and
generates engines_languages.json with each engine's supported language.
8 years ago
marc
92c6e88ad3
small fixes
8 years ago
marc
a11948c71b
Add language support for more engines.
8 years ago
Noémi Ványi
b034356825
add year filter to engines with time range support && tests
...
Following engines does not support "Last year":
* Bing News
* DeviantArt
* DuckDuckGo
* Yahoo
* YouTube (noapi)
8 years ago
Noémi Ványi
c59c76e6ee
add year to time range to engines which support "Last year"
...
Engines:
* Bing images
* Flickr (noapi)
* Google
* Google Images
* Google News
8 years ago
Adam Tauber
9b2bd6847f
[fix] 500px unit tests
8 years ago
Adam Tauber
f6e9c074bb
[fix] vimeo engine change follow-up
8 years ago
Adam Tauber
a2c94895c1
[fix] google news engine change follow-up
8 years ago
Adam Tauber
8116e341cc
[fix] kickass test html escaping
8 years ago
Adam Tauber
fdf63940e8
[fix] tests ++ flickr error
8 years ago
Adam Tauber
971ed0abd1
[enh] add quick answer functionality with an example answerer
8 years ago
Adam Tauber
044809e298
[fix] search mocking in webapp test
8 years ago
Adam Tauber
16ff8d06c7
[fix] bing paging and language support
...
see https://msdn.microsoft.com/en-us/library/ff795620.aspx for bing
specific search operators
closes #755
8 years ago
dalf
fbb080f358
Change plugin API :
...
- pre_search(request, search)
- post_search(request, search)
- on_result(request, search, result)
with
- request is the Flask request
- search a searx.Search instance
- result a searx result as usual
8 years ago
dalf
67e11c42b9
Clean up the architecture
...
Purposes :
- isolate the plugins calls
- distinction between parsing the web request and running the search (Search class). To be able to test code easily, to run searx code outside a web server, to filter the search query parameters with plugins more easily, etc...
Details :
- request.request_data contains request.form or request.args (initialize inside pre_request() function)
- Query class is renamed RawTextQuery
- SearchQuery class defines all search parameters
- get_search_query_from_webapp create a SearchQuery instance (basically the previous Search.__init__ code)
- Search class and SearchWithPlugins class takes a SearchQuery instance as class constructor parameter
- SearchWithPlugins class inherites from Search class, and run plugins
- A dedicated function search_with_plugins executes plugins to have a well define locals() (which is used by the plugins code).
- All plugins code is executed inside the try...except block (webapp.py, index function)
- advanced_search HTTP parameter value stays in webapp.py (it is only part of UI)
- multiple calls to result_container.get_ordered_results() doesn't compute the order multiple time (note : this method was call only once before)
- paging value is stored in the result_container class (compute in the extend method)
- test about engine.suspend_end_time is done during search method call (instead of __init__)
- check that the format parameter value is one of these : html, rss, json, rss (before the html value was assumed but some text formatting wasn't not done)
8 years ago
Noémi Ványi
1490d6bc93
add time range search for flickr
8 years ago
Noémi Ványi
5c02b9ef31
add time range support for bing images
8 years ago
Noémi Ványi
d2e0cbafb2
add time range search for youtube noapi
8 years ago
Noémi Ványi
f70d405202
add time range search for bing news
8 years ago
Noémi Ványi
53c9fde992
fix kickass torrents engine
8 years ago
Adam Tauber
629fa4355a
[fix] gecko driver cannot detect the page load finish on preferences form submit. It now works with this dirty workaround..
8 years ago
Alexander Minges
3c5883408c
initial commit of pdbe engine
...
Adds support for queries to the Protein Data Bank Europe (PDBe).
8 years ago
Pydo
55a5b686ed
Merge branch 'master' of https://github.com/asciimoo/searx into feature/seedpeer-engine-integration
...
Resolved conflict searx/settings.yml
8 years ago
Adam Tauber
295fc9ce96
[fix] robot test engine names II.
8 years ago
Adam Tauber
bee7b497a3
[mod] rename "default" theme to "legacy"
8 years ago
marc
d1d4ed4376
[fix] results with digbit don't truncate anymore
8 years ago
firebovine
f67db1e107
fix tests to reflect: Wolfram|Alpha (input) response
8 years ago
marc
09ee2aa69d
[fix] Result text in Wolfram|Alpha ( #607 )
8 years ago
Pydo
ec4a03628e
Put html fixture in file to be pep8 line length compliant
8 years ago
Pydo
aa85045a7d
Added seedpeer unitests
8 years ago
Adam Tauber
7591c8bfe3
Merge pull request #672 from kvch/self-info-paging
...
Self information on later pages
8 years ago
Adam Tauber
3043c404e4
Merge pull request #671 from kvch/custom-404
...
Custom 404 message - fixes #317
8 years ago
Noemi Vanyi
aa227180e2
remove self info answers from pages after first one
8 years ago
Noemi Vanyi
0056c4035e
add custom 404 page
8 years ago
Adam Tauber
f693a54ad7
Merge pull request #664 from kvch/uncapitalize-searx
...
uncapitalize searx in templates
8 years ago
Noémi Ványi
4bd44cd98f
uncapitalize searx in templates
8 years ago
Noemi Vanyi
6dd5f7a8c9
add digbt unittest
8 years ago
Adam Tauber
eb57481450
[fix] google images paging - closes #571
8 years ago
marc
ad58b14be7
[fix] merge infoboxes based on weight
...
also minor changes in attributes and images from wikidata
8 years ago
marc
a0a1284998
wikidata refactor and more attributes (see issue #560 )
8 years ago
Adam Tauber
350a84520d
[fix] time range detection
8 years ago
Noemi Vanyi
ba590de7f1
[fix] deviantart engine xpaths
8 years ago
Adam Tauber
aa09f963eb
Merge pull request #621 from stepshal/anomalous-backslash-in-string
...
Fix anomalous backslash in string
8 years ago
Adam Tauber
c75d0d1532
[fix] result count tests
8 years ago
Adam Tauber
f7a3e9771d
Merge pull request #615 from mmuman/scanr
...
Add ScanR structures search engine
8 years ago
a01200356
8fc4b8156a
[fix] multilingual duckduckgo
...
only works if both country and language are set
8 years ago
François Revol
1dba6dcbac
Add ScanR structures search engine
...
In theory ScanR should also search for projects but the API is different,
so we'd need another engine.
8 years ago
stepshal
b3ab221b98
Fix anomalous backslash in string
8 years ago
stepshal
cd9b494cb5
Fix quantity of blank lines after code object.
8 years ago
François Revol
b538de568a
Add INA search engine
8 years ago
Adam Tauber
257e52954a
[enh][fix] update to latest dependencies ++ fix tests & travis test runner
...
WARNING: dependency changes
8 years ago
Maxqia
56e53c9297
[fix] currency_convert: make unit tests pass
...
also make regex more efficent and normalize 's'es
9 years ago
a01200356
6dca14e95d
[enh] multilingual duckduckgo_definitions
9 years ago
a01200356
8d335dbdae
[enh] wikipedia infobox
...
creates simple multilingual infobox using wikipedia's api
9 years ago
Kirill Isakov
d98024c26e
Update Reddit unit test -- check for img_src
9 years ago
Adam Tauber
85c0351dca
Merge pull request #526 from ukwt/anime
...
Add a few search engines
9 years ago
Kirill Isakov
d54e82dfb9
Revert to using random UA in Reddit search engine
9 years ago
Adam Tauber
5544fdb756
Merge pull request #531 from guyou/add-doku-engine
...
Add doku engine
9 years ago
Noemi Vanyi
f0fd9ad628
add unit && robot tests
9 years ago
Noemi Vanyi
fe691a0988
new preferences handling
...
Preferences class was introduced in order to handle user preferences. Right now
it parses cookies and the form in preferences. Also it can retrieve settings
based on the name of the setting.
ATTENTION
Please note that engine preferences are handled differently from now on. So it
introduces incompatible changes. Every user who has saved preferences should reset and
save his/her settings again.
This change was needed, because everytime a default disabled engine was
added saved user preferences would broke. Now engine setting tracking is
fixed.
9 years ago
Adam Tauber
9331fc28a8
[fix] broken google images parsing
9 years ago
Guilhem Bonnefille
bb29a910f2
Fix pep8 about too long lines
...
Code is refactored and example data are truncated.
9 years ago
Guilhem Bonnefille
cf09b500f3
Add support for dokuwiki engine
9 years ago
Kirill Isakov
d0001f10e6
Add searx user agent to Reddit engine request to comply with API usage terms
9 years ago
Kirill Isakov
c1d456b136
Add F-Droid search engine
9 years ago
Kirill Isakov
547b8a8765
Add Tokyo Toshokan search engine
9 years ago
Adam Tauber
71de593a5b
[fix] bing images page change
9 years ago
Kirill Isakov
7fbc12ee4e
Add Torrentz.eu search engine
9 years ago
Kirill Isakov
d026a97e42
Add Reddit search engine
9 years ago
Kirill Isakov
8b7dc2acb9
Remove content field from ArchWiki results; reformat code in archlinux.py
...
Content field in Arch Wiki search results is of no real use, more often
than not it contains no usable information and includes too many markup
tags which make the text unreadable. It is safe to remove it.
9 years ago
Kirill Isakov
e5677ae6b6
Add Nyaa.se search engine
9 years ago
Kirill Isakov
d748b8419a
Add Arch Linux Wiki search engine
9 years ago
a01200356
751c9a346e
[fix] duckduckgo's xpaths changed
...
test_duckduckgo modified to reflect changes in duckduckgo's html
9 years ago
a01200356
8f3b33de23
[fix] remove unnecesary async calls in wolframalpha_noapi
...
setting async to false in the request did the job, lol.
9 years ago
a01200356
4d8996eb4d
[enh] unit tests for wolframalpha
9 years ago
Adam Tauber
d06178139f
[fix] wolframalpha page changes
...
related issues: #508 #509
9 years ago
Noemi Vanyi
baf16b8869
better test data
9 years ago
Noemi Vanyi
3034c85163
fix pep8 errors
9 years ago
Noemi Vanyi
33fa3da2e8
add Frinkiac engine & tests
9 years ago
Adam Tauber
0491b78f70
[fix] pep8
9 years ago
Adam Tauber
33ff1de9be
[fix] missing gigablast tests
9 years ago
Adam Tauber
ebbf33343a
[fix] bing images changes
9 years ago
a01200356
c2e034f52a
move two tests
9 years ago
Adam Tauber
53979a7bf7
[mod] remove buildout/makefile infrastructure
9 years ago