Alexandre Flament
01e2648e93
Simplify search.py, basically updated PR #518
...
The timeouts in settings.yml is about the total time (not only the HTTP request but also the prepare the request and parsing the response)
It was more or less the case before since the threaded_requests function ignores the thread after the timeout even the HTTP request is ended.
New / changed stats :
* page_load_time : record the HTTP request time
* page_load_count: the number of HTTP request
* engine_time : the execution total time of an engine
* engine_time_count : the number of "engine_time" measure
The avg response times in the preferences are the engine response time (engine_load_time / engine_load_count)
To sum up :
* Search.search() filters the engines that can't process the request
* Search.search() call search_multiple_requests function
* search_multiple_requests creates one thread per engine, each thread runs the search_one_request function
* search_one_request calls the request function, make the HTTP request, calls the response function, extends the result_container
* search_multiple_requests waits for the the thread to finish (or timeout)
8 years ago
Noémi Ványi
1490d6bc93
add time range search for flickr
8 years ago
Noémi Ványi
5c02b9ef31
add time range support for bing images
8 years ago
Noémi Ványi
d2e0cbafb2
add time range search for youtube noapi
8 years ago
Noémi Ványi
f70d405202
add time range search for bing news
8 years ago
Noémi Ványi
53c9fde992
fix kickass torrents engine
8 years ago
Alexander Minges
3c5883408c
initial commit of pdbe engine
...
Adds support for queries to the Protein Data Bank Europe (PDBe).
8 years ago
Pydo
55a5b686ed
Merge branch 'master' of https://github.com/asciimoo/searx into feature/seedpeer-engine-integration
...
Resolved conflict searx/settings.yml
8 years ago
Adam Tauber
86daef2063
[fix] do not allow underscore in engine names - closes #708
8 years ago
marc
d1d4ed4376
[fix] results with digbit don't truncate anymore
8 years ago
firebovine
e145fdb86d
#607 - noapi fix
8 years ago
Lorenzo J. Lucchini
a8907224a1
Improving Wolfram Alpha search hit content
...
Making WA search hits contain
- the (parsed) input inside the "title" instead of just "Wolfram|Alpha", to better match other hit titles and to confirm correct parsing of input to the user
- the first output field that contains any text (skipping ones that are only pictures; this is usually the most meaningful "result" field) instead of the raw input as the "content", making it additionally possible to obtain WA computations from JSON API calls
8 years ago
marc
09ee2aa69d
[fix] Result text in Wolfram|Alpha ( #607 )
8 years ago
Adam Tauber
3144ec1d59
[fix] unicode urls
8 years ago
potato
983415bc38
[enh] is_valid_lang moved to utils
8 years ago
potato
b7d578ae80
[enh] engine header comments
8 years ago
potato
8c72a22757
[enh] api_key usage, disable the engine by default
8 years ago
potato
c051e6a2c3
[fix] pep8
8 years ago
potato
ab471fd13b
[enh] mymemory translated engine added for multi-word translations
8 years ago
potato
22bd39fd42
[fix] only 1-word search triggers the engine
8 years ago
potato
7bf1013c15
[enh] removed missing params; [fix] pep8
8 years ago
potato
bc806bfab1
[fix] no lambda anymore, cgi.escape
8 years ago
potato
5416f0f248
[enh] dictionary engine renamed to dictzone
8 years ago
potato
84ff6e289e
[enh] filter non-existing language code/name containing requests
8 years ago
potato
b808a2e266
[fix] don't merge with suggestions
8 years ago
potato
3f4cc2146c
[enh] return results instead of answers
8 years ago
potato
f4525880ed
[enh] dictionary engine added
8 years ago
Pydo
2c2123b2e8
Added seepeer to config and added seepeer search parser
8 years ago
Adam Tauber
13bed1f872
Merge pull request #639 from kvch/digbt-engine
...
add digbt engine - fixes #638
8 years ago
David A Roberts
1e9dab08e6
[fix] behaviour for page_size>1 and first_page_num>0
...
eg. pageno=1,21,41,... instead of 20,40,60,...
8 years ago
Noemi Vanyi
3a1c5876b1
add digbt engine
...
Unfortunately, it is quite slow so it is disabled.
Furthermore, the display of number of files is wrong
on digbt.org, so it is not displayed on searx.
8 years ago
Adam Tauber
104cdb7d03
[fix] remove unused imports ++ int token timestamp by default
8 years ago
Adam Tauber
eb57481450
[fix] google images paging - closes #571
8 years ago
David A Roberts
61fe5daf30
Add Microsoft Academic search engine
8 years ago
Adam Tauber
31866d9f33
Merge pull request #629 from stepshal/E305
...
Add missing blank lines after class or function definition.
8 years ago
stepshal
7e16c9d360
Add missing blank lines after class or function definition.
8 years ago
David A Roberts
117d36b66e
Add Crossref search engine and DOAI rewrite plugin
8 years ago
marc
ad58b14be7
[fix] merge infoboxes based on weight
...
also minor changes in attributes and images from wikidata
8 years ago
marc
c2e4014287
[fix] urls merge in infobox ( #593 )
...
TODO:
merge attributes
8 years ago
marc
a0a1284998
wikidata refactor and more attributes (see issue #560 )
8 years ago
a01200356
93ef11adc0
[enh] multilingual wikidata
...
disambiguation and tags are in local language
TOFIX:
needs to query the api every time to know each label's name
8 years ago
Adam Tauber
350a84520d
[fix] time range detection
8 years ago
Adam Tauber
7d9c898170
Merge pull request #634 from kvch/advanced-search
...
support time range search
8 years ago
Adam Tauber
54d987636e
[fix] do not load engines which cannot be initialized - closes #585
8 years ago
Noemi Vanyi
a7c8d5882c
fix pep8
8 years ago
Noemi Vanyi
e7baf24ec1
add time range search for deviantart
8 years ago
Noemi Vanyi
e9a78f1434
add time range search for google images
8 years ago
Noemi Vanyi
f13b9fa36a
add time range search for duckduckgo
8 years ago
Noemi Vanyi
2e5839503f
add time range search for google
8 years ago
Noemi Vanyi
93c0c49e9a
add time range search with yahoo
8 years ago
Noemi Vanyi
ba590de7f1
[fix] deviantart engine xpaths
8 years ago
Adam Tauber
aa09f963eb
Merge pull request #621 from stepshal/anomalous-backslash-in-string
...
Fix anomalous backslash in string
8 years ago
Adam Tauber
f7a3e9771d
Merge pull request #615 from mmuman/scanr
...
Add ScanR structures search engine
8 years ago
marc
aac4ed6971
[fix] guess country code if not specified in duckduckgo engine
8 years ago
a01200356
8fc4b8156a
[fix] multilingual duckduckgo
...
only works if both country and language are set
8 years ago
François Revol
1dba6dcbac
Add ScanR structures search engine
...
In theory ScanR should also search for projects but the API is different,
so we'd need another engine.
8 years ago
stepshal
b3ab221b98
Fix anomalous backslash in string
8 years ago
stepshal
cd9b494cb5
Fix quantity of blank lines after code object.
8 years ago
François Revol
b538de568a
Add INA search engine
8 years ago
Adam Tauber
257e52954a
[enh][fix] update to latest dependencies ++ fix tests & travis test runner
...
WARNING: dependency changes
8 years ago
Adam Tauber
2f7752b410
[enh] display number of results
8 years ago
Adam Tauber
5dd2340bc6
[fix] gigablast redirect - closes #575
8 years ago
Maxqia
56e53c9297
[fix] currency_convert: make unit tests pass
...
also make regex more efficent and normalize 's'es
9 years ago
Maxqia
fe2ce83e41
[enh] currency_convert: allow strings like google's "convert 1 euro to dollars"
9 years ago
a01200356
a44faa7716
[fix] compile regex in ddg_definitions
9 years ago
a01200356
6dca14e95d
[enh] multilingual duckduckgo_definitions
9 years ago
a01200356
8d335dbdae
[enh] wikipedia infobox
...
creates simple multilingual infobox using wikipedia's api
9 years ago
Kirill Isakov
4eeb5fb76b
[fix] incorrect URLs in Reddit results - closes #538
9 years ago
Adam Tauber
85c0351dca
Merge pull request #526 from ukwt/anime
...
Add a few search engines
9 years ago
Kirill Isakov
90c51cb449
Fix a few typos in Google search engine
9 years ago
Kirill Isakov
d54e82dfb9
Revert to using random UA in Reddit search engine
9 years ago
Adam Tauber
5544fdb756
Merge pull request #531 from guyou/add-doku-engine
...
Add doku engine
9 years ago
Adam Tauber
9331fc28a8
[fix] broken google images parsing
9 years ago
Guilhem Bonnefille
2733a92383
Fix query encoding
9 years ago
Guilhem Bonnefille
f26f0dab2e
Fix pep8 E302
...
Cf. http://legacy.python.org/dev/peps/pep-0008/#blank-lines
9 years ago
Guilhem Bonnefille
cf09b500f3
Add support for dokuwiki engine
9 years ago
Kirill Isakov
d0001f10e6
Add searx user agent to Reddit engine request to comply with API usage terms
9 years ago
jibe-b
092e15cc1d
[enh] Add BASE engine in category Science. Basic and advanced search.
9 years ago
Kirill Isakov
bacc9a3df1
Add paging support to XPath & Erowid engines
9 years ago
Kirill Isakov
e202c6fbec
Fix PEP8 warnings for Nyaa.py
9 years ago
Kirill Isakov
a8832574fa
Shorten content field for very long Reddit search results
9 years ago
Kirill Isakov
c1d456b136
Add F-Droid search engine
9 years ago
Kirill Isakov
547b8a8765
Add Tokyo Toshokan search engine
9 years ago
Adam Tauber
71de593a5b
[fix] bing images page change
9 years ago
Kirill Isakov
7fbc12ee4e
Add Torrentz.eu search engine
9 years ago
Adam Tauber
6d55642ab4
[fix] no more redirect ++ explicitly specify search language to avoid googles ip based heuristics
9 years ago
Kirill Isakov
d026a97e42
Add Reddit search engine
9 years ago
Kirill Isakov
8b7dc2acb9
Remove content field from ArchWiki results; reformat code in archlinux.py
...
Content field in Arch Wiki search results is of no real use, more often
than not it contains no usable information and includes too many markup
tags which make the text unreadable. It is safe to remove it.
9 years ago
Kirill Isakov
e5677ae6b6
Add Nyaa.se search engine
9 years ago
Kirill Isakov
d748b8419a
Add Arch Linux Wiki search engine
9 years ago
a01200356
751c9a346e
[fix] duckduckgo's xpaths changed
...
test_duckduckgo modified to reflect changes in duckduckgo's html
9 years ago
a01200356
8f3b33de23
[fix] remove unnecesary async calls in wolframalpha_noapi
...
setting async to false in the request did the job, lol.
9 years ago
a01200356
4cea71e3bb
[fix] merge with 79705450df
9 years ago
a01200356
4d8996eb4d
[enh] unit tests for wolframalpha
9 years ago
a01200356
78d3f3d6b1
[enh] infobox for wolframalpha
...
TODO:
- infobox styles
- unit tests
ISSUES:
- no_api version needs to re-call server for additional pods, such
as plots. therefore, it's even slower than before. comment out the
part that calls get_async_pod if requests reach timeout or increase
timeout in settings.yml.
9 years ago
Adam Tauber
79705450df
[fix] wolframalpha unicode inputs
9 years ago
Adam Tauber
10947536aa
[enh] suspend engines after error
...
The duration is based on the number of continuous errors, but maximized in one minute
9 years ago
Adam Tauber
17b0c9f74a
[mod] engine load refactor
9 years ago
Adam Tauber
db72fc6449
[fix] encapsulate wolframalpha token fetching errors
9 years ago
Adam Tauber
d06178139f
[fix] wolframalpha page changes
...
related issues: #508 #509
9 years ago