Alexandre Flament
b329058c1a
Revert "[enh] test: load each engine to check for syntax errors"
...
This reverts commit 4fb3ed2c63
.
4 years ago
Dalf
4fb3ed2c63
[enh] test: load each engine to check for syntax errors
4 years ago
Noémi Ványi
e3282748d0
add display_error_messages option to engine settings
...
A new option is added to engines to hide error messages from users. It
is called `display_error_messages` and by default it is set to `True`.
If it is set to `False` error messages do not show up on the UI.
Keep in mind that engines are still suspended if needed regardless of
this setting.
Closes #1828
4 years ago
Noémi Ványi
99435381a8
[enh] introduce private engines
...
This PR adds a new setting to engines named `tokens`.
It expects a list of tokens which lets searx validate
if the request should be accepted or not.
5 years ago
Noémi Ványi
5796dc60c9
fix pep 8 check
5 years ago
Noémi Ványi
a6f20caf32
add initial support for offline engines && command engine
5 years ago
Dalf
23611897ec
[fix] make sure then engine name is lower case
...
Minor fix: "%s engine initialized" display the right engine name
5 years ago
Marc Abonce Seguin
51111c2594
[fix] always set langauge_aliases even if it's empty
6 years ago
Marc Abonce Seguin
772c048d01
refactor engine's search language handling
...
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.
Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
7 years ago
Adam Tauber
2f69eaeb2f
[fix] fix engine initialization
7 years ago
Marc Abonce Seguin
829032f306
[fix] read utf-8 files (settings, languages, currency) with python3.5
...
Related to discussion in #1124
The io.open import is necessary for python2
7 years ago
Joseph Nuthalapati
bdc803e185
Make Python 3 able to read settings files with Unicode characters
...
SearX currently doesn't start up when run with Python 3 as it tries to parse the
settings.yml file with ASCII codecs.
There are similar problems with engines_languages.json and currencies.json
Python 3 requires that files with Unicode characters be read with a 'b' flag.
This also works with Python 2 and hence can be integrated into the main source
code.
Tested with the latest Python 3.6.4rc1 on Debian unstable.
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
7 years ago
Adam Tauber
0f6612bb40
[mod] separate engine load and initialization
7 years ago
Adam Tauber
1794f6a4d3
[enh] add "inactive" attribute to engines
...
This modification allows us to deactivate engines in settings.yml
without commenting them out
7 years ago
Adam Tauber
343ac7197d
[fix] pep8
7 years ago
Adam Tauber
78365ffb8a
[enh] add init function to engines which loads parallel
7 years ago
Adam Tauber
52e615dede
[enh] py3 compatibility
8 years ago
Alexandre Flament
12d91c1d67
[mod] searx doesn't crash at startup when an engine can't be loaded (see #884 )
8 years ago
Adam Tauber
8bff42f049
Merge branch 'master' into languages
8 years ago
Adam Tauber
ea034fafa9
[fix] proper engine init
8 years ago
Adam Tauber
a605377c40
[enh] explicit engine init
8 years ago
marc
af35eee10b
tests for _fetch_supported_languages in engines
...
and refactor method to make it testable without making requests
8 years ago
marc
f62ce21f50
[mod] fetch supported languages for several engines
...
utils/fetch_languages.py gets languages supported by each engine and
generates engines_languages.json with each engine's supported language.
8 years ago
marc
149802c569
[enh] add supported_languages on engines and auto-generate languages.py
8 years ago
Alexandre Flament
e48f07a367
Merge branch 'master' into searchpy2
8 years ago
Adam Tauber
55dc538398
[mod] move load_module function to utils
8 years ago
Alexandre Flament
01e2648e93
Simplify search.py, basically updated PR #518
...
The timeouts in settings.yml is about the total time (not only the HTTP request but also the prepare the request and parsing the response)
It was more or less the case before since the threaded_requests function ignores the thread after the timeout even the HTTP request is ended.
New / changed stats :
* page_load_time : record the HTTP request time
* page_load_count: the number of HTTP request
* engine_time : the execution total time of an engine
* engine_time_count : the number of "engine_time" measure
The avg response times in the preferences are the engine response time (engine_load_time / engine_load_count)
To sum up :
* Search.search() filters the engines that can't process the request
* Search.search() call search_multiple_requests function
* search_multiple_requests creates one thread per engine, each thread runs the search_one_request function
* search_one_request calls the request function, make the HTTP request, calls the response function, extends the result_container
* search_multiple_requests waits for the the thread to finish (or timeout)
8 years ago
Adam Tauber
86daef2063
[fix] do not allow underscore in engine names - closes #708
8 years ago
Adam Tauber
7d9c898170
Merge pull request #634 from kvch/advanced-search
...
support time range search
8 years ago
Adam Tauber
54d987636e
[fix] do not load engines which cannot be initialized - closes #585
8 years ago
Noemi Vanyi
93c0c49e9a
add time range search with yahoo
8 years ago
Adam Tauber
257e52954a
[enh][fix] update to latest dependencies ++ fix tests & travis test runner
...
WARNING: dependency changes
8 years ago
Adam Tauber
10947536aa
[enh] suspend engines after error
...
The duration is based on the number of continuous errors, but maximized in one minute
9 years ago
Adam Tauber
17b0c9f74a
[mod] engine load refactor
9 years ago
Adam Tauber
1fcf066a81
[mod] change settings file structure according to #314
9 years ago
Alexandre Flament
6765823c56
[fix] set engine.safesearch to False if not defined
10 years ago
Alexandre Flament
4689fe341c
update versions.cfg to use the current up-to-date packages
10 years ago
Adam Tauber
b19e681cce
[enh] default disabled engines - closes #109
10 years ago
Adam Tauber
299a80a1eb
[enh] using the logger
10 years ago
Cqoicebordel
5d977056f7
Flake8 and Twitter corrections
...
Lots of Flake8 corrections
Maybe we should change the rule to allow lines of 120 chars. It seems more usable.
Big twitter correction : now it outputs the words in right order...
10 years ago
Adam Tauber
2e41bfcbdb
[enh] engine shortcut validation
10 years ago
Adam Tauber
2cfa760b57
[mod] engine load refactor
10 years ago
Adam Tauber
5740cfbf1c
[fix] pep8 part II.
10 years ago
Thomas Pointhuber
fdad5f7cad
fix float division by zero bug in score calculation
10 years ago
Adam Tauber
b0ba367a1a
[enh][mod] search refactor
10 years ago
Adam Tauber
a07b2b514c
[fix] url path unquoted check to avoid duplications
10 years ago
Adam Tauber
b226e6462b
[fix] www. domain duplications
11 years ago
Adam Tauber
2be94c2078
[enh] engine default category init
11 years ago
asciimoo
5b2680be4b
[enh] new engine parameter: timeout
11 years ago
asciimoo
c1d7d30b8e
[mod] len() removed from conditions
11 years ago
asciimoo
9eb31c0186
[fix] unicode and pep8 fixes
11 years ago
Gabor Nagy
22fd0746d9
Improves PEP8 compatibility.
11 years ago
asciimoo
88425f05d6
[enh] shortcut support added to engines
11 years ago
asciimoo
2a788c8f29
[enh] search language support init
11 years ago
asciimoo
8bb94e3dc4
[enh] paging support
11 years ago
asciimoo
2b1d2dbc84
[enh] i18n support
11 years ago
asciimoo
b2492c94f4
[fix] pep/flake8 compatibility
11 years ago
Matej Cotman
dd4662978d
fix: robot fw, entry points, some flake8, package searx egg
11 years ago
asciimoo
3afdd1d994
[enh] settings unification - new dependency: pyyaml
11 years ago
asciimoo
39ebe1d519
[fix] weight configurable via engines.cfg
11 years ago
asciimoo
046e190721
[mod] function name
11 years ago
asciimoo
a65070a72d
[enh] own useragent handling init
11 years ago
asciimoo
7b4ec5c5e9
[fix] highlighting only html
11 years ago
asciimoo
a5544ab95e
[enh] score per result stat added
11 years ago
asciimoo
53878fde2d
[fix] error percentage @ stats page
11 years ago
asciimoo
5f065886ff
[fix] not highlighting Nonetype
11 years ago
asciimoo
3755f437e4
[fix] utf8 content highlighting
11 years ago
asciimoo
66ff28bb9d
[enh] suggestion support added
11 years ago
asciimoo
9688495b9f
[mod] result scoring separated
11 years ago
asciimoo
b9ebf18f59
[fix] query highlight fix
11 years ago
asciimoo
cbb397939d
[enh] incasesensitive query highlighting
11 years ago
asciimoo
6d956a7d1d
[enh] engine score weight support @ config and engines
11 years ago
asciimoo
14a53e3430
[enh] search query highlight in result texts
11 years ago
asciimoo
1a2cf205f9
[enh] skipping empty/nonetype categories
11 years ago
asciimoo
4cc391169a
[enh] ignoring empty request urls
11 years ago
asciimoo
70278567ec
[mod] searching by selected engines
11 years ago
asciimoo
d2a952ecdf
[enh] request arg refactor
11 years ago
asciimoo
443af2ba83
[fix] varname
11 years ago
asciimoo
a8797839a1
[enh] request timeout added
11 years ago
asciimoo
326a33fafe
[enh] better url comparison
11 years ago
asciimoo
cd268f90e2
[fix] correct scheme @ parsed url
11 years ago
asciimoo
ec95fddce2
[mod] scoring
11 years ago
asciimoo
e13fe033ed
[enh] engine exception handling ++ stats
11 years ago
asciimoo
ca0800306f
[mod] scoring mods
11 years ago
asciimoo
21e9c87305
[mod] pageload stat order
11 years ago
asciimoo
f01e67dbd4
[mod] scoring mods
11 years ago
asciimoo
0f301adf60
[enh] score stat mods
11 years ago
asciimoo
09154c1f01
[mod] results engines mods
11 years ago
asciimoo
22c94ec149
[fix] using https links
11 years ago
asciimoo
e2c1c10657
[enh] engine score stats
11 years ago
asciimoo
13a4582a40
[enh] search count
11 years ago
asciimoo
afe767d8e9
[enh] stat bar graphs
11 years ago
asciimoo
4f4ec44bb3
[enh] stat updates
11 years ago
asciimoo
43ac547bc1
[enh] new stat: average request duration
11 years ago
asciimoo
b8191f43e7
[enh] engine statistics
11 years ago
asciimoo
09e7353307
[enh] config checks
11 years ago
asciimoo
89b68242d3
[mod] var rename
11 years ago
asciimoo
34bdeda6d4
[enh] basic config check
11 years ago
asciimoo
5dcddbc0ab
[enh] engine params to response callbacks
11 years ago
asciimoo
39d229e110
[enh][mod] engines.cfg support
11 years ago