Commit Graph

438 Commits (2f76b570ab4dae3b2c85aaddddc41831548b1ff8)

Author SHA1 Message Date
Adam Tauber da8b227044 [fix] use base_url everywhere if it is defined in settings.yml 4 years ago
Alexandre Flament bfdad7bc0f [fix] opensearch.xml URL contains method and autocomplete parameters
When the user add searx as a search engine, the browser loads the /opensearch.xml URL without the cookies.
Without the query parameters, the user preferences are ignored (method and autocomplete).

In addition, opensearch.xml is modified to support automatic updates,
see https://developer.mozilla.org/en-US/docs/Web/OpenSearch
4 years ago
Alexandre Flament 6c39917c4d [mod] webapp.py: update engines initialization condition
Always call initialize engines except on the first run of werkzeug with the reload feature.

the reload feature is activated when:
* searx_debug is True (SEARX_DEBUG environment variable or settings.yml)
* FLASK_APP=searx/webapp.py FLASK_ENV=development flask run (see https://flask.palletsprojects.com/en/1.1.x/cli/ )

Fix SEARX_DEBUG=0 make docs
docs/admin/engines.rst : engines are initialized
See https://github.com/searx/searx/issues/2204#issuecomment-701373438
4 years ago
Adam Tauber 8d47142f35
Merge pull request #2189 from dalf/architecture-clean-up
Architecture clean up
4 years ago
Alexandre Flament 93f7f7eee2 [mod] upgrade requests to version 2.24.0. use ssl instead of pyopenssl.
requests 2.24.0 uses the ssl module except if it doesn't support SNI, in this case searx fallbacks to pyopenssl.
searx logs a critical message and exit if the ssl modules doesn't support SNI and pyOpenSSL is not installed.
searx logs a critical message and exit if the ssl version is older than 1.0.2.
in requirements.txt, pyopenssl is still required to install searx as a fallback.
4 years ago
Alexandre Flament f2f3300bde [mod] more typing 4 years ago
Alexandre Flament 678699beaf [mod] searx/webadapter.py: add get_selected_categories share common code with get_search_query_from_webapp
Update searx/webapp.py to use get_selected_categories
Close #2142
4 years ago
Alexandre Flament 691d12726b [mod] check the engine tokens in searx/webadapter.py instead of searx/search.py 4 years ago
Alexandre Flament 2dbc0de0cd [mod] add searx/webadapter.py
* move searx.search.get_search_query_from_webapp to searx.webadapter
* move searx.query.SearchQuery to searx.search
4 years ago
Alexandre Flament edd8dccd07 [mod] searx.query.RawTextQuery: getSearchQuery and changeSearchQuery rename to getQuery and changeQuery
getSearchQuery is confusing, the method returns a str not a SearchQuery object
4 years ago
Alexandre Flament ad0758e52a [mod] add searx/webutils.py
contains utility functions and classes used only by webapp.py
4 years ago
Alexandre Flament f9664037a6 [mod] refactor searx.webapp.get_locale
* Log each call to get_locale: display the URL, the locale and the source (browser, preferences, form).
* Rename _get_browser_language to _get_browser_or_settings_language to match the actual code.
4 years ago
Alexandre Flament 2fd582d457 [fix] /autocompleter: return HTTP 400 when q is empty as intent.
Fix commit c225db45c8
4 years ago
Alexandre Flament df12ed6e55 [mod] searx.RawTextQuery: the constructor call parse_query 4 years ago
Dalf c225db45c8 Drop Python 2 (4/n): SearchQuery.query is a str instead of bytes 4 years ago
Dalf 7888377743 Drop Python 2 (3/n): objects 4 years ago
Dalf 78df10fb55 Drop Python 2 (2/n): templates 4 years ago
Dalf 1022228d95 Drop Python 2 (1/n): remove unicode string and url_utils 4 years ago
Alexandre Flament b329058c1a Revert "[enh] test: load each engine to check for syntax errors"
This reverts commit 4fb3ed2c63.
4 years ago
Dalf 4fb3ed2c63 [enh] test: load each engine to check for syntax errors 4 years ago
Émilien Devos a5e6423c39
[fix] opensearch on chromium for android (#2132) 4 years ago
Mohamad Safadieh 1ea35605d1
Use query params for browser autocomplete
Sending query params over GET seems to be the only way to be able to
enable autocomplete in the browser. This commit adds the necessary URL
formatting to opensearch.xml. In order to identify queries coming from
the URL bar (rather than an AJAX request), which requires a different
JSON format and MIME type, the request headers are checked for
"X-Requested-With: XMLHttpRequest" which is added by jQuery request.
4 years ago
Adam Tauber 1f2dc6c647 [enh] add external plugin support 4 years ago
Adam Tauber e693bccaf4 [mod] drop py2 support 4 years ago
Lukas van den Berk 4829a76aae
Created new plugin type custom_results. Added new plugin bang_redirect (#2027)
* Made first attempt at the bangs redirects plugin.

* It redirects. But in a messy way via javascript.

* First version with custom plugin

* Added a help page and a operator to see all the bangs available.

* Changed to .format because of support

* Changed to .format because of support

* Removed : in params

* Fixed path to json file and changed bang operator

* Changed bang operator back to &

* Made first attempt at the bangs redirects plugin.

* It redirects. But in a messy way via javascript.

* First version with custom plugin

* Added a help page and a operator to see all the bangs available.

* Changed to .format because of support

* Changed to .format because of support

* Removed : in params

* Fixed path to json file and changed bang operator

* Changed bang operator back to &

* Refactored getting search query. Also changed bang operator to ! and is now working.

* Removed prints

* Removed temporary bangs_redirect.js file. Updated plugin documentation

* Added unit test for the bangs plugin

* Fixed a unit test and added 2 more for bangs plugin

* Changed back to default settings.yml

* Added myself to AUTHORS.rst

* Refacored working of custom plugin.

* Refactored _get_bangs_data from list to dict to improve search speed.

* Decoupled bangs plugin from webserver with redirect_url

* Refactored bangs unit tests

* Fixed unit test bangs. Removed dubbel parsing in bangs.py

* Removed a dumb print statement

* Refactored bangs plugin to core engine.

* Removed bangs plugin.

* Refactored external bangs unit tests from plugin to core.

* Removed custom_results/bangs documentation from plugins.rst

* Added newline in settings.yml so the PR stays clean.

* Changed searx/plugins/__init__.py back to the old file

* Removed newline search.py

* Refactored get_external_bang_operator from utils to external_bang.py

* Removed unnecessary import form test_plugins.py

* Removed _parseExternalBang and _isExternalBang from query.py

* Removed get_external_bang_operator since it was not necessary

* Simplified external_bang.py

* Simplified external_bang.py

* Moved external_bangs unit tests to test_webapp.py. Fixed return in search with external_bang

* Refactored query parsing to unicode to support python2

* Refactored query parsing to unicode to support python2

* Refactored bangs plugin to core engine.

* Refactored search parameter to search_query in external_bang.py
4 years ago
Dalf 4c7b787004 [mod] don't try to proxify data URL.
Previously only image/jpeg was not proxied.
This commit don't proxify all MIME types starting with "image/".

This is a quick fix for the PR #1985 : the google_image engine can returns some data URL.
4 years ago
Adam Tauber 08c13daf85 [enh] update opensearch.xml to match major search engines opensearch.xml 4 years ago
Adam Tauber 4ca0d8cb0f [enh] add translatable strings to javascript - closes #461 4 years ago
Adam Tauber de1b08a941 [enh] add unique class for each endpoint - closes #1133 4 years ago
Noémi Ványi 640da73a9e make category order configurable using ui.categories_order 4 years ago
Noémi Ványi a3e15a3df6 Fix Farsi language in Oscar && adjust tables with RTL 4 years ago
Noémi Ványi bce3830b8d [fix] translate engine errors to Occitan when configured 4 years ago
Noémi Ványi ba7c8d7b96 [fix] remove usage of request context where not available 4 years ago
Markus Heiser 4d64828237 webapp.py: expose the brand variable in the /config URL.
E.g. helpful for searx-stats2

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser ace7d30aed webapp.py: partial code review (no functional change)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Markus Heiser 04c687403e [fix] brands: add variables from build env to jinja templating
We have some variables in the build environment which are also needed in the
templating process.  Theses variables are relavant if one creates a fork with
its own branding.  We treat these variables under the term 'brands'.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Adam Tauber 822aee94a2 [fix] remove debug print 4 years ago
Adam Tauber baca55c94e [fix] handle weights in accept language parsing - fixes w3ms en;q=1.0 4 years ago
Adam Tauber 58a630308a [fix] convert query to string to produce valid filename for csv output 4 years ago
Adam Tauber 018b681841 [fix] add answers, suggestions, corrections to csv output
fixes #1888
4 years ago
Adam Tauber 9bc24080bf [fix] add answers, suggestions, corrections to rss output
fixes #1888
4 years ago
Marc Abonce Seguin 51e78211de monkey patch babel get_translations to support Occitan 4 years ago
Marc Abonce Seguin c0006cadf7 fix default locale and language issues 4 years ago
Markus Heiser 7c79eb9b1b Merge commit 'refs/pull/1621/head' of https://github.com/asciimoo/searx
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
4 years ago
Noémi Ványi 99435381a8 [enh] introduce private engines
This PR adds a new setting to engines named `tokens`.
It expects a list of tokens which lets searx validate
if the request should be accepted or not.
4 years ago
Nicolas Gelot b8b13372c8 Fix deprecated werkzeug import
Close: #1830
Signed-off-by: Nicolas Gelot <nicolas.gelot@e.email>
4 years ago
Adam Tauber 8e3bd3fcbd [mod] add py2 deprecation warning to webapp 4 years ago
Marc Abonce Seguin 495ae59b31 hide suggestions box if empty
This bug happens only in python3
because map returns an iterator.
5 years ago
Adam Tauber 731e34299d
Merge pull request #1744 from dalf/optimizations
[mod] speed optimization
5 years ago
lorddavidiii 5e5ff0cbf8 webapp.py: use html.escape if cgi.escape is not available
- cgi.escape was removed in python 3.8
- also use html.escape in framalibre.py
5 years ago
Dalf 85b3723345 [mod] speed optimization
compile XPath only once
avoid redundant call to urlparse
get_locale(webapp.py): avoid useless call to request.accept_languages.best_match
5 years ago
Marc Abonce Seguin b0f89ed477 [fix] preserve bangs in corrections 5 years ago
Noémi Ványi a6f20caf32 add initial support for offline engines && command engine 5 years ago
Alexandre Flament 72029d27de
[enh] Add timeout limit per request (#1640)
The new url parameter "timeout_limit" set timeout limit defined in second.
Example "timeout_limit=1.5" means the timeout limit is 1.5 seconds.

In addition, the query can start with <[number] to set the timeout limit.

For number between 0 and 99, the unit is the second :
Example: "<30 searx" means the timeout limit is 3 seconds

For number above 100, the unit is the millisecond:
Example: "<850 searx" means the timeout is 850 milliseconds.

In addition, there is a new optional setting: outgoing.max_request_timeout.
If not set, the user timeout can't go above searx configuration (as before: the max timeout of selected engine for a query).

If the value is set, the user can set a timeout between 0 and max_request_timeout using
<[number] or timeout_limit query parameter.

Related to #1077
Updated version of PR #1413 from @isj-privacore
5 years ago
Marc Abonce Seguin 59b1370824 [fix] small fixes in preferences view's text 5 years ago
Dalf 2fab23ab9a [fix] fix the debug message "starting webserver on ip:port"
was "port:ip"
5 years ago
Alexandre Flament 554a21e1d0
[enh] Add Server-Timing header (#1637)
Server Timing specification: https://www.w3.org/TR/server-timing/

In the browser Dev Tools, focus on the main request, there are the responses per engine in the Timing tab.
5 years ago
Dalf c7220b40ea Suggestions: use RawTextQuery to make the suggestions URLs. Update all themes accordingly. 5 years ago
Marc Abonce Seguin 6626f01365 [fix] preserve bangs in suggestions and corrections 5 years ago
Marc Abonce Seguin 40272b0044 [fix] never pass bangs to autocomplete suggestions 5 years ago
Nicolas Gelot 373a66be30 Fix locale and search language
Locale and search language was always defined with english value.

This patch inits the locale on `pre_request` in order to define the
default value of locale and language preferences.

Plus the `best_match` function provided by flask babel library did not
work as expected. So the function `match_language` provided
by searx is used to detect that the language from Accepted-Language
header can be used in searx project.
5 years ago
Nicolas Gelot a818d3241d Remove get local from request.args
args parameters are merged in form in pre_request, so
this patch removes that useless operation.
5 years ago
rachmadaniHaryono 535fff109f fix: dev: python3 dict keys jsonify 5 years ago
Venca24 e8734b37c9 [FIX] image_proxify 6 years ago
Noémi Ványi b63d645a52 Revert "remove 'all' option from search languages"
This reverts commit 4d1770398a.
6 years ago
Marc Abonce Seguin 772c048d01 refactor engine's search language handling
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.

Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
6 years ago
Adam Tauber b9d4c0523e [fix] get doi_rewriters from settings - fixes #1245 6 years ago
Adam Tauber b70ce84eb6 [enh] update translations - add taiwanese and filipino 6 years ago
Adam Tauber 49b845051f
Merge pull request #973 from MarcAbonce/languages
Remove 'all' option from search languages
7 years ago
marc 4d1770398a remove 'all' option from search languages 7 years ago
Adam Tauber e060aedc16 [enh] make custom oscar option configurable from url 7 years ago
Adam Tauber f65f90bf3c [fix] autocompleter py3 compatibility - fixes #1088 7 years ago
yujunqiang ed1c78d923 fix bug: TypeError: set(['bing', 'google']) is not JSON serializable, when curl 'http://127.0.0.1:8888/?q=xxxxxxxx&categories=general&format=json' 7 years ago
Noémi Ványi 1b10abfc92 minor fixes of doi resolver
Closes #1047
7 years ago
jibe-b 575159b194 [enh] oa_doi_rewrite plugin broadens doai_rewrite 7 years ago
Adam Tauber 5f758b2d39 [mod] compress saved preferences in url 7 years ago
Noémi Ványi e73cb14889 fix hmac python3 compatibility 7 years ago
misnyo ae99f9070c [fix] static path fixed in templates and webapp, line endings fixed 7 years ago
Adam Tauber f2b13a7de0 [enh] allow default empty values for normal and result proxies 7 years ago
Noémi Ványi 243d3e4298 show engine errors in infobox && add new error alert 7 years ago
Adam Tauber 38d6ba4066 [enh] generate search url of saved preferences 7 years ago
Adam Tauber 6d76d9b202 [fix] load engines if debug mode is enabled using uwsgi 7 years ago
Alexandre Flament 87f673336f [fix] rss feed : the jinja template was not found when the rss feed returns an valid feed including an error. 7 years ago
Alexandre Flament 9c91ab33f8 [mod] settings.yml can be /etc/searx/settings.yml
The exact order is
* first from SEARX_SETTINGS_PATH,
* if not found then from searx code base,
* if not found then from /etc/searx/settings.yml
* if not found an exception stops searx loading
7 years ago
Alexandre Flament ee080feaed [mod] the static and templates directories can be defined in the settings.yml 7 years ago
Adam Tauber 52e615dede [enh] py3 compatibility 7 years ago
Alexandre Flament 181c12ae04 [mod] upgrade requirements.txt 7 years ago
Alexandre Flament e67dfaaac7 Merge branch 'master' into flask_perimeter 8 years ago
Alexandre Flament 15eef0ebdb [enh] validate input and raise an exception inside search.py. The exception message is output in json and rss format. 8 years ago
Alexandre Flament 7fdfeca3a4 [mod] add a __common__ template that can't be selected but that provides a common place for shared templates.
What has been moved into this template :
* opensearch*.xml is always the same whatever the themes.
* the text inside */about.html
8 years ago
David A Roberts 1d30141c20 [enh] show spelling corrections 8 years ago
Alexandre Flament 7a16aca346 Merge branch 'master' into flask_perimeter 8 years ago
Dalf 06bea82935 [mod] /config API : add language_support, paging, safesearch, supported_languages, time_range_support, timeout fields. 8 years ago
Alexandre Flament 84a2c97a65 [mod] searx uses flask framework only in webapp.py. Make migration to another framework easier. 8 years ago
Adam Tauber 7388067f15 [fix] load engines with uwsgi too 8 years ago
Adam Tauber 8bff42f049 Merge branch 'master' into languages 8 years ago
Adam Tauber 68cbf0448f [fix] do not reload engines twice if started with debug mode 8 years ago
Adam Tauber a605377c40 [enh] explicit engine init 8 years ago
Adam Tauber 14f58bdaec [fix] robot tests ++ set default value for server http protocol version 8 years ago
Adam Tauber a98bbefbcf [enh] configurable listening http protocol version 8 years ago