mirror of https://github.com/searxng/searxng
You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
78 lines
2.4 KiB
ReStructuredText
78 lines
2.4 KiB
ReStructuredText
5 years ago
|
===============================
|
||
5 years ago
|
Preparation for offline engines
|
||
|
===============================
|
||
|
|
||
|
Offline engines
|
||
5 years ago
|
===============
|
||
5 years ago
|
|
||
5 years ago
|
To extend the functionality of searx, offline engines are going to be
|
||
|
introduced. An offline engine is an engine which does not need Internet
|
||
|
connection to perform a search and does not use HTTP to communicate.
|
||
5 years ago
|
|
||
5 years ago
|
Offline engines can be configured as online engines, by adding those to the
|
||
|
`engines` list of :origin:`settings.yml <searx/settings.yml>`. Thus, searx
|
||
|
finds the engine file and imports it.
|
||
5 years ago
|
|
||
|
Example skeleton for the new engines:
|
||
|
|
||
|
.. code:: python
|
||
|
|
||
5 years ago
|
from subprocess import PIPE, Popen
|
||
5 years ago
|
|
||
5 years ago
|
categories = ['general']
|
||
|
offline = True
|
||
5 years ago
|
|
||
5 years ago
|
def init(settings):
|
||
|
pass
|
||
5 years ago
|
|
||
5 years ago
|
def search(query, params):
|
||
|
process = Popen(['ls', query], stdout=PIPE)
|
||
|
return_code = process.wait()
|
||
|
if return_code != 0:
|
||
|
raise RuntimeError('non-zero return code', return_code)
|
||
5 years ago
|
|
||
5 years ago
|
results = []
|
||
|
line = process.stdout.readline()
|
||
|
while line:
|
||
|
result = parse_line(line)
|
||
|
results.append(results)
|
||
5 years ago
|
|
||
5 years ago
|
line = process.stdout.readline()
|
||
5 years ago
|
|
||
5 years ago
|
return results
|
||
5 years ago
|
|
||
|
|
||
|
Development progress
|
||
5 years ago
|
====================
|
||
5 years ago
|
|
||
5 years ago
|
First, a proposal has been created as a Github issue. Then it was moved to the
|
||
|
wiki as a design document. You can read it here: :wiki:`Offline-engines`.
|
||
5 years ago
|
|
||
5 years ago
|
In this development step, searx core was prepared to accept and perform offline
|
||
|
searches. Offline search requests are scheduled together with regular offline
|
||
|
requests.
|
||
5 years ago
|
|
||
5 years ago
|
As offline searches can return arbitrary results depending on the engine, the
|
||
|
current result templates were insufficient to present such results. Thus, a new
|
||
|
template is introduced which is caplable of presenting arbitrary key value pairs
|
||
|
as a table. You can check out the pull request for more details see
|
||
3 years ago
|
:pull-searx:`1700`.
|
||
5 years ago
|
|
||
|
Next steps
|
||
5 years ago
|
==========
|
||
5 years ago
|
|
||
|
Today, it is possible to create/run an offline engine. However, it is going to be publicly available for everyone who knows the searx instance. So the next step is to introduce token based access for engines. This way administrators are able to limit the access to private engines.
|
||
|
|
||
|
Acknowledgement
|
||
5 years ago
|
===============
|
||
5 years ago
|
|
||
|
This development was sponsored by `Search and Discovery Fund`_ of `NLnet Foundation`_ .
|
||
|
|
||
|
.. _Search and Discovery Fund: https://nlnet.nl/discovery
|
||
|
.. _NLnet Foundation: https://nlnet.nl/
|
||
|
|
||
|
|
||
|
| Happy hacking.
|
||
|
| kvch // 2019.10.21 17:03
|
||
|
|