Go to file
Markus Heiser 1ec325adcc [mod] limiter -> botdetection: modularization and documentation
In order to be able to meet the outstanding requirements, the implementation is
modularized and supplemented with documentation.

This patch does not contain functional change, except it fixes issue #2455

----

Aktivate limiter in the settings.yml and simulate a bot request by::

    curl -H 'Accept-Language: de-DE,en-US;q=0.7,en;q=0.3' \
         -H 'Accept: text/html'
         -H 'User-Agent: xyz' \
         -H 'Accept-Encoding: gzip' \
         'http://127.0.0.1:8888/search?q=foo'

In the LOG:

    DEBUG   searx.botdetection.link_token : missing ping for this request: .....

Since ``BURST_MAX_SUSPICIOUS = 2`` you can repeat the query above two time
before you get a "Too Many Requests" response.

Closes: https://github.com/searxng/searxng/issues/2455
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-05-29 14:54:56 +02:00
.devcontainer Add GitHub Codespaces configuration 2023-03-21 20:58:13 +00:00
.github [CI] drop Python 3.7 from CI 2023-04-19 14:55:17 +02:00
.vscode Add GitHub Codespaces configuration 2023-03-21 20:58:13 +00:00
dockerfiles Let uWSGI workers and threads be adjustable from env 2023-01-11 14:53:36 +01:00
docs [mod] limiter -> botdetection: modularization and documentation 2023-05-29 14:54:56 +02:00
examples Fix whitespaces 2016-07-11 18:52:37 +07:00
searx [mod] limiter -> botdetection: modularization and documentation 2023-05-29 14:54:56 +02:00
searxng_extra [fix] Israeli flag emoji in locale dropdown 2023-04-16 08:40:23 +02:00
src/brand [simple] ImageLayout.watch: img_load_error.svg if img load fails 2021-11-29 21:10:13 +01:00
tests [mod] replace utils.match_language by locales.match_locale 2023-03-24 10:37:42 +01:00
utils [doc] update & fix documentation of the "SearXNG LXC suite" 2023-05-21 18:12:39 +02:00
.coveragerc [mod] use github actions instead of travis 2020-11-17 15:09:06 +01:00
.dir-locals.el [emacs] .dir-locals.el: add Language Server Protocol Support (LSP) 2023-02-11 13:36:53 +01:00
.dockerignore [fix] tidy up ignore lists .gitignore & .dockerignore 2021-06-22 16:55:30 +02:00
.editorconfig [enh] add .editorconfig 2023-03-20 12:29:12 +01:00
.gitattributes [fix] update .gitattributes 2021-06-22 20:34:39 +02:00
.gitignore SearXNG version: fix make docker 2023-01-20 10:46:00 +00:00
.nvmrc Node: update to node 16.15.1 2022-06-25 14:07:39 +02:00
.pylintrc [mod] make python code pylint 2.16.1 compliant 2023-02-10 13:59:21 +01:00
.weblate weblate: migration to https://translate.codeberg.org/ 2023-01-21 15:45:12 +00:00
.yamllint.yml [enh] add test.yamllint - lint yaml files 2021-06-05 17:41:24 +02:00
AUTHORS.rst Update AUTHORS.rst 2023-04-12 14:01:54 +02:00
babel.cfg [fix] jinja/babel: WithExtension and AutoEscapeExtension are built-in now. 2022-03-25 09:42:12 +01:00
CHANGELOG.rst [releases] rolling release: commit history replaces the CHANGELOG 2023-05-06 14:49:33 +02:00
CONTRIBUTING.md reference docs.searxng.org 2022-01-02 21:18:29 +01:00
Dockerfile Bump Alpine 3.18 (Python 3.11) 2023-05-10 10:36:15 +02:00
LICENSE [fix] full AGPLv3+ license according to #382 2015-07-04 18:23:54 +02:00
Makefile [fix] ./manage: replace data.languages by data.traits 2023-04-05 07:44:29 +02:00
manage [fix] ./manage: replace data.languages by data.traits 2023-04-05 07:44:29 +02:00
package.json [mod] install developer tools by: make node.env.dev 2023-02-11 13:33:43 +01:00
PULL_REQUEST_TEMPLATE.md Add PR template and contribution guidelines 2020-07-10 17:10:02 +02:00
pyrightconfig-ci.json [mod] add test.pyright to test & ci.test targets 2022-01-23 08:00:39 +01:00
pyrightconfig.json [fix] pyrightconfig.json include only dedicated folders in the test 2022-01-23 08:00:39 +01:00
README.rst [mod] donation_url: disable by default 2023-05-15 09:19:17 +02:00
requirements-dev.txt Bump yamllint from 1.31.0 to 1.32.0 2023-05-26 07:56:43 +00:00
requirements.txt Bump typing-extensions from 4.5.0 to 4.6.2 2023-05-26 07:56:37 +00:00
SECURITY.md [enh] add security policy 2022-01-25 00:56:20 +01:00
setup.py [mod] replace /help by /info pages and include pages in project docs 2022-03-12 11:36:31 +01:00

.. SPDX-License-Identifier: AGPL-3.0-or-later

----

.. figure:: https://raw.githubusercontent.com/searxng/searxng/master/src/brand/searxng.svg
   :target: https://docs.searxng.org/
   :alt: SearXNG
   :width: 100%
   :align: center

----

Privacy-respecting, hackable `metasearch engine`_

Searx.space_ lists ready-to-use running instances.

A user_, admin_ and developer_ handbook is available on the homepage_.

|SearXNG install|
|SearXNG homepage|
|SearXNG wiki|
|AGPL License|
|Issues|
|commits|
|weblate|
|SearXNG logo|

----

.. _searx.space: https://searx.space
.. _user: https://docs.searxng.org/user
.. _admin: https://docs.searxng.org/admin
.. _developer: https://docs.searxng.org/dev
.. _homepage: https://docs.searxng.org/
.. _metasearch engine: https://en.wikipedia.org/wiki/Metasearch_engine

.. |SearXNG logo| image:: https://raw.githubusercontent.com/searxng/searxng/master/src/brand/searxng-wordmark.svg
   :target: https://docs.searxng.org/
   :width: 5%

.. |SearXNG install| image:: https://img.shields.io/badge/-install-blue
   :target: https://docs.searxng.org/admin/installation.html

.. |SearXNG homepage| image:: https://img.shields.io/badge/-homepage-blue
   :target: https://docs.searxng.org/

.. |SearXNG wiki| image:: https://img.shields.io/badge/-wiki-blue
   :target: https://github.com/searxng/searxng/wiki

.. |AGPL License|  image:: https://img.shields.io/badge/license-AGPL-blue.svg
   :target: https://github.com/searxng/searxng/blob/master/LICENSE

.. |Issues| image:: https://img.shields.io/github/issues/searxng/searxng?color=yellow&label=issues
   :target: https://github.com/searxng/searxng/issues

.. |PR| image:: https://img.shields.io/github/issues-pr-raw/searxng/searxng?color=yellow&label=PR
   :target: https://github.com/searxng/searxng/pulls

.. |commits| image:: https://img.shields.io/github/commit-activity/y/searxng/searxng?color=yellow&label=commits
   :target: https://github.com/searxng/searxng/commits/master

.. |weblate| image:: https://translate.codeberg.org/widgets/searxng/-/searxng/svg-badge.svg
   :target: https://translate.codeberg.org/projects/searxng/


Contact
=======

Ask questions or just chat about SearXNG on

IRC
  `#searxng on libera.chat <https://web.libera.chat/?channel=#searxng>`_
  which is bridged to Matrix.

Matrix
  `#searxng:matrix.org <https://matrix.to/#/#searxng:matrix.org>`_

Differences to searx
====================

SearXNG is a fork of `searx`_, with notable changes:

.. _searx: https://github.com/searx/searx


User experience
---------------

- Reworked (and still simple) theme:

  * Usable on desktop, tablet and mobile.
  * Light and dark versions (available in the preferences).
  * Right-to-left language support.
  * `Screenshots <https://dev.searxng.org/screenshots.html>`_

- The translations are up to date, you can contribute on `Weblate`_
- The preferences page has been updated:

  * Browse which engines are reliable or not.
  * Engines are grouped inside each tab.
  * Each engine has a description.

- Thanks to the anonymous metrics, it is easier to report malfunctioning engines,
  so they get fixed quicker

  - `Turn off metrics on the server
    <https://docs.searxng.org/admin/engines/settings.html#general>`_ if you don't want them recorded.

- Administrators can `block and/or replace the URLs in the search results
  <https://github.com/searxng/searxng/blob/5c1c0817c3996c5670a545d05831d234d21e6217/searx/settings.yml#L191-L199>`_


Setup
-----

- No need for `Morty`_ to proxy images, even on a public instance.
- No need for `Filtron`_ to block bots, as there is now a built-in `limiter`_.
- A well maintained `Docker image`_, now also built for ARM64 and ARM/v7 architectures.
  (Alternatively there are up to date installation scripts.)

.. _Docker image: https://github.com/searxng/searxng-docker


Contributing
------------

- Readable debug log.
- Contributing is easier, thanks to the `Development Quickstart`_ guide.
- A lot of code cleanup and bugfixes.
- Up to date list dependencies.

.. _Morty: https://github.com/asciimoo/morty
.. _Filtron: https://github.com/searxng/filtron
.. _limiter: https://docs.searxng.org/src/searx.plugins.limiter.html
.. _Weblate: https://translate.codeberg.org/projects/searxng/searxng/
.. _Development Quickstart: https://docs.searxng.org/dev/quickstart.html


Translations
============

Help translate SearXNG at `Weblate`_

.. figure:: https://translate.codeberg.org/widgets/searxng/-/multi-auto.svg
   :target: https://translate.codeberg.org/projects/searxng/


Codespaces
==========

You can contribute from your browser using `GitHub Codespaces`_:

- Fork the repository
- Click on the ``<> Code`` green button
- Click on the ``Codespaces`` tab instead of ``Local``
- Click on ``Create codespace on master``
- VSCode is going to start in the browser
- Wait for ``git pull && make install`` to appears and then to disapear
- You have `120 hours per month`_ (see also your `list of existing Codespaces`_)
- You can start SearXNG using ``make run`` in the terminal or by pressing ``Ctrl+Shift+B``.

.. _GitHub Codespaces: https://docs.github.com/en/codespaces/overview
.. _120 hours per month: https://github.com/settings/billing
.. _list of existing Codespaces: https://github.com/codespaces