Fix input cursor position bug on mobile

Refactors how search suggestions are added to the view

Fixes #1133

.github/workflows/buildx.yml

@@ -42,10 +42,10 @@ jobs:
           docker buildx ls
           docker buildx build --push \
             --tag benbusby/whoogle-search:latest \
-            --platform linux/amd64,linux/arm/v7,linux/arm64 .
+            --platform linux/amd64,linux/arm64 .
           docker buildx build --push \
             --tag ghcr.io/benbusby/whoogle-search:latest \
-            --platform linux/amd64,linux/arm/v7,linux/arm64 .
+            --platform linux/amd64,linux/arm64 .
       - name: build and push tag
         if: startsWith(github.ref, 'refs/tags')
         run: |

.gitignore

@@ -1,4 +1,5 @@
 venv/
+.venv/
 .idea/
 __pycache__/
 *.pyc
@@ -10,7 +11,8 @@ test/static
 flask_session/
 app/static/config
 app/static/custom_config
-app/static/bangs
+app/static/bangs/*
+!app/static/bangs/00-whoogle.json
 
 # pip stuff
 /build/

README.md

@@ -20,6 +20,7 @@ Contents
 1. [Features](#features)
 3. [Install/Deploy Options](#install)
     1. [Heroku Quick Deploy](#heroku-quick-deploy)
+    1. [Render.com](#render)
     1. [Repl.it](#replit)
     1. [Fly.io](#flyio)
     1. [Koyeb](#koyeb)
@@ -34,6 +35,7 @@ Contents
 6. [Extra Steps](#extra-steps)
     1. [Set Primary Search Engine](#set-whoogle-as-your-primary-search-engine)
 	2. [Custom Redirecting](#custom-redirecting)
+	2. [Custom Bangs](#custom-bangs)
     3. [Prevent Downtime (Heroku Only)](#prevent-downtime-heroku-only)
     4. [Manual HTTPS Enforcement](#https-enforcement)
     5. [Using with Firefox Containers](#using-with-firefox-containers)
@@ -60,6 +62,7 @@ Contents
 - Randomly generated User Agent
 - Easy to install/deploy
 - DDG-style bang (i.e. `!<tag> <query>`) searches
+- User-defined [custom bangs](#custom-bangs)
 - Optional location-based searching (i.e. results near \<city\>)
 - Optional NoJS mode to view search results in a separate window with JavaScript blocked
 
@@ -87,6 +90,16 @@ Notes:
 
 ___
 
+### [Render](https://render.com)
+
+Create an account on [render.com](https://render.com) and import the Whoogle repo with the following settings:
+
+- Runtime: `Python 3`
+- Build Command: `pip install -r requirements.txt`
+- Run Command: `./run`
+
+___
+
 ### [Repl.it](https://repl.it)
 [![Run on Repl.it](https://repl.it/badge/github/benbusby/whoogle-search)](https://repl.it/github/benbusby/whoogle-search)
 
@@ -422,6 +435,8 @@ There are a few optional environment variables available for customizing a Whoog
 | WHOOGLE_TOR_SERVICE  | Enable/disable the Tor service on startup. Default on -- use '0' to disable.              |
 | WHOOGLE_TOR_USE_PASS | Use password authentication for tor control port. |
 | WHOOGLE_TOR_CONF | The absolute path to the config file containing the password for the tor control port. Default: ./misc/tor/control.conf WHOOGLE_TOR_PASS must be 1 for this to work.|
+| WHOOGLE_SHOW_FAVICONS | Show/hide favicons next to search result URLs. Default on.                               |
+| WHOOGLE_UPDATE_CHECK  | Enable/disable the automatic daily check for new versions of Whoogle. Default on.        |
 
 ### Config Environment Variables
 These environment variables allow setting default config values, but can be overwritten manually by using the home page config menu. These allow a shortcut for destroying/rebuilding an instance to the same config state every time.
@@ -526,6 +541,14 @@ WHOOGLE_REDIRECTS="badA.com:goodA.com,badB.com:goodB.com"
 
 NOTE: Do not include "http(s)://" when defining your redirect.
 
+### Custom Bangs
+You can create your own custom bangs. By default, bangs are stored in 
+`app/static/bangs`. See [`00-whoogle.json`](https://github.com/benbusby/whoogle-search/blob/main/app/static/bangs/00-whoogle.json)
+for an example. These are parsed in alphabetical order with later files
+overriding bangs set in earlier files, with the exception that DDG bangs
+(downloaded to `app/static/bangs/bangs.json`) are always parsed first. Thus,
+any custom bangs will always override the DDG ones.
+
 ### Prevent Downtime (Heroku only)
 Part of the deal with Heroku's free tier is that you're allocated 550 hours/month (meaning it can't stay active 24/7), and the app is temporarily shut down after 30 minutes of inactivity. Once it becomes inactive, any Whoogle searches will still work, but it'll take an extra 10-15 seconds for the app to come back online before displaying the result, which can be frustrating if you're in a hurry.
 
@@ -663,12 +686,13 @@ A lot of the app currently piggybacks on Google's existing support for fetching
 | [https://whoogle.lunar.icu](https://whoogle.lunar.icu) | 🇩🇪 DE | Multi-choice | ✅ |
 | [https://wgl.frail.duckdns.org](https://wgl.frail.duckdns.org) | 🇧🇷 BR | Multi-choice | |
 | [https://whoogle.no-logs.com](https://whoogle.no-logs.com/) | 🇸🇪 SE | Multi-choice | |
-| [https://search.rubberverse.xyz](https://search.rubberverse.xyz) | 🇵🇱 PL | English | |
 | [https://whoogle.ftw.lol](https://whoogle.ftw.lol) | 🇩🇪 DE | Multi-choice | |
 | [https://whoogle-search--replitcomreside.repl.co](https://whoogle-search--replitcomreside.repl.co) | 🇺🇸 US | English |  |
 | [https://search.notrustverify.ch](https://search.notrustverify.ch) | 🇨🇭 CH | Multi-choice |  |
 | [https://whoogle.datura.network](https://whoogle.datura.network) | 🇩🇪 DE | Multi-choice | |
 | [https://whoogle.yepserver.xyz](https://whoogle.yepserver.xyz) | 🇺🇦 UA | Multi-choice | |
+| [https://search.nezumi.party](https://search.nezumi.party) | 🇮🇹 IT | Multi-choice | |
+| [https://search.snine.nl](https://search.snine.nl) | 🇳🇱 NL | Mult-choice | ✅ |
 
 
 * A checkmark in the "Cloudflare" category here refers to the use of the reverse proxy, [Cloudflare](https://cloudflare.com). The checkmark will not be listed for a site which uses Cloudflare DNS but rather the proxying service which grants Cloudflare the ability to monitor traffic to the website.

app/__init__.py

@@ -1,7 +1,7 @@
 from app.filter import clean_query
 from app.request import send_tor_signal
 from app.utils.session import generate_key
-from app.utils.bangs import gen_bangs_json
+from app.utils.bangs import gen_bangs_json, load_all_bangs
 from app.utils.misc import gen_file_hash, read_config_bool
 from base64 import b64encode
 from bs4 import MarkupResemblesLocatorWarning
@@ -101,7 +101,10 @@ if not os.path.exists(app.config['BUILD_FOLDER']):
 # Session values
 app_key_path = os.path.join(app.config['CONFIG_PATH'], 'whoogle.key')
 if os.path.exists(app_key_path):
-    app.config['SECRET_KEY'] = open(app_key_path, 'r').read()
+    try:
+        app.config['SECRET_KEY'] = open(app_key_path, 'r').read()
+    except PermissionError:
+        app.config['SECRET_KEY'] = str(b64encode(os.urandom(32)))
 else:
     app.config['SECRET_KEY'] = str(b64encode(os.urandom(32)))
     with open(app_key_path, 'w') as key_file:
@@ -139,7 +142,9 @@ app.config['CSP'] = 'default-src \'none\';' \
                     'connect-src \'self\';'
 
 # Generate DDG bang filter
+generating_bangs = False
 if not os.path.exists(app.config['BANG_FILE']):
+    generating_bangs = True
     json.dump({}, open(app.config['BANG_FILE'], 'w'))
     bangs_thread = threading.Thread(
         target=gen_bangs_json,
@@ -181,6 +186,11 @@ warnings.simplefilter('ignore', MarkupResemblesLocatorWarning)
 
 from app import routes  # noqa
 
+# The gen_bangs_json function takes care of loading bangs, so skip it here if
+# it's already being loaded
+if not generating_bangs:
+    load_all_bangs(app.config['BANG_FILE'])
+
 # Disable logging from imported modules
 logging.config.dictConfig({
     'version': 1,

app/filter.py

@@ -244,7 +244,9 @@ class Filter:
             None (The soup object is modified directly)
         """
         # Skip empty, parentless, or internal links
-        if not link or not link.parent or not link['href'].startswith('http'):
+        show_favicons = read_config_bool('WHOOGLE_SHOW_FAVICONS', True)
+        is_valid_link = link and link.parent and link['href'].startswith('http')
+        if not show_favicons or not is_valid_link:
             return
 
         parent = link.parent

app/routes.py

@@ -8,6 +8,8 @@ import re
 import urllib.parse as urlparse
 import uuid
 import validators
+import sys
+import traceback
 from datetime import datetime, timedelta
 from functools import wraps
 
@@ -16,7 +18,7 @@ from app import app
 from app.models.config import Config
 from app.models.endpoint import Endpoint
 from app.request import Request, TorError
-from app.utils.bangs import resolve_bang
+from app.utils.bangs import suggest_bang, resolve_bang
 from app.utils.misc import empty_gif, placeholder_img, get_proxy_host_url, \
     fetch_favicon
 from app.filter import Filter
@@ -36,9 +38,6 @@ from cryptography.fernet import Fernet, InvalidToken
 from cryptography.exceptions import InvalidSignature
 from werkzeug.datastructures import MultiDict
 
-# Load DDG bang json files only on init
-bang_json = json.load(open(app.config['BANG_FILE'])) or {}
-
 ac_var = 'WHOOGLE_AUTOCOMPLETE'
 autocomplete_enabled = os.getenv(ac_var, '1')
 
@@ -130,12 +129,12 @@ def session_required(f):
 
 @app.before_request
 def before_request_func():
-    global bang_json
     session.permanent = True
 
     # Check for latest version if needed
     now = datetime.now()
-    if now - timedelta(hours=24) > app.config['LAST_UPDATE_CHECK']:
+    needs_update_check = now - timedelta(hours=24) > app.config['LAST_UPDATE_CHECK']
+    if read_config_bool('WHOOGLE_UPDATE_CHECK', True) and needs_update_check:
         app.config['LAST_UPDATE_CHECK'] = now
         app.config['HAS_UPDATE'] = check_for_update(
             app.config['RELEASES_URL'],
@@ -171,15 +170,6 @@ def before_request_func():
 
     g.app_location = g.user_config.url
 
-    # Attempt to reload bangs json if not generated yet
-    if not bang_json and os.path.getsize(app.config['BANG_FILE']) > 4:
-        try:
-            bang_json = json.load(open(app.config['BANG_FILE']))
-        except json.decoder.JSONDecodeError:
-            # Ignore decoding error, can occur if file is still
-            # being written
-            pass
-
 
 @app.after_request
 def after_request_func(resp):
@@ -281,8 +271,7 @@ def autocomplete():
 
     # Search bangs if the query begins with "!", but not "! " (feeling lucky)
     if q.startswith('!') and len(q) > 1 and not q.startswith('! '):
-        return jsonify([q, [bang_json[_]['suggestion'] for _ in bang_json if
-                            _.startswith(q)]])
+        return jsonify([q, suggest_bang(q)])
 
     if not q and not request.data:
         return jsonify({'?': []})
@@ -313,7 +302,7 @@ def search():
     search_util = Search(request, g.user_config, g.session_key)
     query = search_util.new_search_query()
 
-    bang = resolve_bang(query, bang_json)
+    bang = resolve_bang(query)
     if bang:
         return redirect(bang)
 
@@ -594,7 +583,7 @@ def window():
     )
 
 
-@app.route(f'/robots.txt')
+@app.route('/robots.txt')
 def robots():
     response = make_response(
 '''User-Agent: *
@@ -603,11 +592,45 @@ Disallow: /''', 200)
     return response
 
 
+@app.route('/favicon.ico')
+def favicon():
+    return app.send_static_file('img/favicon.ico')
+
+
 @app.errorhandler(404)
 def page_not_found(e):
     return render_template('error.html', error_message=str(e)), 404
 
 
+@app.errorhandler(Exception)
+def internal_error(e):
+    query = ''
+    if request.method == 'POST':
+        query = request.form.get('q')
+    else:
+        query = request.args.get('q')
+
+    # Attempt to parse the query
+    try:
+        search_util = Search(request, g.user_config, g.session_key)
+        query = search_util.new_search_query()
+    except Exception:
+        pass
+
+    print(traceback.format_exc(), file=sys.stderr)
+
+    localization_lang = g.user_config.get_localization_lang()
+    translation = app.config['TRANSLATIONS'][localization_lang]
+    return render_template(
+            'error.html',
+            error_message='Internal server error (500)',
+            translation=translation,
+            farside='https://farside.link',
+            config=g.user_config,
+            query=urlparse.unquote(query),
+            params=g.user_config.to_params(keys=['preferences'])), 500
+
+
 def run_app() -> None:
     parser = argparse.ArgumentParser(
         description='Whoogle Search console runner')
@@ -626,6 +649,11 @@ def run_app() -> None:
         default='',
         metavar='</path/to/unix.sock>',
         help='Listen for app on unix socket instead of host:port')
+    parser.add_argument(
+        '--unix-socket-perms',
+        default='600',
+        metavar='<octal permissions>',
+        help='Octal permissions to use for the Unix domain socket (default 600)')
     parser.add_argument(
         '--debug',
         default=False,
@@ -677,7 +705,7 @@ def run_app() -> None:
     if args.debug:
         app.run(host=args.host, port=args.port, debug=args.debug)
     elif args.unix_socket:
-        waitress.serve(app, unix_socket=args.unix_socket)
+        waitress.serve(app, unix_socket=args.unix_socket, unix_socket_perms=args.unix_socket_perms)
     else:
         waitress.serve(
             app,

app/static/bangs/00-whoogle.json

@@ -0,0 +1,14 @@
+{
+  "!i": {
+    "url": "search?q={}&tbm=isch",
+    "suggestion": "!i (Whoogle Images)"
+  },
+  "!v": {
+    "url": "search?q={}&tbm=vid",
+    "suggestion": "!v (Whoogle Videos)"
+  },
+  "!n": {
+    "url": "search?q={}&tbm=nws",
+    "suggestion": "!n (Whoogle News)"
+  }
+}

app/static/css/search.css

@@ -64,7 +64,7 @@ details summary span {
 	padding-right: 5px;
 }
 
-.sCuL3 {
+.has-favicon .sCuL3 {
 	padding-left: 30px;
 }
 

app/static/js/autocomplete.js

@@ -21,16 +21,6 @@ const handleUserInput = () => {
     xhrRequest.send('q=' + searchInput.value);
 };
 
-const closeAllLists = el => {
-    // Close all autocomplete suggestions
-    let suggestions = document.getElementsByClassName("autocomplete-items");
-    for (let i = 0; i < suggestions.length; i++) {
-        if (el !== suggestions[i] && el !== searchInput) {
-            suggestions[i].parentNode.removeChild(suggestions[i]);
-        }
-    }
-};
-
 const removeActive = suggestion => {
     // Remove "autocomplete-active" class from previously active suggestion
     for (let i = 0; i < suggestion.length; i++) {
@@ -71,7 +61,7 @@ const addActive = (suggestion) => {
 
 const autocompleteInput = (e) => {
     // Handle navigation between autocomplete suggestions
-    let suggestion = document.getElementById(this.id + "-autocomplete-list");
+    let suggestion = document.getElementById("autocomplete-list");
     if (suggestion) suggestion = suggestion.getElementsByTagName("div");
     if (e.keyCode === 40) { // down
         e.preventDefault();
@@ -92,29 +82,28 @@ const autocompleteInput = (e) => {
 };
 
 const updateAutocompleteList = () => {
-    let autocompleteList, autocompleteItem, i;
+    let autocompleteItem, i;
     let val = originalSearch;
-    closeAllLists();
+
+    let autocompleteList = document.getElementById("autocomplete-list");
+    autocompleteList.innerHTML = "";
 
     if (!val || !autocompleteResults) {
         return false;
     }
 
     currentFocus = -1;
-    autocompleteList = document.createElement("div");
-    autocompleteList.setAttribute("id", this.id + "-autocomplete-list");
-    autocompleteList.setAttribute("class", "autocomplete-items");
-    searchInput.parentNode.appendChild(autocompleteList);
 
     for (i = 0; i < autocompleteResults.length; i++) {
         if (autocompleteResults[i].substr(0, val.length).toUpperCase() === val.toUpperCase()) {
             autocompleteItem = document.createElement("div");
+            autocompleteItem.setAttribute("class", "autocomplete-item");
             autocompleteItem.innerHTML = "<strong>" + autocompleteResults[i].substr(0, val.length) + "</strong>";
             autocompleteItem.innerHTML += autocompleteResults[i].substr(val.length);
             autocompleteItem.innerHTML += "<input type=\"hidden\" value=\"" + autocompleteResults[i] + "\">";
             autocompleteItem.addEventListener("click", function () {
                 searchInput.value = this.getElementsByTagName("input")[0].value;
-                closeAllLists();
+                autocompleteList.innerHTML = "";
                 document.getElementById("search-form").submit();
             });
             autocompleteList.appendChild(autocompleteItem);
@@ -123,10 +112,16 @@ const updateAutocompleteList = () => {
 };
 
 document.addEventListener("DOMContentLoaded", function() {
+    let autocompleteList = document.createElement("div");
+    autocompleteList.setAttribute("id", "autocomplete-list");
+    autocompleteList.setAttribute("class", "autocomplete-items");
+
     searchInput = document.getElementById("search-bar");
+    searchInput.parentNode.appendChild(autocompleteList);
+
     searchInput.addEventListener("keydown", (event) => autocompleteInput(event));
 
     document.addEventListener("click", function (e) {
-        closeAllLists(e.target);
+        autocompleteList.innerHTML = "";
     });
-});
+});

app/templates/error.html

@@ -19,22 +19,88 @@
         {{ error_message }}
     </p>
     <hr>
-    <p>
-    {% if blocked is defined %}
-        <h4><a class="link" href="https://farside.link">{{ translation['continue-search'] }}</a></h4>
-        Whoogle:
-        <br>
-        <a class="link-color" href="{{farside}}/whoogle/search?q={{query}}{{params}}">
-            {{farside}}/whoogle/search?q={{query}}
-        </a>
-        <br><br>
-        Searx:
-        <br>
-        <a class="link-color" href="{{farside}}/searx/search?q={{query}}">
-            {{farside}}/searx/search?q={{query}}
-        </a>
-        <hr>
+    {% if query and translation %}
+        <p>
+            <h4><a class="link" href="https://farside.link">{{ translation['continue-search'] }}</a></h4>
+            <ul>
+                <li>
+                    <a href="https://github.com/benbusby/whoogle-search">Whoogle</a>
+                    <ul>
+                        <li>
+                            <a class="link-color" href="{{farside}}/whoogle/search?q={{query}}{{params}}">
+                                {{farside}}/whoogle/search?q={{query}}
+                            </a>
+                        </li>
+                    </ul>
+                </li>
+                <li>
+                    <a href="https://github.com/searxng/searxng">SearXNG</a>
+                    <ul>
+                        <li>
+                            <a class="link-color" href="{{farside}}/searxng/search?q={{query}}">
+                                {{farside}}/searxng/search?q={{query}}
+                            </a>
+                        </li>
+                    </ul>
+                </li>
+            </ul>
+            <hr>
+            <h4>Other options:</h4>
+            <ul>
+                <li>
+                    <a href="https://kagi.com">Kagi</a>
+                    <ul>
+                        <li>Requires account</li>
+                        <li>
+                            <a class="link-color" href="https://kagi.com/search?q={{query}}">
+                                kagi.com/search?q={{query}}
+                            </a>
+                        </li>
+                    </ul>
+                </li>
+                <li>
+                    <a href="https://duckduckgo.com">DuckDuckGo</a>
+                    <ul>
+                        <li>
+                            <a class="link-color" href="https://duckduckgo.com/search?q={{query}}">
+                                duckduckgo.com/search?q={{query}}
+                            </a>
+                        </li>
+                    </ul>
+                </li>
+                <li>
+                    <a href="https://search.brave.com">Brave Search</a>
+                    <ul>
+                        <li>
+                            <a class="link-color" href="https://search.brave.com/search?q={{query}}">
+                                search.brave.com/search?q={{query}}
+                            </a>
+                        </li>
+                    </ul>
+                </li>
+                <li>
+                    <a href="https://ecosia.com">Ecosia</a>
+                    <ul>
+                        <li>
+                            <a class="link-color" href="https://ecosia.com/search?q={{query}}">
+                                ecosia.com/search?q={{query}}
+                            </a>
+                        </li>
+                    </ul>
+                </li>
+                <li>
+                    <a href="https://google.com">Google</a>
+                    <ul>
+                        <li>
+                            <a class="link-color" href="https://google.com/search?q={{query}}">
+                                google.com/search?q={{query}}
+                            </a>
+                        </li>
+                    </ul>
+                </li>
+            </ul>
+            <hr>
+        </p>
     {% endif %}
-    </p>
     <a class="link" href="home">Return Home</a>
 </div>

app/utils/bangs.py

@@ -1,8 +1,56 @@
 import json
 import requests
 import urllib.parse as urlparse
+import os
+import glob
 
-DDG_BANGS = 'https://duckduckgo.com/bang.v255.js'
+bangs_dict = {}
+DDG_BANGS = 'https://duckduckgo.com/bang.js'
+
+
+def load_all_bangs(ddg_bangs_file: str, ddg_bangs: dict = {}):
+    """Loads all the bang files in alphabetical order
+
+    Args:
+        ddg_bangs_file: The str path to the new DDG bangs json file
+        ddg_bangs: The dict of ddg bangs. If this is empty, it will load the
+                   bangs from the file
+
+    Returns:
+        None
+
+    """
+    global bangs_dict
+    ddg_bangs_file = os.path.normpath(ddg_bangs_file)
+
+    if (bangs_dict and not ddg_bangs) or os.path.getsize(ddg_bangs_file) <= 4:
+        return
+
+    bangs = {}
+    bangs_dir = os.path.dirname(ddg_bangs_file)
+    bang_files = glob.glob(os.path.join(bangs_dir, '*.json'))
+
+    # Normalize the paths
+    bang_files = [os.path.normpath(f) for f in bang_files]
+
+    # Move the ddg bangs file to the beginning
+    bang_files = sorted([f for f in bang_files if f != ddg_bangs_file])
+
+    if ddg_bangs:
+        bangs |= ddg_bangs
+    else:
+        bang_files.insert(0, ddg_bangs_file)
+
+    for i, bang_file in enumerate(bang_files):
+        try:
+            bangs |= json.load(open(bang_file))
+        except json.decoder.JSONDecodeError:
+            # Ignore decoding error only for the ddg bangs file, since this can
+            # occur if file is still being written
+            if i != 0:
+                raise
+
+    bangs_dict = dict(sorted(bangs.items()))
 
 
 def gen_bangs_json(bangs_file: str) -> None:
@@ -37,22 +85,35 @@ def gen_bangs_json(bangs_file: str) -> None:
 
     json.dump(bangs_data, open(bangs_file, 'w'))
     print('* Finished creating ddg bangs json')
+    load_all_bangs(bangs_file, bangs_data)
+
+
+def suggest_bang(query: str) -> list[str]:
+    """Suggests bangs for a user's query
+
+    Args:
+        query: The search query
+
+    Returns:
+        list[str]: A list of bang suggestions
+
+    """
+    global bangs_dict
+    return [bangs_dict[_]['suggestion'] for _ in bangs_dict if _.startswith(query)]
 
 
-def resolve_bang(query: str, bangs_dict: dict) -> str:
+def resolve_bang(query: str) -> str:
     """Transform's a user's query to a bang search, if an operator is found
 
     Args:
         query: The search query
-        bangs_dict: The dict of available bang operators, with corresponding
-                    format string search URLs
-                    (i.e. "!w": "https://en.wikipedia.org...?search={}")
 
     Returns:
         str: A formatted redirect for a bang search, or an empty str if there
              wasn't a match or didn't contain a bang operator
 
     """
+    global bangs_dict
 
     #if ! not in query simply return (speed up processing)
     if '!' not in query:

app/utils/misc.py

@@ -56,8 +56,8 @@ def gen_file_hash(path: str, static_file: str) -> str:
     return filename_split[0] + '.' + file_hash + filename_split[-1]
 
 
-def read_config_bool(var: str) -> bool:
-    val = os.getenv(var, '0')
+def read_config_bool(var: str, default: bool=False) -> bool:
+    val = os.getenv(var, '1' if default else '0')
     # user can specify one of the following values as 'true' inputs (all
     # variants with upper case letters will also work):
     # ('true', 't', '1', 'yes', 'y')

app/utils/results.py

@@ -144,12 +144,26 @@ def get_first_link(soup: BeautifulSoup) -> str:
         str: A str link to the first result
 
     """
+    first_link = ''
+    orig_details = []
+
+    # Temporarily remove details so we don't grab those links
+    for details in soup.find_all('details'):
+        temp_details = soup.new_tag('removed_details')
+        orig_details.append(details.replace_with(temp_details))
+
     # Replace hrefs with only the intended destination (no "utm" type tags)
     for a in soup.find_all('a', href=True):
         # Return the first search result URL
-        if 'url?q=' in a['href']:
-            return filter_link_args(a['href'])
-    return ''
+        if a['href'].startswith('http://') or a['href'].startswith('https://'):
+            first_link = a['href']
+            break
+
+    # Add the details back
+    for orig_detail, details in zip(orig_details, soup.find_all('removed_details')):
+        details.replace_with(orig_detail)
+
+    return first_link
 
 
 def get_site_alt(link: str, site_alts: dict = SITE_ALTS) -> str:

app/utils/search.py

@@ -102,9 +102,15 @@ class Search:
             except InvalidToken:
                 pass
 
-        # Strip leading '! ' for "feeling lucky" queries
-        self.feeling_lucky = q.startswith('! ')
-        self.query = q[2:] if self.feeling_lucky else q
+        # Strip '!' for "feeling lucky" queries
+        if match := re.search("(^|\s)!($|\s)", q):
+            self.feeling_lucky = True
+            start, end = match.span()
+            self.query = " ".join([seg for seg in [q[:start], q[end:]] if seg])
+        else:
+            self.feeling_lucky = False
+            self.query = q
+
         # Check for possible widgets
         self.widget = "ip" if re.search("([^a-z0-9]|^)my *[^a-z0-9] *(ip|internet protocol)" +
                         "($|( *[^a-z0-9] *(((addres|address|adres|" +
@@ -161,22 +167,25 @@ class Search:
         if g.user_request.tor_valid:
             html_soup.insert(0, bsoup(TOR_BANNER, 'html.parser'))
 
+        formatted_results = content_filter.clean(html_soup)
         if self.feeling_lucky:
-            return get_first_link(html_soup)
-        else:
-            formatted_results = content_filter.clean(html_soup)
-
-            # Append user config to all search links, if available
-            param_str = ''.join('&{}={}'.format(k, v)
-                                for k, v in
-                                self.request_params.to_dict(flat=True).items()
-                                if self.config.is_safe_key(k))
-            for link in formatted_results.find_all('a', href=True):
-                link['rel'] = "nofollow noopener noreferrer"
-                if 'search?' not in link['href'] or link['href'].index(
-                        'search?') > 1:
-                    continue
-                link['href'] += param_str
-
-            return str(formatted_results)
+            if lucky_link := get_first_link(formatted_results):
+                return lucky_link
+
+            # Fall through to regular search if unable to find link
+            self.feeling_lucky = False
+
+        # Append user config to all search links, if available
+        param_str = ''.join('&{}={}'.format(k, v)
+                            for k, v in
+                            self.request_params.to_dict(flat=True).items()
+                            if self.config.is_safe_key(k))
+        for link in formatted_results.find_all('a', href=True):
+            link['rel'] = "nofollow noopener noreferrer"
+            if 'search?' not in link['href'] or link['href'].index(
+                    'search?') > 1:
+                continue
+            link['href'] += param_str
+
+        return str(formatted_results)
 

misc/instances.txt

@@ -1,6 +1,7 @@
 https://search.albony.xyz
 https://search.garudalinux.org
 https://search.dr460nf1r3.org
+https://search.nezumi.party
 https://s.tokhmi.xyz
 https://search.sethforprivacy.com
 https://whoogle.dcs0.hu
@@ -15,9 +16,9 @@ https://whoogle2.ungovernable.men
 https://whoogle3.ungovernable.men
 https://wgl.frail.duckdns.org
 https://whoogle.no-logs.com
-https://search.rubberverse.xyz
 https://whoogle.ftw.lol
 https://whoogle-search--replitcomreside.repl.co
 https://search.notrustverify.ch
 https://whoogle.datura.network
 https://whoogle.yepserver.xyz
+https://search.snine.nl

requirements.txt

@@ -7,13 +7,13 @@ cffi==1.15.1
 chardet==5.1.0
 click==8.1.3
 cryptography==3.3.2; platform_machine == 'armv7l'
-cryptography==41.0.4; platform_machine != 'armv7l'
+cryptography==42.0.4; platform_machine != 'armv7l'
 cssutils==2.6.0
 defusedxml==0.7.1
 Flask==2.3.2
 idna==3.4
 itsdangerous==2.1.2
-Jinja2==3.1.2
+Jinja2==3.1.3
 MarkupSafe==2.1.2
 more-itertools==9.0.0
 packaging==23.0
@@ -21,7 +21,7 @@ pluggy==1.0.0
 pycodestyle==2.10.0
 pycparser==2.21
 pyOpenSSL==19.1.0; platform_machine == 'armv7l'
-pyOpenSSL==23.2.0; platform_machine != 'armv7l'
+pyOpenSSL==24.0.0; platform_machine != 'armv7l'
 pyparsing==3.0.9
 PySocks==1.7.1
 pytest==7.2.1

test/test_results.py

@@ -96,13 +96,13 @@ def test_view_my_ip(client):
 
 def test_recent_results(client):
     times = {
-        'past year': 365,
-        'past month': 31,
-        'past week': 7
+        'tbs=qdr:y': 365,
+        'tbs=qdr:m': 31,
+        'tbs=qdr:w': 7
     }
 
     for time, num_days in times.items():
-        rv = client.get(f'/{Endpoint.search}?q=test :' + time)
+        rv = client.get(f'/{Endpoint.search}?q=test&' + time)
         result_divs = get_search_results(rv.data)
 
         current_date = datetime.now()

test/test_routes.py

@@ -17,8 +17,15 @@ def test_search(client):
 
 
 def test_feeling_lucky(client):
-    rv = client.get(f'/{Endpoint.search}?q=!%20test')
+    # Bang at beginning of query
+    rv = client.get(f'/{Endpoint.search}?q=!%20wikipedia')
     assert rv._status_code == 303
+    assert rv.headers.get('Location').startswith('https://www.wikipedia.org')
+
+    # Move bang to end of query
+    rv = client.get(f'/{Endpoint.search}?q=github%20!')
+    assert rv._status_code == 303
+    assert rv.headers.get('Location').startswith('https://github.com')
 
 
 def test_ddg_bang(client):
@@ -48,6 +55,13 @@ def test_ddg_bang(client):
     assert rv.headers.get('Location').startswith('https://github.com')
 
 
+def test_custom_bang(client):
+    # Bang at beginning of query
+    rv = client.get(f'/{Endpoint.search}?q=!i%20whoogle')
+    assert rv._status_code == 302
+    assert rv.headers.get('Location').startswith('search?q=')
+
+
 def test_config(client):
     rv = client.post(f'/{Endpoint.config}', data=demo_config)
     assert rv._status_code == 302