wip

HELP.md

@@ -0,0 +1,75 @@
+# The problems with overwriting keys
+
+If you had one keyboard layout for your mouse that writes SHIFT keys on
+keycode 10, and one for your keyboard that is normal and writes 1/! on
+keycode 10, then you would not be able to write ! by pressing that mouse
+button and that keyboard button at the same time. Keycodes may not clash.
+
+The first idea was to write special keycodes known only to key-mapper
+(256 - 511) into an input device in /dev/input, and map those to SHIFT and
+such, whenever a button is clicked. A mapping would have existed to prevent
+the original keycode 10 from writing a 1. But X seems to ignore anything
+greater than 255, or even crash in some cases, for regular keyboard events.
+Mouse buttons can use those though, but they cannot be remapped, which I
+guess is another indicator of that.
+
+The second idea is to create a new input device that uses 8 - 255, just like
+other layouts, and key-mapper always tries to use the same keycodes for
+SHIFT as already used in the system default. The pipeline is like this:
+
+1. A human thumb presses an extra-button of the device "mouse"
+2. key-mapper uses evdev to get the event from "mouse", sees "ahh, it's a
+   10, I know that one and will now write 50 into my own device". 50 is
+   the keycode for SHIFT on my regular keyboard, so it won't clash anymore
+   with alphanumeric keys and such.
+3. X has key-mappers configs for the key-mapper device loaded and
+   checks in it's keycodes config file "50, that would be <50>", then looks
+   into it's symbols config "<50> is mapped to SHIFT", and then it actually
+   presses the SHIFT down to modify all other future buttons.
+4. X has another config for "mouse" loaded, which prevents any system default
+   mapping to print the overwritten key "1" into the session.
+
+# How I would have liked it to be
+
+setxkbmap -layout ~/.config/key-mapper/mouse -device 13
+
+config looks like:
+```
+10 = a, A
+11 = Shift_L
+```
+
+done. Without crashing X. Without printing generic useless errors. If it was
+that easy, an app to map keys would have already existed.
+
+# Folder Structure of Key Mapper in /usr
+
+Stuff has to be placed in `/usr/share/X11/xkb` to my knowledge.
+
+Every user gets a path within that `/usr/...` directory which is very
+unconventional, but it works. This way the presets of multiple users
+don't clash.
+
+**Presets**
+
+- `/usr/share/X11/xkb/symbols/key-mapper/<user>/<device>/<preset>`
+
+This is how a single preset is stored.
+
+**Defaults**
+
+- `/usr/share/X11/xkb/symbols/key-mapper/<user>/default`
+
+This is where key-mapper stores the defaults. They are generated from the
+parsed output of `xmodmap` and used to keep the unmapped keys at their system
+defaults.
+
+**Keycodes**
+
+- `/usr/share/X11/xkb/keycodes/key-mapper`
+
+Because the concept of "reasonable symbolic names" ([www.x.org](https://www.x.org/releases/X11R7.7/doc/xorg-docs/input/XKB-Enhancing.html))
+doesn't apply when mouse buttons are all over the place, an identity mapping
+to make generating "symbols" files easier/possible exists. A keycode of
+10 will be known as "<10>" in symbols configs. This has the added benefit
+that keycodes reported by xev can be identified in the symbols file.

PATHS.md

@@ -1,31 +0,0 @@
-# Folder Structure of Key Mapper
-
-Stuff has to be placed in `/usr/share/X11/xkb` to my knowledge.
-
-Every user gets a path within that `/usr/...` directory which is very
-unconventional, but it works. This way the presets of multiple users
-don't clash.
-
-**Presets**
-
-- `/usr/share/X11/xkb/symbols/key-mapper/<user>/<device>/<preset>`
-
-This is how a single preset is stored.
-
-**Defaults**
-
-- `/usr/share/X11/xkb/symbols/key-mapper/<user>/default`
-
-This is where key-mapper stores the defaults. They are generated from the
-parsed output of `xmodmap` and used to keep the unmapped keys at their system
-defaults.
-
-**Keycodes**
-
-- `/usr/share/X11/xkb/keycodes/key-mapper`
-
-Because the concept of "reasonable symbolic names" ([www.x.org](https://www.x.org/releases/X11R7.7/doc/xorg-docs/input/XKB-Enhancing.html))
-doesn't apply when mouse buttons are all over the place, an identity mapping
-to make generating "symbols" files easier/possible exists. A keycode of
-10 will be known as "<10>" in symbols configs. This has the added benefit
-that keycodes reported by xev can be identified in the symbols file.

README.md

@@ -49,3 +49,6 @@ sudo python3 setup.py install && python3 tests/test.py
 - [x] ask for administrator permissions using polkit
 - [ ] make it work on wayland
 - [ ] add to the AUR, provide .deb and .appimage files
+
+This is incredibly overcomplicated due to various obstacles with xkb. If you
+have questions about the code, feel free to open an issue. See HELP.md

keymapper/X.py

@@ -42,7 +42,8 @@ from keymapper.paths import get_usr_path, KEYCODES_PATH, DEFAULT_SYMBOLS, \
 from keymapper.logger import logger
 from keymapper.data import get_data_path
 from keymapper.linux import get_devices
-from keymapper.mapping import custom_mapping, Mapping
+from keymapper.mapping import custom_mapping, system_mapping, \
+    Mapping, MIN_KEYCODE, MAX_KEYCODE
 
 
 permissions = stat.S_IREAD | stat.S_IWRITE | stat.S_IRGRP | stat.S_IROTH
@@ -210,9 +211,9 @@ def create_identity_mapping():
 
     xkb_keycodes = []
     # the maximum specified in /usr/share/X11/xkb/keycodes is usually 255
-    # and the minimum 8
-    maximum = 255
-    minimum = 8
+    # and the minimum 8 TODO update comment
+    maximum = MAX_KEYCODE
+    minimum = MIN_KEYCODE
     for keycode in range(minimum, maximum + 1):
         xkb_keycodes.append(f'<{keycode}> = {keycode};')
 
@@ -230,6 +231,8 @@ def create_identity_mapping():
         logger.debug('Creating "%s"', KEYCODES_PATH)
         os.makedirs(os.path.dirname(KEYCODES_PATH), exist_ok=True)
         os.mknod(KEYCODES_PATH)
+        os.chmod(KEYCODES_PATH, permissions)
+
     with open(KEYCODES_PATH, 'w') as keycodes:
         keycodes.write(result)
 
@@ -264,8 +267,7 @@ def generate_symbols(name, include=DEFAULT_SYMBOLS_NAME, mapping=custom_mapping)
         keycodes = re.findall(r'<.+?>', f.read())
 
     xkb_symbols = []
-    for keycode, output in mapping:
-        target_keycode, character = output
+    for _, (keycode, character) in mapping:
         if f'<{keycode}>' not in keycodes:
             logger.error(f'Unknown keycode <{keycode}> for "{character}"')
             # don't append that one, otherwise X would crash when loading
@@ -332,12 +334,24 @@ def parse_symbols_file(device, preset):
         result = re.findall(r'\n\s+?key <(.+?)>.+?\[\s+(.+?)\s+\]', content)
         logger.debug('Found %d mappings in preset "%s"', len(result), preset)
         for keycode, character in result:
-            if ', ' in character:
-                character = [char.strip() for char in character.split(',')]
-            custom_mapping.change(None, int(keycode), character)
+            keycode = int(keycode)
+            custom_mapping.write_from_keymapper_symbols(keycode, character)
         custom_mapping.changed = False
 
 
+def parse_xmodmap():
+    """Read the output of xmodmap as a Mapping object."""
+    xmodmap = subprocess.check_output(['xmodmap', '-pke']).decode() + '\n'
+    mappings = re.findall(r'(\d+) = (.+)\n', xmodmap)
+    for keycode, characters in mappings:
+        system_mapping.change(None, int(keycode), characters.split())
+
+
+# TODO verify that this is the system default and not changed when I
+#  setxkbmap my mouse
+parse_xmodmap()
+
+
 def create_default_symbols():
     """Parse the output of xmodmap and create a default symbols file.
 
@@ -352,13 +366,7 @@ def create_default_symbols():
         logger.debug('Found the default mapping at %s', DEFAULT_SYMBOLS)
         return
 
-    xmodmap = subprocess.check_output(['xmodmap', '-pke']).decode() + '\n'
-    mappings = re.findall(r'(\d+) = (.+)\n', xmodmap)
-    defaults = Mapping()
-    for keycode, characters in mappings:
-        defaults.change(None, int(keycode), characters.split())
-
-    contents = generate_symbols(DEFAULT_SYMBOLS_NAME, None, defaults)
+    contents = generate_symbols(DEFAULT_SYMBOLS_NAME, None, system_mapping)
 
     if not os.path.exists(DEFAULT_SYMBOLS):
         logger.info('Creating %s', DEFAULT_SYMBOLS)

keymapper/gtk/window.py

@@ -33,7 +33,7 @@ from keymapper.X import create_setxkbmap_config, apply_preset, \
 from keymapper.presets import get_presets, find_newest_preset, \
     delete_preset, rename_preset
 from keymapper.logger import logger
-from keymapper.linux import get_devices
+from keymapper.linux import get_devices, KeycodeReader
 from keymapper.gtk.row import Row
 from keymapper.gtk.unsaved import unsaved_changes_dialog, GO_BACK
 
@@ -247,6 +247,7 @@ class Window:
             CTX_APPLY,
             f'Applied "{self.selected_preset}"'
         )
+        keycode_reader = KeycodeReader(self.selected_device)
 
     def on_select_device(self, dropdown):
         """List all presets, create one if none exist yet."""

keymapper/linux.py

@@ -25,10 +25,12 @@
 import subprocess
 import multiprocessing
 import threading
+import asyncio
 
 import evdev
 
 from keymapper.logger import logger
+from keymapper.mapping import custom_mapping, MAX_KEYCODE, MIN_KEYCODE
 
 
 def can_grab(path):
@@ -52,6 +54,7 @@ class KeycodeReader:
     def __init__(self, device):
         self.device = device
         self.virtual_devices = []
+        self.start_injecting()
 
     def clear(self):
         """Next time when reading don't return the previous keycode."""
@@ -61,21 +64,36 @@ class KeycodeReader:
                 pass
 
     def start_injecting_worker(self, path):
-        """Inject keycodes for one of the virtual devices."""
+        """Inject keycodes for one of the virtual devices.
+
+        This depends on a setxkbmap-loaded symbol file that contains
+        the mappings for keycodes in the range of MIN and MAX_KEYCODE.
+        """
+        loop = asyncio.new_event_loop()
+        asyncio.set_event_loop(loop)
+        # TODO use uinput instead, then register somewhere its existance in
+        #  order to setxkbmap it. Or give it a constant path that is known
+        #  application wide
         device = evdev.InputDevice(path)
-        uinput = evdev.UInput
+        uinput = evdev.UInput()
+
         for event in device.read_loop():
-            if event.type == evdev.ecodes.EV_KEY and event.value == 1:
-                # value: 1 for down, 0 for up, 2 for hold.
-                # this happens to report key codes that are 8 lower
-                # than the ones reported by xev
-                # TODO the mapping writes something starting from 256 to not
-                #  clash with existing mappings of other devices
-                input_keycode = event.code
-                output_keycode = custom_mapping.get_keycode(event.code)
-                output_char = custom_mapping.get_character(event.code)
-                print('output', output_keycode, output_char)
-                uinput.write(evdev.ecodes.EV_KEY, evdev.ecodes.KEY_A)
+            if event.type != evdev.ecodes.EV_KEY:
+                continue
+
+            # this happens to report key codes that are 8 lower
+            # than the ones reported by xev
+            input_keycode = event.code + 8
+            output_keycode = custom_mapping.get_keycode(input_keycode) - 8
+
+            print(input_keycode, output_keycode)
+
+            if output_keycode > MAX_KEYCODE or output_keycode < MIN_KEYCODE:
+                continue
+
+            # value: 1 for down, 0 for up, 2 for hold.
+            device.write(evdev.ecodes.EV_KEY, output_keycode, event.value)
+            device.write(evdev.ecodes.EV_SYN, evdev.ecodes.SYN_REPORT, 0)
 
     def start_injecting(self):
         """Read keycodes and inject the mapped character forever."""
@@ -88,10 +106,11 @@ class KeycodeReader:
         )
 
         # Watch over each one of the potentially multiple devices per hardware
-        virtual_devices = [
-            evdev.InputDevice(path)
-            for path in paths
-        ]
+        for path in paths:
+            threading.Thread(
+                target=self.start_injecting_worker,
+                args=(path,)
+            ).start()
 
     def read(self):
         """Get the newest key or None if none was pressed."""
@@ -109,9 +128,6 @@ class KeycodeReader:
         return newest_keycode
 
 
-# keycode_reader = KeycodeReader()
-
-
 _devices = None
 
 
@@ -122,6 +138,7 @@ class GetDevicesProcess(multiprocessing.Process):
     asynchronously so that they can take as much time as they want without
     slowing down the initialization. To avoid evdevs asyncio stuff spamming
     errors, do this with multiprocessing and not multithreading.
+    TODO to threading, make eventloop
     """
     def __init__(self, pipe):
         """Construct the process.

keymapper/mapping.py

@@ -25,6 +25,26 @@
 from keymapper.logger import logger
 
 
+# if MIN_KEYCODE < 255 and MAX_KEYCODE > 255: X crashes
+MAX_KEYCODE = 255
+MIN_KEYCODE = 8
+
+
+def get_input_keycode(keycode):
+    """Same as get_output_keycode, but vice versa."""
+    return keycode - MIN_KEYCODE
+
+
+def get_target_keycode(character):
+    # see if any modifiers are inside
+    if 'shift' in character.lower():
+        # yes, now try to return what a normal keyboard would have for that
+        # (for shift it would usually be 50)
+        system_keycode = system_mapping.find_keycode(character)
+        if custom_mapping.get_character(system_mapping) is not None:
+            # already taken!
+
+
 class Mapping:
     """Contains and manages mappings.
 
@@ -42,6 +62,21 @@ class Mapping:
     def __len__(self):
         return len(self._mapping)
 
+    def find_keycode(self, character, case=False):
+        """For a given character, find the used keycode in the mapping."""
+        # TODO test
+        if not case:
+            character = character.lower()
+        for keycode, (mapped_keycode, mapped_character) in self._mapping:
+            # keycode is what the system would use for that key,
+            # mapped_keycode is what we use instead by writing into /dev,
+            # and mapped_character is what we expect to appear.
+            # mapped_character might be multiple things, like "a, A"
+            if not case:
+                mapped_character = mapped_character.lower()
+            if character in [c.strip() for c in mapped_character.split(',')]:
+                return keycode, mapped_keycode
+
     def change(self, previous_keycode, new_keycode, character):
         """Replace the mapping of a keycode with a different one.
 
@@ -71,19 +106,9 @@ class Mapping:
                 logger.error('Cannot use %s as keycode', previous_keycode)
                 return False
 
-        if isinstance(character, list):
-            character = ', '.join([str(c) for c in character])
-
         if new_keycode and character:
-            target_keycode = new_keycode + 256
-            if target_keycode >= 512:
-                # because key-mappers keycodes file has a maximum of 511,
-                # while all system keycodes have a maximum of 255.
-                # To avoid clashes, keycodes <= 255 should not be injected.
-                raise ValueError(
-                    f'Expected target_keycode {target_keycode} to not '
-                    f'be >= 512. '
-                )
+            target_keycode = get_target_keycode(character)
+
             self._mapping[new_keycode] = (target_keycode, str(character))
             if new_keycode != previous_keycode:
                 # clear previous mapping of that code, because the line
@@ -94,6 +119,16 @@ class Mapping:
 
         return False
 
+    def write_from_keymapper_symbols(self, keycode, character):
+        """Write something from a key-mapper symbols file into the mapping."""
+        keycode = int(keycode)
+        if keycode <= 255:
+            logger.error(
+                'Expected keycodes in key-mapper symbols to be > 255 ',
+                f'but got {keycode} for "{character}"'
+            )
+        self._mapping[get_input_keycode(keycode)] = (keycode, character)
+
     def clear(self, keycode):
         """Remove a keycode from the mapping.
 
@@ -127,3 +162,6 @@ class Mapping:
 # one mapping object for the whole application that holds all
 # customizations
 custom_mapping = Mapping()
+
+# one mapping that represents the xmodmap output
+system_mapping = Mapping()

tests/testcases/mapping.py

@@ -35,33 +35,39 @@ class TestMapping(unittest.TestCase):
         self.assertTrue(self.mapping.changed)
         self.assertIsNone(self.mapping.get_character(1))
         self.assertEqual(self.mapping.get_character(2), 'a')
+        self.assertEqual(self.mapping.get_keycode(2), 258)
         self.assertEqual(len(self.mapping), 1)
 
         # change 2 to 3 and change a to b
         self.mapping.change(2, 3, 'b')
         self.assertIsNone(self.mapping.get_character(2))
         self.assertEqual(self.mapping.get_character(3), 'b')
+        self.assertEqual(self.mapping.get_keycode(3), 259)
         self.assertEqual(len(self.mapping), 1)
 
         # add 4
         self.mapping.change(None, 4, 'c')
         self.assertEqual(self.mapping.get_character(3), 'b')
         self.assertEqual(self.mapping.get_character(4), 'c')
+        self.assertEqual(self.mapping.get_keycode(4), 260)
         self.assertEqual(len(self.mapping), 2)
 
         # change the mapping of 4 to d
         self.mapping.change(None, 4, 'd')
         self.assertEqual(self.mapping.get_character(4), 'd')
+        self.assertEqual(self.mapping.get_keycode(4), 260)
         self.assertEqual(len(self.mapping), 2)
 
         # this also works in the same way
         self.mapping.change(4, 4, 'e')
         self.assertEqual(self.mapping.get_character(4), 'e')
+        self.assertEqual(self.mapping.get_keycode(4), 260)
         self.assertEqual(len(self.mapping), 2)
 
         # and this
         self.mapping.change('4', '4', 'f')
         self.assertEqual(self.mapping.get_character(4), 'f')
+        self.assertEqual(self.mapping.get_keycode(4), 260)
         self.assertEqual(len(self.mapping), 2)
 
         # non-int keycodes are ignored
@@ -71,10 +77,13 @@ class TestMapping(unittest.TestCase):
     def test_change_multiple(self):
         self.mapping.change(None, 1, ['a', 'A'])
         self.assertEqual(self.mapping.get_character(1), 'a, A')
+        self.assertEqual(self.mapping.get_keycode(1), 257)
 
         self.mapping.change(1, 2, ['b', 'B'])
         self.assertEqual(self.mapping.get_character(2), 'b, B')
+        self.assertEqual(self.mapping.get_keycode(2), 258)
         self.assertIsNone(self.mapping.get_character(1))
+        self.assertIsNone(self.mapping.get_keycode(1))
 
     def test_clear(self):
         # does nothing
@@ -94,7 +103,7 @@ class TestMapping(unittest.TestCase):
         self.assertIsNone(self.mapping.get_character(20))
         self.assertEqual(self.mapping.get_character(30), 'KP_3')
 
-    def test_iterate_and_convert(self):
+    def test_iterate_and_convert_to_string(self):
         self.mapping.change(None, 10, 1)
         self.mapping.change(None, 20, 2)
         self.mapping.change(None, 30, 3)