wip

HELP.md

@@ -5,15 +5,15 @@ keycode 10, and one for your keyboard that is normal and writes 1/! on
 keycode 10, then you would not be able to write ! by pressing that mouse
 button and that keyboard button at the same time. Keycodes may not clash.
 
-The first idea was to write special keycodes known only to key-mapper
+**The first idea** was to write special keycodes known only to key-mapper
-(256 - 511) into an input device in /dev/input, and map those to SHIFT and
+(256 - 511) into the input device of your mouse in /dev/input, and map
-such, whenever a button is clicked. A mapping would have existed to prevent
+those to SHIFT and such, whenever a button is clicked. A mapping would have
-the original keycode 10 from writing a 1. But X seems to ignore anything
+existed to prevent the original keycode 10 from writing a 1. But X/Linux seem
-greater than 255, or even crash in some cases, for regular keyboard events.
+to ignore anything greater than 255 for regular keyboard events, or even
-Mouse buttons can use those though, but they cannot be remapped, which I
+crash in some cases. Mouse click buttons can use those high keycodes though,
-guess is another indicator of that.
+but they cannot be remapped, which I guess is another indicator of that.
 
-The second idea is to create a new input device that uses 8 - 255, just like
+**The second idea** is to create a new input device that uses 8 - 255, just like
 other layouts, and key-mapper always tries to use the same keycodes for
 SHIFT as already used in the system default. The pipeline is like this:
 
@@ -29,6 +29,34 @@ SHIFT as already used in the system default. The pipeline is like this:
 4. X has another config for "mouse" loaded, which prevents any system default
    mapping to print the overwritten key "1" into the session.
    
+But this is a rather complicated approach. The mapping of 10 -> 50 would
+have to be stored somewhere as well.
+
+**Third idea**: Based on the first idea, instead of using keycodes greater
+than 255, use unused keycodes starting from 255, going down. Issues existed
+when two buttons with the same keycode are pressed at the same time,
+so the goal is to avoid such overlaps. For example, if keycode 10 should be
+mapped to Shift_L. It is impossible to write "!" using this mapped button
+and a second keyboard, except if pressing key 10 triggers key-mapper to write
+key 253 into the /dev device, while mapping key 10 to nothing. Unfortunately
+linux just completely ignores some keycodes. 140 works, 145 won't, 150 works.
+
+So back to the second idea.
+
+# The various mappings
+
+There are three mappings:
+
+The first one is in the keycodes file and contains "<10> = 10", which is
+super redundant but needed for xkb.
+
+The second one maps "<10>" to characters, modifiers, etc. using symbol files
+in xkb.
+
+The third mapping reads the input keycodes from your mouse (also known as
+system_keycode here) and writes a different one into /dev (also known as
+target_keycode here). It is explained above why.
+
 # How I would have liked it to be
 
 setxkbmap -layout ~/.config/key-mapper/mouse -device 13
@@ -39,8 +67,9 @@ config looks like:
 11 = Shift_L
 ```
 
-done. Without crashing X. Without printing generic useless errors. If it was
+done. Without crashing X. Without printing generic useless errors. Without
-that easy, an app to map keys would have already existed.
+colliding with other devices using the same keycodes. If it was that easy,
+an app to map keys would have already existed.
 
 # Folder Structure of Key Mapper in /usr
 

keymapper/X.py

@@ -39,7 +39,7 @@ import subprocess
 
 from keymapper.paths import get_usr_path, KEYCODES_PATH, DEFAULT_SYMBOLS, \
     X11_SYMBOLS
-from keymapper.logger import logger
+from keymapper.logger import logger, is_debug
 from keymapper.data import get_data_path
 from keymapper.linux import get_devices
 from keymapper.mapping import custom_mapping, system_mapping, \
@@ -80,13 +80,6 @@ def create_preset(device, name=None):
 def create_setxkbmap_config(device, preset):
     """Generate a config file for setxkbmap.
 
-    The file is created in ~/.config/key-mapper/<device>/<preset> and,
-    in order to find all presets in the home dir to make backing them up
-    more intuitive, a symlink is created in
-    /usr/share/X11/xkb/symbols/key-mapper/<device>/<preset> to point to it.
-    The file in home doesn't have underscore to be more beautiful on the
-    frontend, while the symlink doesn't contain any whitespaces.
-
     Parameters
     ----------
     device : string
@@ -194,7 +187,7 @@ def setxkbmap(device, layout):
 
         device_cmd = cmd + ['-device', str(xinput_id)]
         logger.debug('Running `%s`', ' '.join(device_cmd))
-        subprocess.run(device_cmd, capture_output=True)
+        subprocess.run(device_cmd, capture_output=(not is_debug()))
 
 
 def create_identity_mapping():
@@ -210,8 +203,6 @@ def create_identity_mapping():
         return
 
     xkb_keycodes = []
-    # the maximum specified in /usr/share/X11/xkb/keycodes is usually 255
-    # and the minimum 8 TODO update comment
     maximum = MAX_KEYCODE
     minimum = MIN_KEYCODE
     for keycode in range(minimum, maximum + 1):
@@ -237,7 +228,9 @@ def create_identity_mapping():
         keycodes.write(result)
 
 
-def generate_symbols(name, include=DEFAULT_SYMBOLS_NAME, mapping=custom_mapping):
+def generate_symbols(
+    name, include=DEFAULT_SYMBOLS_NAME, mapping=custom_mapping
+):
     """Create config contents to be placed in /usr/share/X11/xkb/symbols.
 
     It's the mapping of the preset as expected by X. This function does not
@@ -267,12 +260,32 @@ def generate_symbols(name, include=DEFAULT_SYMBOLS_NAME, mapping=custom_mapping)
         keycodes = re.findall(r'<.+?>', f.read())
 
     xkb_symbols = []
-    for _, (keycode, character) in mapping:
+    for system_keycode, (target_keycode, character) in mapping:
-        if f'<{keycode}>' not in keycodes:
+        if f'<{system_keycode}>' not in keycodes:
-            logger.error(f'Unknown keycode <{keycode}> for "{character}"')
+            logger.error(f'Unknown code <{system_keycode}> for "{character}"')
             # don't append that one, otherwise X would crash when loading
             continue
-        xkb_symbols.append(f'key <{keycode}> {{ [ {character} ] }};')
+
+        # key-mapper will write target_keycode into /dev, while
+        # system_keycode should do nothing to avoid a duplicate keystroke.
+        print('writing', system_keycode, target_keycode, character)
+        if target_keycode is not None:
+            if f'<{target_keycode}>' not in keycodes:
+                logger.error(f'Unknown code <{target_keycode}> for "{character}"')
+                # don't append that one, otherwise X would crash when loading
+                continue
+            xkb_symbols.append(
+                f'key <{system_keycode}> {{ [ ] }}; '
+            )
+            xkb_symbols.append(
+                f'key <{target_keycode}> {{ [ {character} ] }}; '
+                f'// {system_keycode}'
+            )
+            continue
+
+        xkb_symbols.append(
+            f'key <{system_keycode}> {{ [ {character} ] }}; '
+        )
 
     if len(xkb_symbols) == 0:
         logger.error('Failed to populate xkb_symbols')
@@ -329,13 +342,22 @@ def parse_symbols_file(device, preset):
         # from "key <12> { [ 1 ] };" extract 12 and 1,
         # from "key <12> { [ a, A ] };" extract 12 and [a, A]
         # avoid lines that start with special characters
-        # (might be comments)ś
+        # (might be comments)
+        # And only find those lines that have a system-keycode written
+        # after them, because I need that one to show in the ui.
         content = f.read()
-        result = re.findall(r'\n\s+?key <(.+?)>.+?\[\s+(.+?)\s+\]', content)
+        result = re.findall(
+            r'\n\s+?key <(.+?)>.+?\[\s+(.+?)\s+\]\s+?}; // (\d+)',
+            content
+        )
         logger.debug('Found %d mappings in preset "%s"', len(result), preset)
-        for keycode, character in result:
+        for target_keycode, character, system_keycode in result:
-            keycode = int(keycode)
+            custom_mapping.change(
-            custom_mapping.write_from_keymapper_symbols(keycode, character)
+                previous_keycode=None,
+                new_keycode=system_keycode,
+                character=character,
+                target_keycode=int(target_keycode)
+            )
         custom_mapping.changed = False
 
 
@@ -343,8 +365,14 @@ def parse_xmodmap():
     """Read the output of xmodmap as a Mapping object."""
     xmodmap = subprocess.check_output(['xmodmap', '-pke']).decode() + '\n'
     mappings = re.findall(r'(\d+) = (.+)\n', xmodmap)
+    # TODO is this tested?
     for keycode, characters in mappings:
-        system_mapping.change(None, int(keycode), characters.split())
+        system_mapping.change(
+            previous_keycode=None,
+            new_keycode=int(keycode),
+            character=', '.join(characters.split()),
+            target_keycode=None
+        )
 
 
 # TODO verify that this is the system default and not changed when I

keymapper/gtk/row.py

@@ -27,7 +27,7 @@ gi.require_version('Gtk', '3.0')
 gi.require_version('GLib', '2.0')
 from gi.repository import Gtk
 
-from keymapper.mapping import custom_mapping
+from keymapper.mapping import custom_mapping, DONTMAP, GENERATE
 from keymapper.logger import logger
 
 
@@ -69,7 +69,12 @@ class Row(Gtk.ListBoxRow):
         self.highlight()
 
         if keycode is not None:
-            custom_mapping.change(None, keycode, character)
+            custom_mapping.change(
+                previous_keycode=None,
+                new_keycode=keycode,
+                character=character,
+                target_keycode=GENERATE
+            )
 
     def on_key_pressed(self, button, event):
         """Check if a keycode has been pressed and if so, display it."""
@@ -105,7 +110,12 @@ class Row(Gtk.ListBoxRow):
             return
 
         # else, the keycode has changed, the character is set, all good
-        custom_mapping.change(previous_keycode, new_keycode, character)
+        custom_mapping.change(
+            previous_keycode=previous_keycode,
+            new_keycode=new_keycode,
+            character=character,
+            target_keycode=GENERATE
+        )
 
     def put_together(self, keycode, character):
         """Create all child GTK widgets and connect their signals."""

keymapper/gtk/window.py

@@ -29,7 +29,8 @@ from gi.repository import Gtk, Gdk, GLib
 
 from keymapper.data import get_data_path
 from keymapper.X import create_setxkbmap_config, apply_preset, \
-    create_preset, custom_mapping, parse_symbols_file, setxkbmap
+    create_preset, custom_mapping, system_mapping, parse_symbols_file, \
+    setxkbmap
 from keymapper.presets import get_presets, find_newest_preset, \
     delete_preset, rename_preset
 from keymapper.logger import logger

keymapper/linux.py

@@ -71,29 +71,31 @@ class KeycodeReader:
         """
         loop = asyncio.new_event_loop()
         asyncio.set_event_loop(loop)
-        # TODO use uinput instead, then register somewhere its existance in
-        #  order to setxkbmap it. Or give it a constant path that is known
-        #  application wide
         device = evdev.InputDevice(path)
-        uinput = evdev.UInput()
+        keymapper_device = evdev.UInput()
 
         for event in device.read_loop():
             if event.type != evdev.ecodes.EV_KEY:
                 continue
 
             # this happens to report key codes that are 8 lower
-            # than the ones reported by xev
+            # than the ones reported by xev and that X expects
             input_keycode = event.code + 8
-            output_keycode = custom_mapping.get_keycode(input_keycode) - 8
 
-            print(input_keycode, output_keycode)
+            if custom_mapping.get_keycode(input_keycode) is None:
-
+                # unknown keycode, skip
-            if output_keycode > MAX_KEYCODE or output_keycode < MIN_KEYCODE:
                 continue
 
-            # value: 1 for down, 0 for up, 2 for hold.
+            target_keycode = custom_mapping.get_keycode(input_keycode)
-            device.write(evdev.ecodes.EV_KEY, output_keycode, event.value)
+
-            device.write(evdev.ecodes.EV_SYN, evdev.ecodes.SYN_REPORT, 0)
+            if target_keycode > MAX_KEYCODE or target_keycode < MIN_KEYCODE:
+                continue
+
+            print('read', input_keycode, 'write', target_keycode, path)
+
+            # TODO test for the stuff put into write
+            keymapper_device.write(evdev.ecodes.EV_KEY, target_keycode, event.value)
+            keymapper_device.syn()
 
     def start_injecting(self):
         """Read keycodes and inject the mapped character forever."""

keymapper/logger.py

@@ -63,6 +63,10 @@ logger.addHandler(handler)
 logger.setLevel(logging.INFO)
 
 
+def is_debug():
+    return logger.level == logging.DEBUG
+
+
 def log_info():
     """Log version and name to the console"""
     # read values from setup.py

keymapper/mapping.py

@@ -26,23 +26,40 @@ from keymapper.logger import logger
 
 
 # if MIN_KEYCODE < 255 and MAX_KEYCODE > 255: X crashes
+# the maximum specified in /usr/share/X11/xkb/keycodes is usually 255
+# and the minimum 8
 MAX_KEYCODE = 255
 MIN_KEYCODE = 8
 
 
+# modes for change:
+GENERATE = -1
+DONTMAP = None
+
+
 def get_input_keycode(keycode):
     """Same as get_output_keycode, but vice versa."""
     return keycode - MIN_KEYCODE
 
 
-def get_target_keycode(character):
+def get_target_keycode():
-    # see if any modifiers are inside
+    # see HELP.md
-    if 'shift' in character.lower():
+    for keycode in range(MAX_KEYCODE, MIN_KEYCODE - 1, -1):
-        # yes, now try to return what a normal keyboard would have for that
+        # starting from the MAX_KEYCODE, find the first keycode that is
-        # (for shift it would usually be 50)
+        # unused in both custom_mapping and system_mapping.
-        system_keycode = system_mapping.find_keycode(character)
+        if not (custom_mapping.has(keycode) or system_mapping.has(keycode)):
-        if custom_mapping.get_character(system_mapping) is not None:
+            return keycode
-            # already taken!
+
+    # no unused keycode found, take the highest keycode that is unused
+    # in the current custom_mapping.
+    for keycode in range(MAX_KEYCODE, MIN_KEYCODE - 1, -1):
+        # starting from the MAX_KEYCODE, find the first keycode that is
+        # unused in both custom_mapping and system_mapping.
+        if not (custom_mapping.has(keycode)):
+            return keycode
+
+    logger.error('All %s keycodes are mapped!', MAX_KEYCODE - MIN_KEYCODE)
+    return None
 
 
 class Mapping:
@@ -52,6 +69,9 @@ class Mapping:
     character.
     """
     def __init__(self):
+        # TODO this is a stupid data structure if there are two keys
+        #  that should be unique individually. system_keycode and
+        #  target_keycode. two _mapping objects maybe?
         self._mapping = {}
         self.changed = False
 
@@ -63,7 +83,7 @@ class Mapping:
         return len(self._mapping)
 
     def find_keycode(self, character, case=False):
-        """For a given character, find the used keycode in the mapping."""
+        """For a given character, find the used keycodes in the mapping."""
         # TODO test
         if not case:
             character = character.lower()
@@ -77,7 +97,7 @@ class Mapping:
             if character in [c.strip() for c in mapped_character.split(',')]:
                 return keycode, mapped_keycode
 
-    def change(self, previous_keycode, new_keycode, character):
+    def change(self, previous_keycode, new_keycode, character, target_keycode):
         """Replace the mapping of a keycode with a different one.
 
         Return True on success.
@@ -85,30 +105,37 @@ class Mapping:
         Parameters
         ----------
         previous_keycode : int or None
-            If None, will not remove any previous mapping.
+            If None, will not remove any previous mapping. If you recently
+            used 10 for new_keycode and want to overwrite that with 11,
+            provide 5 here.
         new_keycode : int
             The source keycode, what the mouse would report without any
             modification.
         character : string or string[]
             If an array of strings, will put something like { [ a, A ] };
             into the symbols file.
+        target_keycode : int or None
+            Which keycode should be used for that key instead. If -1,
+            will figure out a new one. This is for stuff that happens
+            under the hood and the user won't see this unless they open
+            config files. If None, will only map new_keycode to character
+            without any in-between step.
         """
         try:
             new_keycode = int(new_keycode)
-        except ValueError:
+            if target_keycode is not None:
-            logger.error('Cannot use %s as keycode', new_keycode)
+                target_keycode = int(target_keycode)
-            return False
-
             if previous_keycode is not None:
-            try:
                 previous_keycode = int(previous_keycode)
         except ValueError:
-                logger.error('Cannot use %s as keycode', previous_keycode)
+            logger.error('Can only use numbers as keycodes')
             return False
 
-        if new_keycode and character:
+        # TODO test
-            target_keycode = get_target_keycode(character)
+        if target_keycode == GENERATE:
+            target_keycode = get_target_keycode()
 
+        if new_keycode and character:
             self._mapping[new_keycode] = (target_keycode, str(character))
             if new_keycode != previous_keycode:
                 # clear previous mapping of that code, because the line
@@ -119,16 +146,6 @@ class Mapping:
 
         return False
 
-    def write_from_keymapper_symbols(self, keycode, character):
-        """Write something from a key-mapper symbols file into the mapping."""
-        keycode = int(keycode)
-        if keycode <= 255:
-            logger.error(
-                'Expected keycodes in key-mapper symbols to be > 255 ',
-                f'but got {keycode} for "{character}"'
-            )
-        self._mapping[get_input_keycode(keycode)] = (keycode, character)
-
     def clear(self, keycode):
         """Remove a keycode from the mapping.
 
@@ -158,6 +175,21 @@ class Mapping:
         """
         return self._mapping.get(keycode, (None, None))[1]
 
+    def has(self, keycode):
+        """Check if this keycode is going to be a line in the symbols file."""
+        # TODO test
+        if self._mapping.get(keycode) is not None:
+            # the keycode that is disabled, because it is mapped to
+            # something else
+            return True
+
+        for _, (target_keycode, _) in self._mapping.items():
+            if target_keycode == keycode:
+                # the keycode that is actually being mapped
+                return True
+
+        return False
+
 
 # one mapping object for the whole application that holds all
 # customizations