Bug 171295 - Allow shortcuts for input devices other than the keyboard (mouse, lirc, bluetooth, joystick, etc)
Summary: Allow shortcuts for input devices other than the keyboard (mouse, lirc, bluet...
Status: RESOLVED FIXED
Alias: None
Product: systemsettings
Classification: Applications
Component: kcm_keys (show other bugs)
Version: 4.1
Platform: openSUSE Unspecified
: NOR wishlist
Target Milestone: ---
Assignee: System Settings Bugs
URL:
Keywords:
: 207920 211205 403932 (view as bug list)
Depends on: 109374
Blocks: 440076
  Show dependency treegraph
 
Reported: 2008-09-18 22:17 UTC by Todd
Modified: 2024-03-11 04:12 UTC (History)
32 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Todd 2008-09-18 22:17:08 UTC
Version:            (using KDE 4.1.1)
Installed from:    SuSE RPMs

Currently the systemsettings "Keyboard Shortcuts" module only allows assigning shortcuts to combinations of keyboard button presses, as its name implies.  This is pretty limited considering the wide and constantly increasing variety of input devices currently available.  Currently we have keyboards, mice, lirc, bluetooth, joysticks and gamepads, and a variety of more specialized input devices.  Compiz, for instance, allows shortcuts that involve the keyboard and a separate set of shortcuts that involve the mouse (and modifier keys).  This is better than what KDE 4 supports, but is still not optimal in my opinion.  

I think the best solution is that the shortcuts system simply ignores what input devices the button press comes from and handles them all together in the same manner.  Currently when you click the button to assign a shortcut it waits for a the keyboard button press combination and then that shows up in the button.  What I am suggesting is that instead it waits for a button press combination from all input devices and stores that.  So for instance if someone presses a keyboard button combination, that will be the shortcut.  If they press a button or button combination on their lirc remote, that will be the shortcut.  If they press a button or button combination on their mouse, that will be the shortcut.  They could also press a button combination involving their keyboard and their mouse, for instance.  It would not even know what devices the button belongs to, it would only know it as a button

I have some specific use cases.  

Someone has the "present" plugin enabled in kwin.  Since they generally use their mouse when navigating windows and not their keyboard, they would prefer to use the mouse over the keyboard, or use the mouse and a single keyboard button.  In the first case they want the present plugin to activate when they press the back and forward buttons on the mouse simultaneously.  In the second case they want it assigned to a combination of the "win" key and the middle mouse button.  Both cases would be handled the same way, just press the desired combination and then release it.

Another situation is someone wanting to play kgoldrunner with their gamepad.  Instead of needing a separate joystick configuration system to assign joystick buttons to keyboard presses, the user can just open up kgoldrunner's shortcut configuration and assign the gamepad buttons to various functions directly.

Another situation is someone who has an lirc remote and wants to control dragon player, amarok, juk, or some other kde multimedia program.  Currently they have to open either a text file or a separate configuration program and map the lirc buttons to various functions in the program or to various keyboard functions.  They cannot do it within the media player itself.  With this system they could just open up the local or global shortcut configuration (depend on their goals) and push the lirc button they want to assign to each function.  The same would be possible for bluetooth remotes, human interface devices, or even bluetooth cell phones that support the Audio/Video Remote Control profile or the Human Interface Device profile.  They could even use a joystick to control their music.

This allows you to consolidate four or five different configuration programs (keyboard shortcuts, lirc, joystick/gamepad, and one or two bluetooth) into a single, consistent interface that is present in every KDE program.  The people making lirc configurations or configuration programs, for instance, don't need to keep track of how to interface with all the different KDE multimedia programs, that interface is built into KDE and the user can set it up however he or she wants.  Users don't have to map gamepad or joystick buttons to keyboard commands, KDE games can use the gamepad or joystick directly.

This would also require having multiple button combinations be able to be assigned to a single shortcut, since people may want to do the same command in different ways.  For instance people might want to use the play/pause button on their keyboard, on their lirc remote, and on their mouse (there are a few mice that have this I think).

Even analog devices like joysticks (as opposed to digital joystick buttons) could be used if the system was set up to properly interpret and then pass along analog data.

Slightly off-topic, but an extension to this would be the ability to link button presses.  For instance a certain button on the keyboard, a certain button on an lirc remote, and a certain button on a mouse are all play buttons.  User would be able to set up "actions", where different buttons or button combinations are all treated by the shortcut system as being the same.  In this example all these buttons could be assigned to a certain action, which the user would label "play" (there could be a default list of actions available as well).  All three of these buttons are then considered to be the same button by any program using the KDE system.  If you open, say, amarok and assign the "play" shortcut to the play button on the keyboard, amarok will also recognize the play button on the remote and the play button on the mouse as being assigned to that shortcut as well.  

I understand that this is easy to describe but probably extremely difficult to actually implement.  I don't have any expectation that it will be coming out soon if it is even accepted.
Comment 1 Todd 2008-10-02 07:15:32 UTC
As an extension of this idea, I think it would be great of the shortcut system used a extendable backend-based system.  This would be similar to, say, solid or phonon where anyone can write their own backends for the shortcut system to extend the sort of hardware or software that can provide shortcuts.

An example would be a mouse gesture backend.  Instead of having to make their own interface to KDE they can make a backend that would allow the mouse gesture to be used directly in the shortcut system.  The backend would interpet the mouse gesture and translated each mouse gesture into a unique signal or identifier of some sort that the shorcut system can interpret.  There would still need to be a configuration dialog to learn mouse gesture and give each one a unique identifier, but the developer would not have to code their own interface for every KDE programs he or she wants to be able to interact with nor would he or she need to use any ugly hacks like converting the gesture to keyboard shortcuts and then assigning those keyboard shortcuts to the KDE shortcut system.  The programmer would only need a way to translate each specific mouse gesture into something unique to that gesture that could be used to identify it in the shortcut system.  Then the user could assign the gesture to any shortcut he or she might want.

Similarly if someone made a voice-control system for kde it could be implemented as a backend, and the voice control system would convert each voice command into a unique identifier that could be used by the shortcut system.  Once again the programmer would only need a way to uniquely identify each voice command, the shortcut system would do all the work of listening for the identifiers and acting on them when they are present.

Other possible backends include remote control over a network, unusual devices like the wii remote or custom serial port controllers, and even interfaces that haven't been thought up yet.  And it wouldn't have to be a hardware interface, programs could be used to provide the signals for the shortcut system.  So, although this is often a roundabout way of doing it, you could have a timer backend that sends a particular identifier at a preset time.  You could use this to stop a media player at a particular time of day, or to automatically lock your computer when the building closes.  You could also have a powerdevil interface that triggers identifiers based on certain events like a laptop lid closing.  Once again, this is probably not the most efficient way to go about this but it is an example of the sorts of things you can do with this sort of system.
Comment 2 Matt Whitlock 2008-10-07 09:10:38 UTC
*** This bug has been confirmed by popular vote. ***
Comment 3 Matt Whitlock 2008-10-07 09:22:54 UTC
This is fantastic.  A lot of commercial games already support this in their input configuration.  Click the action you want to change and press a keyboard key, click a mouse button, or push a joystick button; they are all handled interchangeably.  This is what KDE Shortcuts should do.

At present, to achieve the desired behavior, I have to run xbindkeys to listen for mouse button presses and generate "fake" keyboard input using the XTEST extension utility 'xte'.  This is a suboptimal solution, since there can be some delay to fork and exec the xte process when the system is under heavy load.  KDE ought to simply react to the button presses itself without the need for this extra layer of hackery.

I have 12 buttons on my mouse: Left (1), Middle (2), Right (3), Scroll Up (4), Scroll Down (5), Back (8), Forward (9), Window (10), Page Up (11), Page Down (12), Tilt Left (13), Tilt Right (14).  Konqueror can't even react to the "Back" mouse button because of this deficiency in the KDE Shortcuts subsystem.  If I could assign "Button 8" to the "Back" action in Konqueror, then I could use the "Back" button on my mouse to go back in Konqueror's history.  As it stands now, I have to use xbindkeys and xte to generate a fake XF86Back key press in response to a button 8 click, and then bind XF86Back to Konqueror's "Back" action.
Comment 4 Todd 2008-10-21 16:58:24 UTC
This also eliminates the overlap between input actions and shortcuts.  Input actions can be used just for controlling things that do not use the shortcut system.  There would be no need to map things like gestures to keyboard shortcuts.  The shortcut module would be the only thing needed to control shortcuts, you would not need to move back and forth between the two modules to get, say, mouse gestures working with kwin.
Comment 5 Cyrill Helg 2008-11-09 13:57:29 UTC
This sounds interesting. I'm really missing the lirc support in kde4, especially to control amarok for example. 

So, is anyone working on something in this direction?
Comment 6 lucas 2008-11-29 04:05:21 UTC

*** This bug has been marked as a duplicate of bug 96431 ***
Comment 7 Todd 2008-11-29 22:19:27 UTC
This isn't a duplicate of bug 96431.  Bug 96431 only deals with mouse shortcuts, this deals with shortcuts using any input device, including the mouse but also including lirc, bluetooth, joystick.  It depends on 96431 but is not the same thing.
Comment 8 Cyrill Helg 2008-12-02 14:22:39 UTC
I agree this bug should not be closed. There are many ideas in here that should be considered.
Comment 9 Todd 2008-12-04 17:16:40 UTC
Lucas, can you explain why think this bug is a duplicate?
Comment 10 Todd 2009-01-05 23:46:38 UTC
So far I still cannot see how this bug is a duplicate.  Although this bug does include mouse-based shortcuts, and thus probably depends on bug 96431, it includes a number of other ideas that are not found in bug 96431.  Even if that bug is fixed it won't help with with bluetooth devices, lirc, joysticks, or anything else besides mouse buttons.
Comment 11 lejocelyn 2009-10-29 11:58:51 UTC
This wish is still for KDE 4.3 and more.

I'd love to see this feature implemented. Thank you Matt Whitlock,Comment #3,   2008-10-07 09:22:54. This will help me doing it while waiting for this feature.
Comment 12 Ben Cooksley 2009-11-23 05:07:17 UTC
*** Bug 207920 has been marked as a duplicate of this bug. ***
Comment 13 Ben Cooksley 2009-11-23 05:07:31 UTC
*** Bug 211205 has been marked as a duplicate of this bug. ***
Comment 14 Lach Sławomir 2010-12-25 08:43:12 UTC
I doubt not better is creating different deamons with different configuration UI, but all of them will reads KHotKeys configuration(reads the same actions).
Comment 15 Franco Pellegrini 2011-03-30 00:52:42 UTC
I currently have up to 3 fingers support in my Synaptics touchpad. I would love a functionality like this, so i can add 2 and 3 finger gestures to present all windows, etc (somthing like osx does)
Comment 16 Rick Stockton 2011-12-09 00:09:38 UTC
Todd: As you know, the prerequisite Qt enhancement is already in place (for mice only, and only under XLIB and XCB. I'll be doing Wayland also, and I might try struggling with Windows later.)

I would vote to forget about joystick support. Do just the mouse buttons for now, but design the code to handle additional devices in the future. (Including a SECOND MOUSE.)
 
I can quickly write a new QJoyStickEvent() Class into Qt Core and the Plugins. For Core, XLIB, and XCB, that's roughly a 12-hour job. (IF AND ONLY IF I can  assume that evdev joystick events have identical mapping and capabilities as the more important Synaptics devices -- after the Synaptics Driver and X11 have finished 'messing' with them.)

And I already own a Logitech Attack-3. But you all would need to remember: Even though we could support two types of devices, you can only have *** ONE! *** pointer device on a Qt desktop! There's no way yet, in Qt5, to distinguish between motion events or location values from different devices. And even more important, the basic concept of 'pointer focus' for Qt Widgets and Areas would need an overhaul.

With this limitation in mind, I think that we really, really want people to use mice devices: they're more accurate in motion, more ergonomic, AND they have more buttons too.
Comment 17 Rick Stockton 2011-12-09 00:33:06 UTC
Or, if we're willing to ignore location and motion (except when 'emulated' via button presses, with fixed distances and orientations per press), then we can handle just the buttons.

Kernel-based 'LIRC' probably goes through evdev too, I could do that as well. But overall, I really, really like the 'key elements' of your proposal:

(1) Multiple shortcuts invoking the same action;
(2) Shortcuts which perform an action WITHOUT going through keyboard emulation.

Let me note a feature which I'd need, to avoid a lot of unnecessary confused about my shortcuts: More ways of indexing the on-screen 'view'. Here are some examples:

   A) list all of the shortcuts for a given Action;

   B) list all of the shortcuts on a given device;

   C) list shortcut Actions (by category, and by program) which are in use, and also those which are available to define (but not yet defined on any device or combination of devices).

   D) list available/unused buttons on a given button-oriented Device;

   E) show the hierarchy of Actions within a given program as we have now.
Comment 18 Rick Stockton 2011-12-09 00:42:56 UTC
Please advise whether you still see a need for Joysticks, IR remote controls, and etc. if they can only be "button devices" (and not Pointer Devices).

Can you lead on this? I have no skill in GUI design, and no idea when KF5 will be ready for high-level coding. This bug, or a private EM, or start a Wiki page - they all work for me, and I'd like to help on this.
Comment 19 Todd 2011-12-12 00:05:38 UTC
Just to be clear up front, this is about buttons, not pointers.  Having a pointers framework might be nice, but is outside the scope of this.

My vision of how this idea would be implemented would be to create a generic, probably DBUS-based, button handling system inside KDE (or Qt).  This system would be plugin-based, with each plugin providing button events in a consistent manner.  The existing KDE keyboard shorcut system would change to be a generic receiver for these events.  It would not care what plugin provides the events, nor would individual applications using the shortcut system.  

Under this structure, the only thing that would be required for the initial Frameworks 5 release would be to implement the plugin system and convert the existing keyboard system to such a plugin.  

Since you have the mouse stuff already that would also be nice, but would not need to be available right away.  Other input devices like joysticks and LIRC remotes could be ported over time later, they would not need to be done right away or need to be ported entirely at one time (for instance the joystick support could be added at one point, and games could port over to using the new shortcut system instead of internal joystick button handlers one at a time later).

The very rough strawman version of the framework follows:

Plugins can send four types of events:

1. Register plugin: this tells the system a new plugin is available.  The plugin would need to provide its name (names should be unique, the system will need to tell the plugin the name is being used so it can pick another), a user-readable name, and probably a user-readable description or comment.

2. Register device: new devices need to be registered.  This should include the device type (mouse, keyboard, joystick, etc), a unique name (once again checked for collisions), a user-readable name, and the unique name of the plugin.

3. Unregister device: when a device is removed, gives the name, says whether to keep the device or forget it (the latter removes all shortcuts using that device), and the unique name of the plugin.

4. Button event: this is sent when a button event takes place.  It needs the following information: 

a. unique plugin name
b. unique device name
c. unique button name (this is NOT checked for collisions, it is assumed the plugin will handle this)
d. human-readable button name
e. event type (press or release)
f. button type: can be one of the following:
   - meta button: like the ctrl or alt key on the keyboard, they cannot be used in shortcuts on their own but can be combined with at least one other non-meta button (and any number of additional meta buttons)
   - standard button: the equivalent of a letter or number key on a keyboard, they cannot be used in shortcuts on their own but can be used in combination with one or more meta keys and one or more other keys.  The left, right, and middle mouse buttons would be in this category.
   - special button: the equivalent of the multimedia keys on a keyboard.  They can be used on their own and/or in combination with other buttons of any type

The shortcut system then takes all of the information from the button event, combines it, and passes it to the application in the following form:

1. long button name (combines the plugin, device, and button name)
2. human-readable name (the same as the name provided by the plugin)

The reason for this is to provide more flexibility down the road.  For instance, kremotecontrol has support for profiles, allowing a single remote to essentially have multiple sets of shortcuts.  This might be good to integrate into the shortcut system as a whole later.  If that happened, then the applications shouldn't need to care, it would just be possible to lengthen the long button name without the applications even knowing (they will just see a new set of names). 

This also means that shortcuts can combine buttons from different devices (like alt+meta+left mouse button).  

Also note that there is no mechanism for configuring plugins.  Plugins should handle their configuration on their own (through KCM modules, probably).  

How to divide up devices is also up to the plugin.  For instance an LIRC plugin might want to treat the play button from two remotes as separate buttons, while a keyboard plugin would probably want the shift button from different keyboards to be treated as the same button (to make it easier for laptop users who connect an external keyboard, for instance).  These sorts of decisions should be handled by the plugin before it sends the events.

So by keeping things pretty general and abstract, the exact behavior of the plugins can be tailored to the particular characteristics of the device or devices being handled.

From what I am told, the existing shortcut system already supports an arbitrary number of button combinations for a given shortcut, it simply a limitation in the GUI that we only get one global shortcut and two local shortcuts.  If that is true, then the gui wouldn't even need to change immediately.  The new shortcut system could be implemented along with a keyboard plugin but keeping essentially the same gui.  The more flexible gui could be implemented later as well.

So from a user perspective, at the very beginning nothing would have to change.  Once Frameworks 5 has stabilized, it would be possible to improve the GUI and start porting other input devices to use this system.
Comment 20 Ward 2012-01-13 22:42:19 UTC
I want to be able to scroll vertically by holding the Alt key and scroll with my scroll wheel.
Comment 21 Rick Stockton 2012-01-14 20:31:24 UTC
Ward, I think that you're slightly OT. (Asking about a particular shortcut, in a bug about a 'big scheme' for ALL shortcuts.) But I'm confused by your comment. Please clarify:

Scroll wheel defaults to executing a vertical scroll just about EVERYWHERE, right? Every Application, Every multi-choice menu with a vertical layout, Every Panel which provides vertical scroll of horizontal elements (e.g., web Browsers), and so on. So, why/when do you need to Alt key (or some other modifier) to 'modify' Wheel behavior?

I'm missing your point - please advise further :)
Comment 22 matthias sweertvaegher 2013-09-19 21:37:39 UTC
about joysticks and lirc, my 2 use cases: 
my gamepad cable reaches until my sofa so I could comfortably use it to pause a video, change volume etc. It has lots of buttons and axis => lots of possibilities

when I'm in the kitchen I'd like to be able to change the volume of the music playing through my IR remote. This used to work but now it has become a hassle to setup in KDE..

So I'm in favor of not dropping joystick and lirc support.
Comment 23 David Edmundson 2019-02-05 10:35:02 UTC
*** Bug 403872 has been marked as a duplicate of this bug. ***
Comment 24 David Edmundson 2019-02-05 10:35:37 UTC
*** Bug 403932 has been marked as a duplicate of this bug. ***
Comment 25 Nate Graham 2020-04-30 17:21:06 UTC
I'd really like this particularly for the case of many-button mice. I'd like to be able to assign the side keys to "next/previous" tab.
Comment 26 Matt Whitlock 2020-04-30 18:38:20 UTC
(In reply to Nate Graham from comment #25)
> I'd really like this particularly for the case of many-button mice. I'd like
> to be able to assign the side keys to "next/previous" tab.

I have a mouse with 12 evdev buttons (scroll up and scroll down are "buttons," even though they're physically triggered by a wheel), and I use 'xbindkeys' and 'xte' to map them all to useful actions. I have 'xbindkeys' in my session's 'autostart' configuration, and my ~/.xbindkeys contains the following:

"xte 'key XF86Back'"
	b:8

"xte 'key XF86Forward'"
	b:9

"xte 'key XF86Search'"
	b:10

"xte 'keydown Super_L' 'key Prior' 'keyup Super_L'"
	b:11

"xte 'keydown Super_L' 'key Next' 'keyup Super_L'"
	b:12

"xte 'keydown Control_L' 'key Prior' 'keyup Control_L'"
	b:13

"xte 'keydown Control_L' 'key Next' 'keyup Control_L'"
	b:14

"killall -HUP gpg-agent ; dbus-send --type=method_call --dest=org.kde.kwalletd5 /modules/kwalletd5 org.kde.KWallet.closeAllWallets ; dbus-send --type=method_call --dest=org.kde.kwalletd /modules/kwalletd org.kde.KWallet.closeAllWallets ; dbus-send --type=method_call --dest=org.freedesktop.ScreenSaver /ScreenSaver org.freedesktop.ScreenSaver.Lock ; sleep 1 ; xset dpms force off"
	Release + XF86Sleep

"sleep 1 ; xset dpms force off"
	Shift + Release + XF86Sleep


This gives me the following mappings:
- Rocking the school wheel to the left or right switches to the previous or next tab in most applications.
- Clicking the button above or below the scroll wheel switches to the next or previous virtual desktop.
- Pushing the forward or back thumb button goes forward or back in the navigation history of web browsers, file explorers, and some other apps.
- Pushing the center side button presents all windows in KWin. (I have XF86Search configured as a hotkey in KWin.)
- Pressing the Sleep key on my keyboard locks all wallets, locks my session, and turns off my monitor.
- Pressing Shift+Sleep just turns off my monitor.

I've used this configuration (and mouse) for over a decade with great success.
Comment 27 Nate Graham 2020-04-30 19:09:29 UTC
Yep, I use `xbindkeys` too, it would just be nice to have this doable from the UI. :)
Comment 28 Rod J 2020-05-05 06:33:30 UTC
xbindkeys user here too. 

It's saved my sanity as I became perplexed at a missing shortcut that made so much sense to me when I first started using Ubuntu many years ago (Meta Key + scroll mouse wheel to zoom desktop). It was just so natural to me and I couldn't figure out why the absence of it when I switched to Kubuntu about 2012. We're not all keyboard only users KDE devs! KDE Plasma is the best desktop environment I've ever used but little missing things like this are irritating.
Comment 29 Zamundaaa 2020-12-09 15:59:04 UTC
Making it possible to trigger shortcuts with buttons on pens (like for Wacom tablets) could be pretty useful as well.
Comment 30 Alexander Volkov 2022-08-29 10:09:53 UTC
Related project: https://invent.kde.org/plasma-bigscreen/plasma-remotecontrollers
Comment 31 Alexander Volkov 2022-08-31 10:36:29 UTC
Allow rebinding of extra mouse buttons on Wayland: https://invent.kde.org/plasma/plasma-desktop/-/merge_requests/920
Comment 32 Aleix Pol 2022-09-01 13:16:26 UTC
With plasma remotecontrollers and newer button rebinding features in kwin, I believe this issue is addressed.
Comment 33 JimiJames.Bove 2022-09-01 17:20:15 UTC
Is this issue truly resolved? I can't figure out how to satisfy my use case with the options currently provided. I want to be able to bind Ctrl+Alt+Left and Right click to moving and resizing windows, respectively.

I just opened up my system settings and I'm still unable to add mouse buttons to a shortcut trigger--which is what it says on the tin, as far as what this bug report is for.

As far as workarounds like the ones just mentioned, well, one is Wayland-only and I still can't use Wayland for multitudes of reasons, so that's a non-starter. Plus it's only for *extra* mouse buttons, not left and right click, so the whole thing is moot.

As for plasma-remotecontrollers, is it configurable, or does it just serve as a driver for various remotes? According to its description, in order to use it to accomplish what I need, I would most likely have to bind left and right click to intermediary keys that I could then bind to the kwin shortcuts I need. Unless I can use obscure keycodes that aren't physically on my keyboard anyway, that seems like a very roundabout and problematic way to accomplish this.