Using the Hover

All Bound Up

To recap, now you know how to get the Hover talking to your Raspberry Pi and how to translate what the Hover says to certain key presses.

The last piece of the puzzle is working out what the LXDE key bindings are and how to make your own. Key bindings are the name for the different key combinations mapped to actions on the LXDE desktop. So, for example, while holding down the Alt key and hitting the F4 key (Alt+F4), you can close the active window (in computer desktop parlance, by the way, the active window is said to have the focus).

Try it: Open an empty Leafpad window and hit Alt+F4. This works because Alt+F4 is a default key combination bound to the close window action in LXDE (and many other desktops).

You can modify your Hover_pi.py program to link the close window action to a Hover down swipe. Open Hover_pi.py with your favorite editor (e.g., IDLE) and change the line that says

device = uinput.Device ([uinput.KEY_P, uinput.KEY_I,])

to

device = uinput.Device ([uinput.KEY_P, \
  uinput.KEY_I, uinput.KEY_LEFTALT, \
  uinput.KEY_F4,])

and change the lines that say

elif (message == "00110000"):
        # Swipe down
        pass

to the lines:

elif (message == "00110000"):
   # Swipe down
   device.emit_combo ([uinput.KEY_LEFTALT, uinput.KEY_F4])

Save the file as Hover_LXDEcontrol.py.

If you're not already in the graphical desktop, start it with startx. Open a terminal, navigate to where you are storing your programs, and run Hover_LXDEcontrol.py with:

$ sudo python Hover_LXDEcontrol.py

Now open a new window, such as the calculator or another empty Leafpad session. Make sure the new window has the focus (click on the title bar of the window if you're not sure). If you wave right and left, and the app you opened has an area in which you can type text, the letters p and i should appear in the window. If you wave down, the window will close. I'm pretty sure that by now you're seeing where this is going.

You know that LXDE has Alt+F4 configured by default, but what other default key bindings does LXDE use? To find out, open the file /home/pi/.config/openbox/lxde-rc.xml in Leafpad, scroll down to the <keyboard> … </keyboard> section, and you can see them. The key binding to close a window is the chunk of code that says:

<keybind key="A-F4">
    <action name="Close"/>
  </keybind>

You'll notice the syntax is different from uinput: Here, A stands for the Alt key, F4 stands for the F4 key, and you join the keys together with a hyphen. The Alt key is called a modifier, by the way. You can see the most common modifiers and how they are abbreviated to create key bindings in LXDE in Table 1.

To combine two or more modifiers together, use hyphens. The Ctrl+Alt+Del combo becomes C-A-Delete, for example.

You know that the F4 is F4 for LXDE, and that the Del key is Delete (because I just told you), but what about the rest of the keys? To find out, you can use the xev program (Figure 7). Run it from a terminal within LXDE and a little square window pops up. Maintain the focus on this little window and press any key. Something like what is shown Listing 5 will appear in the terminal window.

KeyRelease event, serial 48, synthetic NO, window 0x1400001,
    root 0x43, subw 0x0, time 12439770, (-564,523), root:(259,923),
    state 0x0, keycode 111 (keysym 0xff52, Up), same_screen YES,
    XLookupString gives 0 bytes:
    XFilterEvent returns: Fals

Figure 7: The xev app lets you find out the name of the keys you press.

The term in bold is the name LXDE assigns to the key that is pressed. In this case, I pressed the Up arrow key and the name LXDE assigned to it is, unsurprisingly, Up.

Now that you know the keys, you need to know the actions you can bind them to (see the <action name="Close"/> line above). You can find a complete list of LXDE actions online [5]. Next, you're going to create four new key bindings to four actions that later you will associate with each of the swipe directions.

In Table 2, you can see the combinations I chose, along with the actions, uinput equivalents, and the swipes I associated with them.

First, create the new bindings in the lxde-rc.xml file. If you still have it open in Leafpad, add the lines you see in Listing 6 inside the <keyboard> … </keyboard> section.

01   <!-- Hover Key bindings -->
02   <keybind key="A-Up">
03         <action name="ToggleShade"/>
04   </keybind>
05
06   <keybind key="A-Down">
07         <action name="Iconify"/>
08   </keybind>
09
10  <keybind key="A-Left">
11         <action name="MoveToEdge"><direction>west</direction></action>
12   </keybind>
13
14   <keybind key="A-Right">
15         <action name="MoveToEdge"><direction>east</direction></action>
16   </keybind>

Save the file and execute

$ openbox --reconfigure

in a terminal window so LXDE picks up your new key bindings.

Now you can try out your bindings. Click on a window to give it the focus, hold down the Alt key, and press the Up arrow key; the window should fold up like a blind, leaving only its title bar visible. Press the combination again, and the window should unfold. Holding down the Alt key and pressing the Down arrow key should minimize the window to the bottom panel. Finally, hold down the Alt key and press the Left or Right arrow keys, and the window should move to the left or right edge of the screen.

Once everything's working, you can now rewrite your Hover_LXDEcontrol.py script. It should look similar to Listing 7.

01 import time
02 from Hover_library import Hover
03 hover = Hover(address=0x42, ts=23, reset=24)
04
05 import uinput
06 device = uinput.Device ([
07                         uinput.KEY_LEFTALT,
08                         uinput.KEY_UP,
09                         uinput.KEY_DOWN,
10                         uinput.KEY_LEFT,
11                         uinput.KEY_RIGHT,
12                        ])
13
14 try:
15   while True:
16
17     if (hover.getStatus() == 0):
18       message = hover.getEvent()
19       type(message)
20       if (message == "01000010"):
21         # West tap
22         pass
23       elif (message == "01010000"):
24         # Center tap
25         pass
26       elif (message == "01001000"):
27         # East tap
28         pass
29       elif (message == "01000001"):
30         # South tap
31         pass
32       elif (message == "01000100"):
33         # North tap
34         pass
35       elif (message == "00100010"):
36         # Swipe right
37         device.emit_combo ([uinput.KEY_LEFTALT, uinput.KEY_RIGHT])
38       elif (message == "00100100"):
39         # Swipe left
40         device.emit_combo ([uinput.KEY_LEFTALT, uinput.KEY_LEFT])
41       elif (message == "00110000"):
42         # Swipe down
43         device.emit_combo ([uinput.KEY_LEFTALT, uinput.KEY_DOWN])
44       elif (message == "00101000"):
45         # Swipe up
46         device.emit_combo ([uinput.KEY_LEFTALT, uinput.KEY_UP])
47
48       hover.setRelease()
49     time.sleep(0.0008)   #sleep for 1ms
50
51 except KeyboardInterrupt:
52   print "Exiting..."
53   hover.end()
54
55 except:
56   print "Something has gone wrong..."
57   hover.end()

Now try it! Run your script with

$ sudo python Hover_LXDEcontrol.py

or, if you prefer to run the script in the background as a daemon, run

$ sudo python Hover_LXDEcontrol.py&

and enjoy.

Conclusion

In this article you learned how to use the Raspberry Pi with the Hover, a cheap (US$  39) hardware add-on that can detect gestures and taps, and Python-uinput to convert input from the Hover (or any GPIO-based hardware) into keystrokes then use those keystrokes to take actions on your LXDE desktop. For all practical purposes, you have a cool system to control your desktop with a wave of your hand.

This setup might not have the fancy looks of the interface Tom Cruise used in Minority Report, but for less than 100 bucks, I dare say it is much cheaper – and, you don't need that clunky glove either.