A Joystick for Your DonkeyCar
At our first AI Racing League event in Minnesota on August 6th 2019 we used ten Nvidia Nano based DonkeyCars. We had many great people teaching students how to train the DonkeyCars how to drive autonomously around the track to build the training sets. However, the process of controlling the car was difficult for many people because the user interface for steering the car was difficult to use. The web-page running on the cell phone web browser didn’t have tilt working on any browsers (even Chrome) and the pressing the web page joystick also proved tricky. So to try to create a better event experience I did some research and found that many other teams also had these problems.
Several other groups have started using an external joystick controller to make it easier to drive the cars. The most common option seemed to be the $40 2.4 GHz Logitech F710 joystick. This device also gets high ratings on Amazon and the UNIX driver support seemed very solid going back four years. So I ordered one and it arrived in two days from Amazon. After a few attempts I finally got it to work. But not without a little suffering. The drivers and the python code still seems a bit buggy (could be my lack of experience) and the software would frequently stop working or crash. However, I did find some very cool ways to isolate problems and diagnose the issues and it is now running reliably most of the time. So here are my debugging notes to help others going down this path.
Logitech F710 JoyStick Controller QuickStart
If you just want to try the basic steps, here they are. Most of the rest of the article will be how to debug the process when things go wrong. Here are the basic steps:
- Order your joystick from somewhere like Amazon. It will come with a USB dongle.
- Plug the USB dongle to a port of your single board computer (Raspberry Pi or Nvidia Nano). I only tested this on my Nano.
- Edit the myconfig.py file and change CONTROLLER_TYPE to be F710
4. Start driving using the standard python drive command with the dash-dash js option (note the two dashes before the “js” joystick option):
The forward-back motion on the right joystick controls the speed and the left-right motion on left joystick controls the steering. If you click back twice on the right joystick you put the car into reverse.
Sounds pretty simple if all the DonkeyCar gods are on your side. Unfortunately for most of us there are sometimes daemons involved and problems occur. The key is to know a few tests to isolate the problems. Once you master these debugging tips you will also be on your way to building customizations to the joystick software.
UNIX Character Device Drivers
The first step is to make sure that the Logitech F710 USB dongle is recognized by the Operating system. I am using a Nvidia Nano running Ubuntu 18 so most of the debugging tips require opening up a UNIX shell while logged on to the single board computer (SBC). In my case the driver for the Logitech F710 was preloaded into the operating system and the dongle was instantly recognized when we pluged it into the USB port. You can see it was detected by running the UNIX “List USB” command called lsusb:
$ lsusb
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 004: ID 0bda:8179 Realtek Semiconductor Corp. RTL8188EUS 802.11n Wireless Network Adapter
Bus 001 Device 005: ID 046d:c21f Logitech, Inc. F710 Wireless Gamepad [XInput Mode]
Bus 001 Device 002: ID 0bda:5411 Realtek Semiconductor Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Note that the USB device with an ID of 046d:c21f has been found in the 4th line above. The first ID before the colon is the device manufacturer (Logitech) and the second is the id of their device (c21f is for the F710 controller). The operating system looks these ID numbers up in their USB driver database and then loads and runes the appropriate device driver for this type of device. Note that the device driver can even do smart things like probe the device to see the Mode of the controller. In this case the lsusb command even verifies that the controller is in the correct default XInput mode (X mode). If you don’t see this then you will need to change the mode setting — more on this later in the blog.
If you don’t seen this line it means that the operating system didn’t recognize the device or your security software prevented the device driver from working.
The driver will create what looks like a device file endpoint in the /dev/input directory called js0. To read and write from the joystick we just do reads and writes to this device. The first time you plug in the USB dongle this file will appear. If you unplug and plug it back in this might be changed to js1.
$ ls -l /dev/input/js0
crw-rw-r — + 1 root input 13, 0 Aug 16 19:30 /dev/input/js0
The first letter “c” says that this is a character I/O device driver that moves data back and forth using single 8-bit characters. This is perfect for a joystick since we only need a few bytes to encode joystick movement. The other type of UNIX device driver is a block driver which is usually for moving large sources of data like to and from a disk drive.
Although this looks like only device file, what happens is that the system actually creates several different device files and links them together. In this case there are other devices that are linked to js0 like /dev/input/event2:
$ ls -l /dev/input/*
crw-rw — — 1 root input 13, 64 Aug 18 08:48 /dev/input/event0
crw-rw — — 1 root input 13, 65 Aug 18 08:48 /dev/input/event1
crw-rw — — + 1 root input 13, 66 Aug 18 08:48 /dev/input/event2
crw-rw-r — + 1 root input 13, 0 Aug 18 08:48 /dev/input/js0
crw-rw — — 1 root input 13, 63 Aug 18 08:48 /dev/input/mice
/dev/input/by-id:
total 0
lrwxrwxrwx 1 root root 9 Aug 18 08:48 usb-Logitech_Wireless_Gamepad_F710_329A9041-event-joystick -> ../event2
lrwxrwxrwx 1 root root 6 Aug 18 08:48 usb-Logitech_Wireless_Gamepad_F710_329A9041-joystick -> ../js0
/dev/input/by-path:
total 0
lrwxrwxrwx 1 root root 9 Aug 18 08:48 platform-70030000.hda-event -> ../event0
lrwxrwxrwx 1 root root 9 Aug 18 08:48 platform-70090000.xusb-usb-0:2.1:1.0-event-joystick -> ../event2
lrwxrwxrwx 1 root root 6 Aug 18 08:48 platform-70090000.xusb-usb-0:2.1:1.0-joystick -> ../js0
lrwxrwxrwx 1 root root 9 Aug 18 08:48 platform-gpio-keys-event -> ../event1
You don’t need to know the details about these links other than the fact that some tools refer to the joystick by the id or by the path.
If you don’t see these files you need to go back and search for your operating system name and version with the string Logitech F710 device driver.
Verifying Device Events with evtest
Now let’s assume that the USB dongle driver is found but your system is still not working. We need to verify that the joystick controller is in fact generating the signals and that the events are arriving correctly. This process is really important to tell if the batteries are charged and the F710 is in the correct XInput mode.
Note that there is a little tiny switch in the front of the F710 — with almost ineligible labels — which MUST set to X (not the D for DirectStream which will not work with the DonkeyCar). It looks a bit like a power switch but it changes the mode of the output format to be compatible with other controller standards.
To watch the stream of incoming controller events there is an extremely handy UNIX shell command call “Event Test” or just evtest for the name of the UNIX command. To install evtest run the following:
$ sudo apt-get install evtest
It is a small file and should download in a few seconds. Once this finished you can run the evtest command directly from the UNIX shell. Remembering that above the joystick was bound to event device 2. We will enter the number “2” when prompted:
$ evtest
No device specified, trying to scan all of /dev/input/event*
Not running as root, no devices may be available.
Available devices:
/dev/input/event2: Logitech Gamepad F710
Select the device event number [0–2]: 2
Logitech Gamepad F710 Input driver version is 1.0.1
Input device ID: bus 0x3 vendor 0x46d product 0xc21f version 0x305
Input device name: “Logitech Gamepad F710”
Supported events:
Event type 0 (EV_SYN)
Testing … (interrupt to exit)
Now as you press any key or move any joystick you should see each event with a slightly different type and code. For the buttons, there will usually one event for a press down and one event for a release. The joysticks will send a stream of events when the are moved away from the neutral position.
When we press the yellow Y button we see:
Event: time 1566006064.962158, type 1 (EV_KEY), code 308 (BTN_WEST), value 1
Event: time 1566006064.962158, — — — — — — — SYN_REPORT — — — — Event: time 1566006065.129981, type 1 (EV_KEY), code 308 (BTN_WEST), value 0
Event: time 1566006065.129981, — — — — — — — SYN_REPORT — — — —
When we press the Blue X
Event: time 1566006110.047015, type 1 (EV_KEY), code 307 (BTN_NORTH), value 1
Event: time 1566006110.047015, — — — — — — — SYN_REPORT — — — —
Event: time 1566006110.182606, type 1 (EV_KEY), code 307 (BTN_NORTH), value 0
Event: time 1566006110.182606, — — — — — — — SYN_REPORT — — — —
When we press the Red B
Event: time 1566006143.423217, type 1 (EV_KEY), code 305 (BTN_EAST), value 1
Event: time 1566006143.423217, — — — — — — — SYN_REPORT — — — — —
Event: time 1566006143.499642, type 1 (EV_KEY), code 305 (BTN_EAST), value 0
Moving the joystick generates the following stream:
Event: time 1566006255.549652, — — — — — — — SYN_REPORT — — —
Event: time 1566006255.553650, type 3 (EV_ABS), code 1 (ABS_Y), value -10923
Event: time 1566006255.553650, — — — — — — — SYN_REPORT — — — — Event: time 1566006255.557650, type 3 (EV_ABS), code 1 (ABS_Y), value -14264
Event: time 1566006255.557650, — — — — — — — SYN_REPORT — — — —
Event: time 1566006255.561652, type 3 (EV_ABS), code 1 (ABS_Y), value -18633
This tool is fantastic for debugging. If you have no events coming in and you may have a problem with the controller. Check the batteries and the X/D mode switch in the front of the controller and make sure it is set to be in the X position. You can also use this to see what the range of the controller is. Mine worked even from across a large room.
After you see the events arriving you know that data is being sent from the joystick controller and processed one at a time by the operating system. Now we need to make sure that the DonkeyCar software can recognize these events and control the steering and throttle.
Tuning the Configuration File
First we need to make a few additional changes in the myconfig.py configuration file. As I mentioned above, the CONTROLLER_TYPE has to be changed. I changed a few of them to allow me to better control the car. Here are the other two settings I have changed that change the scale of the throttle and the steering:
CONTROLLER_TYPE=’F710' #(ps3|ps4|xbox|nimbus|wiiu|F710)USE_JOYSTICK_AS_DEFAULT = True #when starting the manage.py, when True, will not require a — js option to use the joystick
JOYSTICK_MAX_THROTTLE = .8 #this scalar is multiplied with the -1 to 1 throttle value to limit the maximum throttle. This can help if you drop the controller or just don’t need the full speed available.
JOYSTICK_STEERING_SCALE = 1.2 #some people want a steering that is less sensitive. This scalar is multiplied with the steering -1 to 1. It can be negative to reverse dir.
Using Buttons and Adding Custom Functions
I have not had a chance to experiment with controls other than the throttle and steering. There is some documentation on this on the DonkeyCar site but the documentation does not match the buttons on the F710 controller so this will need to be documented on a future blog post.
There are 16 distinct buttons on the F710 that you can program to do various things such as increase or decrease the maximum speed, increase and decrease the scale of the throttle and steering, start and stop recording data and change the mode from user to self-driving modes (angle and throttle). Future versions (I am using the DonkeyCar 3.1 version) could even have hand features that would erase the last five seconds of data if you crash or go off the track. Below you can see an excerpt from the F710 User Manual.
Conclusion
Using the Logitech F710 joystick did give me quite a bit more control when driving the car. I still need to get used to the steering on the left joystick and the throttle on the right joystick. I think with an hour of practice I will have it down. The additional $40 price is a bit of a problem for teachers trying to build low-cost kits for their classrooms. No all students will be training at once so you may not need one for each car but you will need to change the configuration file each time a car uses the joystick. Perhaps a future version of the DonkeyCar software will just look for a joystick controller when it starts up and use it if it finds the controller.
There is also no screen to see the status of the DonkeyCar when driving. There are some controllers that allow you to mount a smartphone above them. The web browser interface does not seem to work with the joystick mode is turned on. A cell phone application with the image streamed to the screen communicating with Bluetooth, using the accelerometers with a tilt to turn and accelerate would be my first choice. Getting feedback on how many images have been captured would be an ideal interface. Perhaps someone within CoderDojo with experience with AppInventor will create an application for us. One other alternative is a $10 phone “clip” that you can attach to the controller.
I should mention a few other things we have not tested. I don’t know how running 10 different F710 controllers in a classroom will work yet. The controllers might interfere with each other and students might mix up the dongles when they put them back in the boxes. I have used 2.4GHz dongles with 20 MBots in classrooms and as long as they are all clearly labeled (device #1, #2 etc.) things worked out well. So I know it is technically possible.
I also know that small parts like dongles get lost easily. Although you can purchase replacements from Logitech, the ideal system would use a built-in wireless interface on the DonkeyCar motherboard.
There are also 264 questions on the Amazon product page about the F710.
References
The DonkeyCar documentation on joystick controllers is here.
The User Guide to the Logitech F710 is here.
The UNIX manual page for evtest is here.
The UNIX manual page for lsusb is here.
