2018SpringTeam4

The goal of our project was to train the robocar to recognize common traffic signs, such as a stop sign and speed limits.

---

Team Members

Christian Gunther

David Moll

Alexander Ng

Ruiqi Wang

Hardware

Brushed DC Motor

Brushless DC motors, like the one initially provided on the chassis of the car, have high output power. Although sufficient for most uses, we found that the motor was too sensitive and could not be precisely controlled at lower speeds. We wanted more control, so we replaced it with a brushed DC motor provided to us by the professors.

Shaft Adapter

There were a couple issues with replacing the motor. First off, the brushless DC motor has a 5mm shaft while the brushed DC motor has a 3mm shaft. To fix this, we needed a shaft adapter that would effectively fill the gap between the gear and the new motor shaft. Looking online, we found a model of a 3mm to 5mm shaft adapter on Thingiverse. It is unlikely that the adapter can be used immediately after printing, due to the 3D printers being imprecise. We needed to sand down the outside and drill the inside for the adapter to fit on both the gear and the shaft. We used the model, adapter_hole_flat.stl, but ended up drilling away the flat portion to fit the shaft. The adapter_hole.stl model may be the better choice because the only thing required is the hole on the adapter; the screw on the gear should be enough to secure it onto the d-shaft of the brushed motor.

Overheating

The brushed DC motor provided is likely not designed to operate on the 1/8 chassis. When driving at full speed over uneven ground, such as grass, the motor will begin to overheat after a couple minutes. Our project did not require us to run the motor at high speeds for an extended period of time, but we believe it is possible to burn out the motor. It should be closely monitored; if a project requires the motor to be running for a while, it may be best to continue using the brushless DC motor.

New Electronic Speed Controller

We needed a new electronic speed controller for the bushed DC motor. Unlike the ESC provided on the chassis, this ESC only uses a single 2-cell battery. One observation we made is that the chassis fits the motor on "backwards", meaning we needed to flip the positive and negative wires for the robocar to move in the expected direction.

• Red wire on ESC → Black wire on motor
• Black wire on ESC → Red wire on motor

This configuration will simply reverse the direction the motor will spin.

Second Camera Setup

We decided to use a second camera to detect road signs separately from the camera used to drive the robocar.

This camera's angle and position did not need the same flexibility as the first camera, so we modeled and printed a rigid structure with a slight downward angle. We gave it the downwards angle because our signs were placed very low on the ground. The height of signs will determine the needed angle for the mount (or vice versa).

Second Raspberry Pi set up

For the second raspberry pi, we installed a number of python packages to enable the detection code. Most notable were opencv, scikit-image, and scipy. These took a long time to pip install, but some were able to be installed using "apt-get install", which was quite a bit faster. We were trying to use tesseract-ocr to identify speed limit digits towards the end of the project, but had trouble installing tesseract-ocr in a reasonable amount of time (<3 hours), so did not move forward.

Overall, the set up was a bit easier, as we did not need many donkey car related steps that were necessary for the first pi. Once the packages were installed and the serial code was written, it was simply a matter of connecting their serial ports and starting the programs on each of them.

Communication between Pis

We used the pyserial package to read and write messages between the pis. One pi was strictly in charge of detection, and the other was strictly in charge of driving, so the connection was needed for the detection pi to tell the driving pi to stop.

Detection Code

In order to detect stop signs, we used a Haar Cascade. This had the dual benefits of not needing to be trained (since stop signs are a commonly classified object) and being fast enough to stop a car in time if it saw one. We were able to quickly find a pretrained XML file, and created a multithreaded program to do the following:

1) Take pictures in a stream

2) Check each picture for the presence of a stop sign.

Below, I break down the code into its component elements. We found that breaking up functionality in the following way made it easier to maintain and change as parts were added.

We initialized the models and serial port outside of the detection loop to reduce the overhead of loading the Haar cascade XML training file.

Inside this code, opencv2 is called to detect objects using the training data loading in the previous pictures. The Haar filter method returns a bounding box which could be used to check the size/closeness of the object, but we did not find it necessary to use that. Instead, we simply tested for whether the list of possible stop sign coordinates was empty or not. This was admittedly slightly less robust. Inside the detection method, we included a line for sending a string to the serial port between the pis. This message is the signal for the pi to alter its behavior.

The main loop. Here we set several variables for the camera, including the resolution, which turned out to be important for performance. A lower resolution meant that the model could keep up with the pictures without too much latency.

Throttle Control Code

In order to allow our car to drive autonomously around the track as well as react to any signs that it might see, we integrated our code into the DonkeyCar framework. By following the flow of control in the code, we isolated Controller.py on the path DonkeyCar/DonkeyCar/Parts as being the optimal place to integrate our code. Inside of Controller.py, we instantiate an instance of PS3JoystickController that is a child class of the base JoystickController. The update() function gets called repeatedly, which allows the car to determine whether a button has been pressed and to act accordingly. By reading in from the serial input in this function, we simulated button presses on a controller.

We changed update() to have two components running simultaneously using multithreading. update() now contains update_controller and update_serial.

update_serial just reads from the serial input, and if it reads anything besides an empty string, it adds that into a queue that update_controller has access to. It also stops reading additional serial inputs for 5 seconds after it detects a sign in order to allow the car to get past the sign and not be stuck at a stop sign.

update_controller has all of the old functionality of update(), with the addition of a queue that it checks every loop. If the queue is not empty, that means that a serial input was read. update_controller() then checks what the value in the queue is, and acts based off of what sign was detected, whether a stop sign or a speed limit sign.

If a stop sign is detected, detect_stop_sign is called. This function stops the car by initializing the throttle to its negative value in order to run the motor in reverse to stop faster. It then stops for 3 seconds, then resumes driving at the initial throttle value.

If a speed sign is detected, detect_speed_sign is called with the old speed and the new speed. Unfortunately, we weren't able to detect what number the speed signs had on them quickly enough for driving in real time, so we implemented a simplified version. If a speed sign is detected, it alternates between throttling up and throttling down. We utilized the preexisting increase_max_throttle() and decrease_max_throttle() to integrate into the existing framework better.