2020FallTeam1

Team Members

Electrical Engineers

Benjamin Crawford

Heather Huntley

Joshua Orozco

Mechanical Engineers

Peggy Tran

Project Overview

Initially the goal of our project was to design an autonomous vehicle that would utilize DonkeyAI framework, open CV, and sensors to detect and gather tennis balls in one area to facilitate collecting them after each set. Due to COVID-19, the scope of our project was pivoted to create a ROS package that enables the RoboCar to use images taken through its external camera to be processed through a mask and proportional-integral-derivative tuning in order to read the yellow lines on the external track and drive autonomously; the RoboCar uses a webcam to read the yellow lines and compute a centroidal value that it follows.

Hardware

Mechanical Design

Camera Mount

Base Plate

This base plate was printed on .25 inch acrylic and was used to hold all electrical components of the RoboCar.

Electronic Components

Jetson Nano Developer Kit

Adafruit PCA9685 16bit PWM controller

USB webcam

High power LEDs

11.1V 3S LiPo battery

Battery voltage sensor

433MHz remote relay

12V-5V DC-DC voltage converter

The car our team used is built around a Traxxas Ford Fiesta chassis containing a DC motor, electronic speed controller and servo motor. The following diagram shows how all of the electronics are wired.

Electronic Wiring Schematic

Software

We set up a GitHub repository where the code used by the vehicle is stored. The repository contains an ROS package which is designed to be used with our car. The following diagram displays the overall structure of the ROS package:

The overall gist of the package is that two commands are used to achieve the goal of the program. When inputting the command roslaunch autocar autocar_pub.launch the publisher node pub is created. This command also begins running the publisher.py which sends the data from the USB webcam into an openCV program and detects lanes and publishes a value to the /steering and /throttle topic. Then to start the subscriber simply write the command roslaunch autocar autocar_sub.launch after this the car should begin driving autonomously.

Topics

Topic Name	Topic Type	Contents
Steering	Float64	Steering Value in range from -1 to 1
Throttle	Float64	Throttle Value in range from -1 to 1

Publisher

The publisher is a modified and upgraded version of the simple_cv_racer that was provided to us. It takes an image from the USB webcam and uses OpenCV to perform several color masks in a thin region of the photo and converts this into steering and throttle values using a PID controller. It then publishes these values to their respective topics. We created a mask for the yellow of the center line, one for the orange of the turn lanes, and finally one for the white of the outer lanes. These masks were denoised using a kernel of ones that way any small patches of color would be eliminated and would not affect the steering of our car. We then find the largest area of pixels in each mask and find the x coordinate of the center of that area. We then calculate a histogram of the amplitudes for each mask and select the position in the histogram with the highest amplitude as another x coordinate. These two x coordinates are averaged and we then look to see what we got. If there was no orange line detected which indicates that the car is on a straight the car will attempt to steer to the yellow position, it will also lock its wheels so that there will be less deviation from the straight line. If it does detect an orange line indicating that it is on a turn it will drive to a weighted average of the two x coordinates and will unlock its wheels so it has its full range of motion. Unfortunately we are not doing anything with the white data although some ideas we had for it are discussed in the future work section.

Subscriber

The subscriber simply subscribes to the two topics and then leverages the Adafruit Servokit library to communicate with the Adafruit PCA9685 and writes the values to the channels specified in the config object.

Milestones

Donkey AI training and AI driving

Using the donkeycar framework and the UCSD GPU cluster to speed up the behavioral modeling process we were able to do five autonomous laps on an outdoor track. As you can see in the video below:
5 autonomous laps using DonkeyAI framework

ROS Architecture

The following link is a video of our team troubleshooting the PID values which are used to control the throttle and steering for the car.

Troubleshooting ROS framework

Calibration and line following using ROS package

With the ROS package that we developed we completed five laps (with a little help) on the track in the Warren tents. Here is a video of that:

Autonomous ROS Laps

Potential Future Work / Unaccomplished Goals

White Lines

We were initially planning on using the white lines of the track to guide us or work as barriers that the car absolutely would not cross. Unfortunately we did not have time to implement this in a meaningful way. Given enough time we would have like to been able to detect both the left and right lines and distinguish between them in order to inform our decision making process.

Hough Transforms

This is something we saw other groups implementing and while we thought it was very cool we opted to stick with centroid method in order to cut down on complexity and time spent debugging. However it is something that given enough time we would have been very excited to try out.

Suggestions For Future Teams

General Suggestions

For software focused members, it's important to brush up on any python skills during the former half of the quarter to prepare for the latter half when you are coding your own RoboCar. For hardware focused members, they should brush up on relevant tolerance standards in order to minimize the amount of material they use when designing the baseplate and cam mount. For all members, they should retain as much as they can on ROS ignite and collaborate on the topics/concepts in order to consolidate their knowledge.

Troubleshooting with lighting and colors detected

Our advice for future teams is to account for the changes in lighting that the car will experience. This may include accounting for the way the time of day, or shadows can change the shade of yellow of the line. We solved this issue by tightening the range of color that the car will detect and follow. By the end of troubleshooting we also used the color sensitivity of the robot to our advantage. We wrote functions in the code that would change the way the car responds at different points in the track. For example, at each of the four turning corners in the track, the track is lined with orange tape. When our car encounters orange tape, it will respond by having a higher range of mobility when turning. The color orange detected helped the car stay straight when going down the long sides of the track while also allowing the car to turn tightly enough to stay on the track.

Troubleshooting with stationary car

Another piece of advice for our future teams is to troubleshoot the car’s reaction to yellow lines by holding it above the track. By holding the car directly above the track, you can take the tension off the wheels of the car, and get a more accurate representation of the reaction the car has to the track. When we first started to troubleshoot the steering of the car by holding the car to the left, right, and center of the track. We were able to see that we flipped the steering direction of the car. When the car was held to the left side of the track the wheels veered left, taking the car away from the yellow line.

Acknowledgements

We would like to thank Professors Jack Silberman and Mauricio de Oliveira for their help throughout the quarter and consistent work in keeping the course engaging and relevant throughout the difficulties imposed by the COVID pandemic. We would also like to recognize the hard work put in by the TA's Haoru and Leeor.

References

Donkey Car
simple_cv_racer
Adafruit Library