Difference between revisions of "2021FallTeam1"
| Line 40: | Line 40: | ||
Our final project aimed to use image detection that would autonomously cue the robot to play a song while it is going around the track. These images will be placed on the side of the track and will be detected by another webcam on the robot that is facing outward with respect to the track. | Our final project aimed to use image detection that would autonomously cue the robot to play a song while it is going around the track. These images will be placed on the side of the track and will be detected by another webcam on the robot that is facing outward with respect to the track. | ||
We were able to do this through a webcam training data collection script using transfer learning from ResNET18. This script allowed us to take images of each class of the musicians faces around the track in order to train a model that would be able to recognize if there was a musician face on the side of the track, and which musician it was. We took roughly around 1200 images around the track for 5 classes: Beethoven, Frank Sinatra, Freddie Mercury, John Lennon, and a none class for when there was no musician. This ResNET18 model was then used in a script that would then output which class the webcam detected onto a text file from a Pytorch prediction pipeline. This text file is constantly getting read by another script which plays a song of the artist outputted on the notepad. The none class keeps playing the song from the artist before but when another artist shows up, their song starts, ending the current song. This script uses a background Linux process to shuffle a song of the artist. | We were able to do this through a webcam training data collection script using transfer learning from ResNET18. This script allowed us to take images of each class of the musicians faces around the track in order to train a model that would be able to recognize if there was a musician face on the side of the track, and which musician it was. We took roughly around 1200 images around the track for 5 classes: Beethoven, Frank Sinatra, Freddie Mercury, John Lennon, and a none class for when there was no musician. This ResNET18 model was then used in a script that would then output which class the webcam detected onto a text file from a Pytorch prediction pipeline. This prediction pipeline only outputs to the text file when the prediction is above a threshold value of 0.95. This text file is constantly getting read by another script which plays a song of the artist outputted on the notepad. The none class keeps playing the song from the artist before but when another artist shows up, their song starts, ending the current song. This script uses a background Linux process to shuffle a song of the artist. | ||
Our original idea was to have the camera pointing directly perpendicular to the track to view the images on the side of the track but realized having the camera at an angle, we chose it to be 45 degrees, was more ideal as it gave the camera more time to detect the image. Additionally, with the camera being perpendicular to the track, the image would be very blurry and would make the image hard to recognize. Putting the camera 45 degrees from the front solved this issue. | Our original idea was to have the camera pointing directly perpendicular to the track to view the images on the side of the track but realized having the camera at an angle, we chose it to be 45 degrees, was more ideal as it gave the camera more time to detect the image. Additionally, with the camera being perpendicular to the track, the image would be very blurry and would make the image hard to recognize. Putting the camera 45 degrees from the front solved this issue. | ||
Revision as of 22:50, 10 December 2021
Team Members
Jacob Ayers - Electrical Engineering (Year)
Ian Singson - Aerospace Engineering (Senior)
Shoh Mollenkamp - Computer Engineering (Year)
Our Robot
Base Plate
The base plate has a grid of m3 holes that are 1.5 cm apart horizontally and 2.5 cm apart vertically. There is an extended hole in the bottom right of the base plate to accommodate for mounting to the vehicle and attaching the Jetson Nano at an angle to allow for more room for the wiring.
Camera Mount
The camera mount was designed with two legs, a right and left shown in the images below respectively. The legs can each be attached to the base plate via two m3 screws. The camera plate is what holds the camera via four m2 screws and attaches to the two legs by a m3 screw in the back of the plate that goes across both legs on their top most hole. The camera plate was designed to be flush against the two legs and not allow any rotation so that only one screw may be used. We found that the camera did not need to be put at an angle because it was high enough so that the camera could see in what was directly in front of the robot but also far away. The lidar plate can be attached to the camera legs via two m3 screws in the middle of the legs. The lidar plate uses two screws to support the weight of the lidar.
Wiring
Donkeycar Autonomous Laps
{{#evu:https://www.youtube.com/watch?v=qh8faK3-3PE?rel=0 |center| }} {{#evu:https://www.youtube.com/watch?v=8UUWC02IGxU?rel=0 |center| }}
ROS2 Autonomous Laps
{{#evu:https://www.youtube.com/watch?v=EyunxlJ1tvg?rel=0 |center| }}
Final Project
Our final project aimed to use image detection that would autonomously cue the robot to play a song while it is going around the track. These images will be placed on the side of the track and will be detected by another webcam on the robot that is facing outward with respect to the track.
We were able to do this through a webcam training data collection script using transfer learning from ResNET18. This script allowed us to take images of each class of the musicians faces around the track in order to train a model that would be able to recognize if there was a musician face on the side of the track, and which musician it was. We took roughly around 1200 images around the track for 5 classes: Beethoven, Frank Sinatra, Freddie Mercury, John Lennon, and a none class for when there was no musician. This ResNET18 model was then used in a script that would then output which class the webcam detected onto a text file from a Pytorch prediction pipeline. This prediction pipeline only outputs to the text file when the prediction is above a threshold value of 0.95. This text file is constantly getting read by another script which plays a song of the artist outputted on the notepad. The none class keeps playing the song from the artist before but when another artist shows up, their song starts, ending the current song. This script uses a background Linux process to shuffle a song of the artist.
Our original idea was to have the camera pointing directly perpendicular to the track to view the images on the side of the track but realized having the camera at an angle, we chose it to be 45 degrees, was more ideal as it gave the camera more time to detect the image. Additionally, with the camera being perpendicular to the track, the image would be very blurry and would make the image hard to recognize. Putting the camera 45 degrees from the front solved this issue.
Second Camera Holder
Our second camera is webcam that is meant to wrap around the monitor so a different design was needed. This design allows the camera to be securely wrapped around the camera holder. The camera had some degrees of rotation already so rotation did not have to be a design factor.
Source Code
Demonstrations
Image Recognition Demo {{#evu:https://www.youtube.com/watch?v=1AGjGcf40Tk?rel=0 |center| }}
ROS2 Linerunner + Image Recognition Demo {{#evu:https://www.youtube.com/watch?v=ApREtvkTF_c?rel=0 |center| }}
Acknowledgements
Thank you to Dom and Harou for being great TA's and huge thanks to Jack for setting up this class and making this possible.