- Ihyun Leo Park - Electrical Chad Engineering
- Brandon Espinoza Balbuena - MAE
- Ashenling Yang's Braised Chicken Rice - MAE(Anagram for LAME, L's just a bonus)
Our project used tesseract and EAST text detection with openCV to detect text, then which the car will go and pick up the specified object using a model trained with the YOLO algorithm. For example, if we show the car the word apple, it will go and pick up an apple. We used openCV and imutils library to control the camera, pyVESC to control the vesc and the motor, and finally the python servo motor library to control the servo motor to pick up the actual item. We used a brushless DC motor, VESC, Jetson Nano, 1080p webcam, and a servo motor.
Software and Hardware List
The following are needed to complete this project.
- Jetson Nano
- 64GB MicroSD card
- Probably a lot more I'm forgetting
The source code of the project can be found here. Good luck running it :): https://github.com/SikGek/ECE_148_Final
- Jetson Nano setup
- Clone git repo: https://github.com/SikGek/ECE_148_Fina
- Download all the libraries that it requires you to do so, you are probably missing at least 3 to 5
- Show the car a text from the list of detectable objects
- Let the car spin around until it finds the shown item and watch it as it goes and pick it up.
Our team was able to successfully have it retrieve 3 different types of items, although 2 is shown for saving time. The throttle is slow to compensate for the delay in camera feed due to poor processing power of the Jetson Nano.
Tesseract is painfully slow due to the model being trained for all the characters in the alphabet, while we reduced the YOLO model to the smallest size for best possible performance.
Entire Project at Work on an Apple and a Banana
The Camera feed to Showcase that it only detects one object
|Inconsistent object detection||Higher resolution camera improved results|
|Little time to develop in ROS||Used python libraries instead|
|Jetson doesn't have enough power to power the servo motor||Use i2c power switch|
|Poor and slow object detection||Training our specific model|
- Use ROS for VESC, camera and servo control
- Obtain a better camera
- Change the camera angle so that it can detect a basket after picking up the object and place it inside of it
- Train a new model with more detectable objects
- Implement speech detection for input instead of text detection for convenience
We would like to give special thanks to:
- Professor Jack Silberman
- TA Dominic Nightingale
- TA Ivan Ferrier
- ECE Makerspace
- All my fantastic classmates
I kinda copied team 7's format, hope that's ok. Their project was very sick, go press a like and subscribe to them. Thank you. ^^7