2022SpringTeam3
Team Members
- Ihyun Leo Park - Electrical Chad Engineering
- Brandon Espinoza Balbuena - MAE
- Ashenling Yang's Braised Chicken Rice - MAE(Anagram for LAME, L's just a bonus)
Project Overview
Our project used tesseract and EAST text detection with openCV to detect text, then which the car will go and pick up the specified object using a model trained with the YOLO algorithm. For example, if we show the car the word apple, it will go and pick up an apple. We used openCV and imutils library to control the camera, pyVESC to control the vesc and the motor, and finally the python servo motor library to control the servo motor to pick up the actual item. We used a brushless DC motor, VESC, Jetson Nano, 1080p webcam, and a servo motor.
Main Components
Physical Setup
Car
Progress
Final Product
Schematic
Implementation
Software and Hardware List
The following are needed to complete this project.
- Jetson Nano
- 64GB MicroSD card
- Camera
- Tesseract
- pyTorch
- seaborn
- imutils
- argparse
- OpenCV
- Probably a lot more I'm forgetting
Source code
The source code of the project can be found here. Good luck running it :): https://github.com/SikGek/ECE_148_Final
Steps Overview
- Jetson Nano setup
- Clone git repo: https://github.com/SikGek/ECE_148_Fina
- Download all the libraries that it requires you to do so, you are probably missing at least 3 to 5
- Show the car a text from the list of detectable objects
- Let the car spin around until it finds the shown item and watch it as it goes and pick it up.
Results
Our team was able to successfully have it retrieve 3 different types of items, although 2 is shown for saving time. The throttle is slow to compensate for the delay in camera feed due to poor processing power of the Jetson Nano.
Tesseract is painfully slow due to the model being trained for all the characters in the alphabet, while we reduced the YOLO model to the smallest size for best possible performance.
Gantt chart
Demonstrations
Entire Project at Work on an Apple and a Banana
The Camera feed to Showcase that it only detects one object
Challenges
| Challenge | Solution |
|---|---|
| Inconsistent object detection | Higher resolution camera improved results |
| Little time to develop in ROS | Used python libraries instead |
| Jetson doesn't have enough power to power the servo motor | Use i2c power switch |
| Poor and slow object detection | Training our specific model |
Potential Improvement
- Use ROS for VESC, camera and servo control
- Obtain a better camera
- Change the camera angle so that it can detect a basket after picking up the object and place it inside of it
- Train a new model with more detectable objects
- Implement speech detection for input instead of text detection for convenience
Weekly Presentations
Final Project Proposal
Progress Week 8
Progress Week 9
Final Presentation
Reference
We would like to give special thanks to:
- Professor Jack Silberman
- TA Dominic Nightingale
- TA Ivan Ferrier
- ECE Makerspace
- All my fantastic classmates
Disclaimer
I kinda copied team 7's format, hope that's ok. Their project was very sick, go press a like and subscribe to them. Thank you. ^^7