From MAE/ECE 148 - Introduction to Autonomous Vehicles
Jump to navigation Jump to search

Team Members

  • Ihyun Leo Park - Electrical Chad Engineering
  • Brandon Espinoza Balbuena - MAE
  • Ashenling Yang's Braised Chicken Rice - MAE(Anagram for LAME, L's just a bonus)

Project Overview

Our project used tesseract and EAST text detection with openCV to detect text, then which the car will go and pick up the specified object using a model trained with the YOLO algorithm. For example, if we show the car the word apple, it will go and pick up an apple. We used openCV and imutils library to control the camera, pyVESC to control the vesc and the motor, and finally the python servo motor library to control the servo motor to pick up the actual item. We used a brushless DC motor, VESC, Jetson Nano, 1080p webcam, and a servo motor.

Main Components

Physical Setup



Car 3.jpg

Final Product

Car 2.jpg


Team3 circ diag.png


Software and Hardware List

The following are needed to complete this project.

  • Jetson Nano
  • 64GB MicroSD card
  • Camera
  • Tesseract
  • pyTorch
  • seaborn
  • imutils
  • argparse
  • OpenCV
  • Probably a lot more I'm forgetting

Source code

The source code of the project can be found here. Good luck running it :): https://github.com/SikGek/ECE_148_Final

Steps Overview

  • Jetson Nano setup
  • Clone git repo: https://github.com/SikGek/ECE_148_Fina
  • Download all the libraries that it requires you to do so, you are probably missing at least 3 to 5
  • Show the car a text from the list of detectable objects
  • Let the car spin around until it finds the shown item and watch it as it goes and pick it up.


Our team was able to successfully have it retrieve 3 different types of items, although 2 is shown for saving time. The throttle is slow to compensate for the delay in camera feed due to poor processing power of the Jetson Nano.

Tesseract is painfully slow due to the model being trained for all the characters in the alphabet, while we reduced the YOLO model to the smallest size for best possible performance.

Gantt chart

Gantt chart3.jpeg


Entire Project at Work on an Apple and a Banana

The Camera feed to Showcase that it only detects one object


Challenge Solution
Inconsistent object detection Higher resolution camera improved results
Little time to develop in ROS Used python libraries instead
Jetson doesn't have enough power to power the servo motor Use i2c power switch
Poor and slow object detection Training our specific model

Potential Improvement

  • Use ROS for VESC, camera and servo control
  • Obtain a better camera
  • Change the camera angle so that it can detect a basket after picking up the object and place it inside of it
  • Train a new model with more detectable objects
  • Implement speech detection for input instead of text detection for convenience

Weekly Presentations

Final Project Proposal
Progress Week 8
Progress Week 9
Final Presentation


We would like to give special thanks to:

  • Professor Jack Silberman
  • TA Dominic Nightingale
  • TA Ivan Ferrier
  • ECE Makerspace
  • All my fantastic classmates


I kinda copied team 7's format, hope that's ok. Their project was very sick, go press a like and subscribe to them. Thank you. ^^7