Team 6: DKar
- Aksharan Saravanan, ECE
- Hieu Luu, DSC
- Katada Siraj, MAE
A robot that acts as an aimbot, in that it follows a target and dynamically adjusts a laser pointing at that target. The robot should use OpenCV to detect the target and ROS to guide steering and throttle based on the target position and distance.
- Have the car be able to autonomously follow a target (e.g. a poster board)
- Modify the car to fit a laser pointer that aims at the target
- (Nice-to-have)Recognize targets of different colors and adjust the following distance.
- Alternative: Use depth data from intel to implement throttle control
- Velcroed the PWM board and DC-DC converter to the bottom of the base, so there would be sufficient space for the camera/servo/laser mechanism
- Thingiverse original cad design for jetson nano case: https://www.thingiverse.com/thing:3532828
- modified this design to allow wires through the back and a more secure fit for the Jetson
- Webcam Camera mounted on top provided a nice FOV without taking up any more space
- Relay is behind the Jetson
~~images (for cad models and lasercut board)~~ (and maybe short description?)
- Intel RealSense RGBD Camera
- Mini Laser
- Micro Servo
Configured the Jetson Nano and compiled OpenCV from source, using the class guide instructions.
Installed the DonkeyCar AI framework on the Jetson, which collected data during manual driving with the joystick. This allowed us to train a behavioral cloning model using that data and then run the robot autonomously based on that model.
3 Laps Video
- If the PWM doesn't seem to be working (the sudo i2cdetect doesn't print out 40/70), it may be that the Bus ports on the Jetson are fried, so can use Bus 1 (instead of Bus 0), but also have to change some lines in config.py
Setup Dominic's docker container on the Jetson, allowing us to run the ROS and calibrate the robot for image detection for the track. Then, launched Dominic's nodes, which autonomously guided the robot through the laps based on our calibrations of error, HSV for yellow, and throttle.
3 Laps Video
- our throttle was changing unexpectedly for the different error modes, so as a last resort, we set the throttle to be a constant value in adafruit_twist_node.py
- calibrate the lane detection just before starting, as different times of the day will have different lighting conditions
- make sure to understand Dominic's instructions on ROS as well as the nodes' code as it will make it much easier when starting on the final project
- the virtual machine makes it easy to work with X11 forwarding
some more explanation
How We Did It
- Image Processing using OpenCV to Detect a Colored Target
- Used Virtual Machine to run a simple python script that detects the largest rectangle which is the color blue, using masking, bound the rectangle, and then detect the line from the point at the center of the object to the point at the actual center of the frame
- Gathered Depth data using an RGBD camera
- Create a custom Target Detection node which Subscribes to Camera Topics and Publishes to the Twist and Servo Topics.
- Node was a python script using ROS2, and utilized Dominic's existing custom ROS2 metapackage to publish/subscribe to.
- Wrote a program which publishes commands to the appropriate nodes based on the gathered data.
- Implemented P controller for throttle which maintains a certain distance from the target
- Implemented a P controller for servo movement
- Integrated code into a ROS package (aimbot_pkg) within the ucsd_robocar_hub2 docker metapackage.
ROS Software Design
Github Link: Project Code
Custom ROS Node Link: Target Detection ROS2 Node
- Determining how to integrate custom ROS code within the Docker Container and getting the Nodes to communicate
- Integrating the Intel RealSense Camera
- Issue of possible “voltage spike” to the ESC causing full throttle when published throttle was suddenly switched from neutral to forward.
- Connecting ideas of OpenCV image detection and doing data processing to publish to ROS topics
- Components including the Jetson, PWM, and switch burned out so had to get new parts
Send reverse throttle controls.
- This would enable our car to be more robust in maintaining a following distance because it would be able to correct itself when overshooting.
Recognize different colors and dynamically adjust following distance.
- This would allow the car to be more responsive to the environment and more robust when following a target.
Thanks to: Professor Silberman, Dominic, Ivan, Professor De Oliveira, as well as the other teams.