- Harou Xue - Electrical Engineering
- Yuhan Zhang - Electrical Engineering
- Cheyenne Herrera - Math/Engineering
The goal of our project is to create a miniature version of a Tesla. We wanted to increase the safety of the self-driving car by implementing rear-end collision prevention as well as apply the lane change safety. The Donkey RoboCar will stop itself when approaching an object in the front using a TOF sensor mounted to it. Additionally, the car will speed up if a vehicle/object is approaching it from behind. Furthermore, the RoboCar will implement lane change on command.
- Jetson Nano with fan and wirless card installed
- PCA9685 PWM (control servo and ESC)
- Steering Servo (control steering)
- Electronic speed controller (ESC) (control throttle)
- Relay (provide emergency stop)
- LED (show emergency stop status)
- USB camera
- Arduino (for connecting ToF sensor)
- Time-of-flight sensor (ToF)
- Lidar and USB controller
Donkey and parts
We use the Donkey Car framework for car control. With the framework, we can easily train deep learning autonomous driving models by recording manual driving. The frameworks use modularized "parts" to manage all the components in a car. When the car runs, it loops through all parts that have been added to it. Not only sensors can be Donkey parts, controllers and actuators are also Donkey parts.
Our project involves new sensors, ToF and Lidar, that have not been included in Donkey. They should be added to Donkey in the form of parts.
We connect the Lidar and ToF(Arduino) using serial ports. We need to add our user to a group that has access to serial ports.
sudo usermod -a -G dialout jetson
To use the PyLidar3 library
Create an instance and connect
lidar = YdLidarX4("/dev/ttyUSB0") lidar.Connect()
scans = self.lidar.StartScanning() for scan in scans: for i in range(360): self.scan[i] = scan[i]
Stop Lidar and disconnect
We use Adruino to connect to the ToF sensor.
myArduino = serial.Serial('/dev/ttyACM0', 115200)
Get sensor reads
myArduino.flushInput() ser_bytes = myArduino.readline() try: distance = float(ser_bytes[0:len(ser_bytes) - 2].decode("utf-8"))
Donkey uses a dictionary to save data that flow through each part. The data used as input or output of parts should be defined when adding parts.
Example: adding Lidar to Vehicle in manage.py
V.add(lidar, outputs=['lidar/scan', 'lidar/time'], threaded=True)
Lane changing behavior training
Donkey framework supports training different behavior states. We can use that to train a model that is able to perform lane changing. https://docs.donkeycar.com/guide/train_autopilot/#training-behavior-models
Rear collision prevention
Our first goal is to provide rear collision prevention. When another car is going to crash into the back of our car and there is no obstacle in front of our car. We can accelerate forward to prevent a collision. To use Lidar to detect objects in the back. The car is 250 millimeters in width so if we want to detect something that wide 200 millimeters away from the back, we need at least about 60 degrees of Lidar measurement.
We set a detection range of 60 degrees, 150 to 210, and from 200 millimeters to 2000 millimeters. We use cosine to calculate the real distance from lidar readings which include angles and distances. Then the median of all 60 distances is selected to be the distance between our car and the object in the back.
First, we just set a threshold, when the distance is lower than 200 millimeters we add throttle by 0.1. It works as intended preventing rear collision but there are some lags and it cannot go faster if the object in the back does not stop.
To mitigate the problem above, we look further and act earlier. Relative speed is calculated using two adjacent distance measure and time between. If the speed of the car is lower than the speed of back object in the same direction, we will add a value proportional to relative speed and inverse proportional to distance. If the distance goes inside 200 millimeters the car will run at its full speed.
Here is part of the code that implements the system described above.
# when lidar is not updated, keep last change if time == self.lasttime: return self.gett(throttle) distance = self.get_d(scan) if distance == 0 or distance > self.maxdis: # out of range is often 0 self.lasttime = time self.lastdis = distance return self.gett(throttle) if distance <= self.mindis self.lasttime = time self.lastdis = distance return 1 if self.lastdis != 0: rspeed = (distance - self.lastdis) / (time - self.lasttime) if rspeed >= 0: self.lasttime = time self.lastdis = distance return self.gett(throttle) self.throttle_change = rspeed/(self.mindis - distance)*50 self.lasttime = time self.lastdis = distance return self.gett(throttle)
gett() here is adding the change to throttle and capped by 1.
Safe lane changing
When changing lane, it is not safe to just look at mirrors. Using Lidar we can detect cars in the blind spot and override steering control to prevent unsafe lane changing.
More specifically, use the yellow regions of the Lidar scan to do this. The black region is blocked by the car itself. It is tested to be under 45 degrees on both sides. Similar to the last part we set a virtual "wall" on both sides of the car: we calculate the distance using sine and set a threshold 450 millimeters. If some object is on one side of our car, we block the ability to turn further in that direction. The measurement range is very large, larger than other cars. Also, sometimes only part of the object can be detected in the range. So we assume there is an object when at least 20% of the distances are under the threshold.