From MAE/ECE 148 - Introduction to Autonomous Vehicles
Revision as of 22:00, 26 January 2018 by Mauricio (talk | contribs) (→‎Build)
Jump to navigation Jump to search


Designing and fabricating a wheel based mobile robot vs. building on the top of a reliable platform such as an R/C truck 1/8 scale.

  • R/C Truck 1/8 Scale
  • Embedded system 4 cores Linux
  • 5 megapixel camera
  • Servo controller
  • Power management
  • LiPo Battery
  • Safety on LiPo charging, use, and storage
  • Wireless Emergency Off (EMO) circuit

For more information on select helpful topics click on:


When you're done you should be able to train and have your car drive autonomously on our track. Here is a video from the Fall 2017 session.

{{#evu:https://www.youtube.com/watch?v=M7cTiZiI0co |alignment=center }}

Raspberry Pi Setup

Follow the instructions from:


to download an image that already contains all that you need to run the donkey car software.

Follow the instructions in Get the Raspberry Pi working.

This next step is very important. If you miss this configuration before you boot for the first time you will not be able to connect to your Pi using the lab network!

Substitute the content of the file /boot/wpa_supplicant.conf given in the section Setup the Pi's WiFi for first boot by the following:

   ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
       psk="** ASK YOUR INSTRUCTOR **"
       psk="** ASK YOUR INSTRUCTOR **"

and follow the instructions for booting your Pi for the first time, including the section Setup Pi's Hostname. After the first boot, this file will be moved to /etc/wpa_supplicant/wpa_supplicant.conf where it may be edited later.

STOP do not run anything starting at Connecting to the Pi.


First remove the donkey and d2 directories

   (env)pi@jackrpi02:~ $ rm -rf donkeycar
   (env)pi@jackrpi02:~ $ rm -rf d2

Download the latest Donkey code

   (env)pi@jackrpi02:~ $ git clone https://github.com/wroscoe/donkey donkeycar

Install Donkeycar:

   (env)pi@jackrpi02:~ $ pip install -e donkeycar

Create a car folder.

   (env)pi@jackrpi02:~ $ donkey createcar --path ~/d2

The latest command will produce an output similar to:

   Using TensorFlow backend.
   Creating car folder: /home/pi/d2
   making dir  /home/pi/d2
   Creating data & model folders.
   making dir  /home/pi/d2/models
   making dir  /home/pi/d2/data
   making dir  /home/pi/d2/logs
   Copying car application template: donkey2
   Copying car config defaults. Adjust these before starting your car.
   Donkey setup complete.

Testing the connection of the RPI to the Pulse Width Modulation (PWM) Controller


   (env)pi@jackrpi02:~ $ sudo i2cdetect -y 1

which should produce an output like:

(env)pi@jackrpi02:~ $ sudo i2cdetect -y 1

        0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
   00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
   10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
   20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
   30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
   40: 40 -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
   50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
   60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
   70: 70 -- -- -- -- -- -- --

The important point here is to get a response from the I2C addresses 40 and 70. If you do not see these numbers you are not connected to the ESC or the Servo controller.

Emergency Stop Circuit

For this part of your robot, you will have to do some hacking.

First read the datasheet of the components provided with the robot kit and discuss them with your teammates how the emergency stop switch (EMO) will work:

  • What is the disable pin the PWM controller?
  • Does is disable on logic 1 or 0?
  • How to wire the wireless relay to provide the logic you need to disable PWM controller? (1 or 0)
  • How to connect the LEDs (Blue and Red) to indicate (RED - hot/enabled), (BLUE - power is on / disabled).

Note: The power to the PWM controller comes from ESC (Electronics Speed Controller). Therefore, you have to connect the robot batteries to the ESC and the ESC to the PWM controller. We are using Channel 2 for the ESC.

After you see that the EMO is working, i.e. wireless remote control disable the PWM, and the LEDs light up as planned, then you need to document your work. Please use Fritzing (http://fritzing.org/home/) to document your electrical schematic.

It may seem we did the opposite way; document after your build. In our case, you learned how to hack. Since you are an engineer you need to document even it if was a hack…

Working in a company you may do fast prototyping first then document. On larger projects you make schematics, diagrams, drawings, work instructions, then build it. Keep that in mind!

Before Calibration

Before you develop code or you can use a someone's code to control a robot, we need to calibrate the steering servo and the brushless DC motor ESC to find out its range of motion compared to the control or command a computer will send to the controllers.

The calibration is car specific. However, because we are all using the same platform, the numbers for the different groups should be close.

We will following the standard connection from RC CAR world, that is we will use Channel 1 for Steering and Channel 2 for Throttle. Donkeycar uses Channels 0 and 1 so we will need to some settings later.

Perform the following steps:

  1. Connect the Steering Servo connector to the Servo controller Channel 1
  2. Connect the Throttle connector to the Servo controller Channel 2


Perform the following steps:

  1. Connect the batteries and batteries monitor
  2. Power the RPI
  3. Power the Electronic Speed Controller (ESC)
  4. Enable the robot (EMO) - LED should be RED

Steering Calibration

The following commands need to be run from the directory you created for your car, i.e., d2

Change directory to d2:

   (env)pi@jackrpi02:~ $ cd ~/d2
   (env)pi@jackrpi02:~/d2 $ python manage.py calibrate

The above command is iterative. Provide the information as in the following output:

   Using TensorFlow backend.
   loading config file: /home/pi/d2/config.py
   config loaded
   Enter the channel your actuator uses (0-15).1
   Enter a PWM setting to test(100-600)365
   Enter a PWM setting to test(100-600)285
   Enter a PWM setting to test(100-600)440

Try some values around 360 to center the steering. Take note of the values that steer completely to the left and completely to the right. If you hear a buzzing noise you have reached the maximum left or right. My values are:

365 - Center 285 - Steering left max 440 - Steering right max

Notes: If the command python manage.py calibrate times-out, just run it again. You can interrupt the calibration by typing CTRL-C

Throttle Calibration

Repeat the commands used to calibrate the steering this time to calibrate the throttle, this time using Channel 2 instead of Channel 1.

   (env)pi@jackrpi02:~/d2 $ python manage.py calibrate

The above command is iterative. Provide the information as in the following output:

   Using TensorFlow backend.
   loading config file: /home/pi/d2/config.py
   config loaded
   Enter the channel your actuator uses (0-15).2
   Enter a PWM setting to test(100-600)370
   Enter a PWM setting to test(100-600)380
   Enter a PWM setting to test(100-600)390

370 powers up the ESC and put it in neutral. Then go in increments of 10~20 until you can no longer hear increase on the speed of the car. Don’t worry much about the max speed since we won’t drive that fast autonomously and during training the speed will be limited.

My max speed forward was at around 460.

Reverse on RC cars is a little tricky because the ESC needs to receive a reverse pulse, zero pulse, then another reverse pulse, to start to go backwards. Any value below 370 is a reverse throttle.

To see how this works, use the same technique as above to set the PWM setting to your zero throttle (lets say 370). Enter the reverse value, then the zero throttle (e.g., 370) value, then the reverse value again. Increment the reverse value by 10~20 to find a reasonable reverse speed. Remember this reverse PWM value.

An example of a reverse calibration session output is:

   (env)pi@jackrpi02:~/d2 $ python manage.py calibrate
   Using TensorFlow backend.
   loading config file: /home/pi/d2/config.py
   config loaded
   Enter the channel your actuator uses (0-15).2
   Enter a PWM setting to test(100-600)360
   Enter a PWM setting to test(100-600)370
   Enter a PWM setting to test(100-600)360
   Enter a PWM setting to test(100-600)350
   Enter a PWM setting to test(100-600)340
   Enter a PWM setting to test(100-600)330
   Enter a PWM setting to test(100-600)320
   Enter a PWM setting to test(100-600)310
   Enter a PWM setting to test(100-600)300
   Enter a PWM setting to test(100-600)290

Update the configuration file

At this point you should have notes of the following extreme values for steering:

  • Center: 365
  • Steering left max: 285
  • Steering right max: 440

and throttle:

  • Neutral: 370
  • Max speed forward: 460
  • Max speed backwards: 280

Now let's write these values into the car configuration file ~/d2/config.py. Edit this file config.py using your favorite editor, e.g.

   (env)pi@jackrpi02:~/ $ cd ~/d2
   (env)pi@jackrpi02:~/d2 $ nano config.py

and locate and change the following values to match the ones you obtained in your calibration:


Also change these:


Connecting the PS3 Keypad/Joystick

See http://docs.donkeycar.com/parts/controllers/ for more information.

Bluetooth Setup

First install the bluetooth software in the RPI:

   (env)pi@jackrpi02:~/d2 $ sudo apt-get install bluetooth libbluetooth3 libusb-dev
   (env)pi@jackrpi02:~/d2 $ sudo systemctl enable bluetooth.service
   (env)pi@jackrpi02:~/d2 $ sudo usermod -G bluetooth -a pi

You must now power cycle your RPI. Do not just reboot, actually shut down, pull the power, wait a few seconds and reconnect. This may be overkill. Do as follows:

   (env)pi@jackrpi02:~/d2 $ sudo shutdown now

Remove the power from the RPI. Wait 5 s then power it back.

Download, compile, and install the following bluetooth configuration software

   (env)pi@jackrpi02:~ $ cd ~/d2
   (env)pi@jackrpi02:~/d2 $ wget http://www.pabr.org/sixlinux/sixpair.c
   (env)pi@jackrpi02:~/d2 $ gcc -o sixpair sixpair.c -lusb

PS3 Controller setup

Now connect your PS3 Controller to the RPI using a USB Cable and run:

   (env)pi@jackrpi02:~/d2 $ sudo ./sixpair

which produces an output such as:

   Current Bluetooth master: b8:27:eb:49:2d:8c
   Setting master bd_addr to b8:27:eb:49:2d:8c

Disconnect your controller from the USB port, and run the ‘bluetoothctl’ command as a regular user (you don’t need to be root for this). Pay attention to your PS3 controller mac address!

   (env)pi@jackrpi02:~/d2 $ bluetoothctl
   [NEW] Controller B8:27:EB:49:2D:8C jackrpi02 [default]
   [NEW] Device 00:16:FE:74:12:B7 PLAYSTATION(R)3 Controller
   [bluetooth]# agent on
   Agent registered
   [bluetooth]# trust 00:16:FE:74:12:B7
   [CHG] Device 00:16:FE:74:12:B7 Trusted: yes
   Changing 00:16:FE:74:12:B7 trust succeeded
   [bluetooth]# quit
   Agent unregistered
   [DEL] Controller B8:27:EB:49:2D:8C jackrpi02 [default]
   (env)pi@jackrpi02:~/d2 $ 

If you inspect the input devices:

   (env)pi@jackrpi02:~/d2 $ ls /dev/input

alll you see is one entry for


Now press the PS button. The lights on the front of the controller should flash for a couple of seconds then stop, leaving a single light on. If you now look again at the contents of /dev/input you should see a new device, probably called something like js0:

   (env)pi@jackrpi02:~/d2 $ ls /dev/input

and you should see

   event0  js0  mice

If a new device has appeared here then congratulations, you have successfully paired your dongle and SixAxis controller. This will persist across reboots, so from now on you can just connect by pressing the PS button on the controller. Pressing and holding this button will shut the controller down - at the moment there’s no timeout so be sure to turn the controller off when you’re not going to be using it for a while.

If you need to remove a device, lets say a JoyStick you don’t use anymore, type:

   (env)pi@rpimine01:~ $ bluetoothctl
   [NEW] Controller B8:27:EB:72:95:A6 rpimine01 [default]
   [NEW] Device 00:1E:3D:D8:EA:15 PLAYSTATION(R)3 Controller
   [NEW] Device 00:16:FE:74:12:B7 PLAYSTATION(R)3 Controller
   [bluetooth]# remove 00:1E:3D:D8:EA:15
   [DEL] Device 00:1E:3D:D8:EA:15 PLAYSTATION(R)3 Controller
   Device has been removed

Driving the Robot to Collect data

Now you are ready to drive you robot to collect data. Make sure to keep the EMO randy and use when needed!

Remember you are driving at ½ (0.50) of the speed of the robot based on the config.py that you edited. To reverse you have to reverse, stop, reverse. This is a feature of the ESC.


  • Use CTRL-C to stop the python manage.py drive
  • If you want to wipe clean the data collected remove the content of the ~/d2/data directory. It should contain subdirectories named tub_*.

It is safe to delete the entire directory then create it again when you're not running python manage.py.

On the RPI issue the command:

   (env)pi@jackrpi02:~/d2 $ python manage.py drive

As you drive around, you should see something like this in your terminal window:

   Using TensorFlow backend.
   loading config file: /home/pi/d2/config.py
   config loaded
   PiCamera loaded.. .warming camera
   Adding part PiCamera.
   Opening /dev/input/js0...
   Device name: PLAYSTATION(R)3 Controller
   Adding part JoystickController.
   Adding part Lambda.
   throttle -0.0
   Adding part KerasCategorical.
   Adding part Lambda.
   Adding part PWMSteering.
   Adding part PWMThrottle.
   tub does NOT exist
   Adding part TubWriter.
   Starting vehicle...
   /home/pi/env/lib/python3.4/site-packages/picamera/encoders.py:544: PiCameraResolutionRounded: frame size rounded up from 160x120 to 160x128 width, height, fwidth, fheight)))
   angle 0.0
   throttle 0.14949491866817224
   throttle 0.18558305612353893
   throttle 0.11340678121280556
   throttle -0.0
   throttle 0.010315256202887051
   throttle 0.19074068422498244
   throttle -0.05668813135166478
   throttle -0.0
   angle -0.010315256202887051
   angle 0.0
   angle 0.05154576250495926
   angle 0.13400677510910367
   angle 0.15463728751487776
   angle 0.14432203131199073

Note: If you get the following error trying to drive it is because the PS3 game controller is of or not connected:

   file "manage.py", line 44, in drive
     File "/home/pi/donkeycar/donkeycar/parts/controllers/joystick.py", line 220, in __init__
     File "/home/pi/donkeycar/donkeycar/parts/controllers/joystick.py", line 106, in init
       self.jsdev = open(self.dev_fn, 'rb')
   FileNotFoundError: [Errno 2] No such file or directory: '/dev/input/js0'

See Driving with Physical Joystick Controller for more information on the buttons of the remote.

Training on the UCSD GPU Cluster

This quarter you will have access to the UCSD GPU cluster. You must use ieng6 to log in.

If your primary UCSD student account is on this server just use that account to log in.

If you do not have an account on ieng6, a course specific account was created for you to log in. To find the course specific account go to the *Account Lookup* link under *Tools* at http://acms.ucsd.edu and click on the details button. There are instructions for setting a password to the account.

Instructions for using the cluster can be found at http://go.ucsd.edu/2CZladZ.


  • Track building - hands-on ~ 40x20 m

See 1/10 scale race rules for details. Note that we have to scale the curves to match out 1/8th scale!

Neural Networks

Download the jupyter notebook tutorial from the github repository


Open CV


Autonomous Vehicles

  • Odometry with wheels encoders and IMU
  • Short range obstacle avoidance using low cost ultrasonic sensors
  • Street light development RED/YELLOW/GREEN
  • Image processing with OpenCV
  • Image segmentation - far view, close view, fast processing