2018SpringTeam6
The objective of this project is to add an voice control over the original car model. This voice feature allows user to emergency stop and restart the car by talking to the car as well as switching back and forth from autonomous and joy stick controls. This project was split into several parts as follows:
- Voice recognition through Sopare running on PC
- Use Socket to communicate Pi and PC
- Interference the DonkeyCar with voice signals
Team Members
Will Xu
Jose Pablo Valle
Siming Deng
Yihan Zhang
Voice Recognition Using Sopare
Sopare stands for SOund PAttern REcognition. Sopare is a python project written by bishoph. This software is able to listen offline in real time to microphone input and detect trained labels.
Installation and usage tutorials
git clone [1]
step-by-step tutorial [2]
Usage
1. Assuming you've created a car folder using the donkey library, begin by changing directory to the d2 folder, and running $ python manage.py drive, along with the model you wish to load. A host IP and a port number should be printed. This is the script that you will run using the onboard Raspberry Pi, as this is the script that drives the car.
2. On the computer connected to the same Wifi network that you want to use to send voice signals, make sure your sopare is trained with the following commands: "drive", "steer", "user", "stop", and "go". Instructions for training are in the tutorial linked above.
3. Run the client.py script in the sopare folder, ensuring that the host and port variables in the script align with those printed by the manage.py script on the other pi. Then, you can drive the car as you normally would, with the addition of voice commands sent from your computer to the pi.
Commands and what they do:
"stop": immediately stops the car until a "go" command is received "go": allows the car to move again after a "stop" command "drive": has the car drive autonomously using the model specified in the options of the drive script "steer": has the car steer autonomously using the model specified in the options of the drive script. Throttle is still controlled by the driver "user": returns total control of the car to the driver
Modifying Sopare
We made an Sopare plugins to allow communication between the driving Pi and voice recognition.
Communication between Pi and PC
Modification to the car's driving was implemented through minor adjustments to the controller.py script in ~/donkeycar/donkeycay/parts. The controller.py has a function called update() which loops infinitely while checking for any button presses from the controller that is being used to control the car. We modified this script by creating a server socket in a separate method that is called when update() is called. This server receives byte strings from the client socket, which is implemented through a code injection of sopare. These byte strings match the commands they represent (e.g. "stop" = b'stop'). As strings are received, they are pushed onto a queue. These values are popped from the queue in the update() loop, and the values are checked in a series of if-statements. The 'stop' command stops the loop until a 'go' command is received. The remaining three commands all adjust the self.mode variable to match the desired input.
Creating the server socket:
Checking for signals in update():
