NBEL RC 1.0: Part III - Software Design

This post is preceded by the first two posts of this series:

In this post we shall elaborate on the software design of the NBEL RC 1.0. First, we will explore its physical programming (sending the commands that actuate the motors), next we will explore several user interfaces (from ‘button pressing’, through simple GUI, to Android application). Finally we will discuss code integration with Google cloud-based Speech recognition engine to allow voice control.

Physical Programming

As you might remember from our previous post, the NBEL RC 1.0 is controlled using the Raspberry Pi. Particularly, we will use Python to code the sequences needed to drive and steer the car. Here, we will assume basic knowledge in OOP with Python (there are numerous sources on the web discussing Python programming - a simple search will expose many of them).

Writing Python for the Raspberry Pi is not different from traditional Python development, with the exception of importing the necessary libraries and getting familiar with controlling the boards’ GPIOs (General Purpose Inputs Outputs). If you are not familiar with it, please go over the basics of Python programming with the Raspberry Pi in: https://www.raspberrypi.org/documentation/usage/python/, and read about GPIO programming in: https://learn.sparkfun.com/tutorials/raspberry-gpio/python-rpigpio-api.

First, we will define a basic Raspberry Pi control module as class names: ‘RPiBasic’. In its constructor we will define the functional of the involved pins (as they were defined in the previous post):

import RPi.GPIO as GPIO

class RPiBasic():
    def __init__(self):
        # Definitions
        self.Drive_F_pin = 22
        self.Drive_B_pin = 23
        self.Steer_L     = 20
        self.Steer_R     = 21
        self.Steer_pwm   = 18
        self.Drive_CAM_L = 17
        self.Drive_CAM_R = 27
        self.CAM_pwm     = 4

        # Inititalizations
        GPIO.setup(self.Drive_B_pin, GPIO.OUT)
        GPIO.setup(self.Drive_F_pin, GPIO.OUT)
        GPIO.setup(self.Steer_R,     GPIO.OUT)
        GPIO.setup(self.Steer_L,     GPIO.OUT)
        GPIO.setup(self.Steer_pwm,   GPIO.OUT)
        GPIO.setup(self.Drive_CAM_L, GPIO.OUT)
        GPIO.setup(self.Drive_CAM_R, GPIO.OUT)
        GPIO.setup(self.CAM_pwm,     GPIO.OUT)

        self.steer_angle = GPIO.PWM(self.Steer_pwm, 490)
        self.CAM_speed   = GPIO.PWM(self.CAM_pwm, 20)
        self.CAM_speed.  start(0)

In the first line in the constructor we defined our pin numbering to follow BCM numbering system. In the case of our board it means that when we are referring to pin 4, we are aiming at pin GPIO4 and not pin 7 (see image below).

Raspberry Pi pin numbering

Raspberry Pi pin numbering

In the definitions section we assign pins with class attributes, next we define them as outputs (initialisation), and finally we assign the frequency of the PWM signals, which control the steering and the camera rotation with 490 and 20 Hz respectively, initialising with 0 duty cycle (we described PWM signals in our previous post).

Now, we shall define the class functions as pin configuration. For example, to move the car forward we will set the Drive_F_pin to a high value (VCC), and the Drive_B_pin to a low value (GND).

def move_forward(self):
        GPIO.output (self.Drive_F_pin, GPIO.HIGH)
        GPIO.output (self.Drive_B_pin, GPIO.LOW)
        print('Moving forward')

    def stop(self):
        GPIO.output (self.Drive_F_pin, GPIO.LOW)
        GPIO.output (self.Drive_B_pin, GPIO.LOW)
        print('Moving forward')

    def move_reverse(self):
        GPIO.output (self.Drive_B_pin, GPIO.HIGH)
        GPIO.output (self.Drive_F_pin, GPIO.LOW)
        print('Moving in reverse')

    def steer_right(self, angle = 100):
        GPIO.output (self.Steer_R, GPIO.HIGH)
        GPIO.output (self.Steer_L, GPIO.LOW)
        print('Steering right')

    def steer_left(self, angle = 100):
        GPIO.output (self.Steer_L, GPIO.HIGH)
        GPIO.output (self.Steer_R, GPIO.LOW)
        print('Steering left')

    def steer_head(self, angle = 100):
        GPIO.output (self.Steer_L, GPIO.LOW)
        GPIO.output (self.Steer_R, GPIO.LOW)
        print('Steering head')

    def CAM_left(self, pwm=20):
        self.CAM_speed.  start(pwm)
        GPIO.output (self.Drive_CAM_L, GPIO.HIGH)
        GPIO.output (self.Drive_CAM_R, GPIO.LOW)
        print('Camera left')

    def CAM_right(self, pwm=20):
        self.CAM_speed.  start(pwm)
        GPIO.output (self.Drive_CAM_R, GPIO.HIGH)
        GPIO.output (self.Drive_CAM_L, GPIO.LOW)
        print('Camera right')

    def clean(self):

Simple interfaces

The easiest way to control the car in this point is by defining a set of buttons, each assigned with a specific function. We will define this simple GUI in a new class: ‘RC_Manual_GUI’ and use Tk/Tcl package to define the graphics. This package is in wide use in Python development, however, it is not the focus here. You can read more details on the package in: https://docs.python.org/3/library/tk.html. We will define this basic GUI as follows:

class RC_Manual_GUI:

    def __init__(self, RPinterface):
        self.window = tk.Tk()
        self.window.title('NBEL RC 1.0 Control Application')
        self.RPi = RPinterface

    def intialize_contol(self):
        self.lbl = Label(self.window, text = "NBEL RC 1.0 Control", font=("Calibri", 12))
        self.lbl.grid(column = 1, row = 0)
        self.btn_drv = Button (self.window, text='Drive',   command = self.RPi.move_forward)
        self.btn_drv.grid(column=1, row=1)
        self.btn_rve = Button (self.window, text='Reverse', command = self.RPi.move_reverse)
        self.btn_rve.grid(column=1, row=2)
        self.btn_lft = Button (self.window, text='Left',    command = self.RPi.steer_left)
        self.btn_lft.grid(column=0, row=3)
        self.btn_rgt = Button (self.window, text='Right',   command = self.RPi.steer_right)
        self.btn_rgt.grid(column=2, row=3)
        self.btn_head = Button (self.window, text='Head',   command = self.RPi.steer_head)
        self.btn_head.grid(column=1, row=3)
        self.btn_stp = Button (self.window, text='Stop',    command = self.RPi.stop)
        self.btn_stp.grid(column=1, row=4)

We can lunch this simple GUI using:

from GUI import *
from RPiControl import *

RPinterface = RPiBasic()
RC_GUI = RC_Manual_GUI(RPinterface)

This buttons-based GUI however is not intuitive, and does not allow for fine control of the steering. We will devise a better GUI, where the control of the car will be based on a single circle. This circle can be dragged within a larger circle, where its location signals the controller the required command. Schematic of the two approaches is given below.

Screen Shot 2019-02-16 at 3.22.46.png

We shall define the ‘Round_GUI’ class as follows:

class RC_Round_GUI(tk.Frame):

    def __init__(self, parent, RPinterface):
        tk.Frame.__init__(self, parent)
        self.RPi = RPinterface
        # create a canvas
        self.canvas = tk.Canvas(width=400, height=400)
        self.canvas.pack(fill="both", expand=True)

        # keeping track of a dragged item
        self._drag_data = {"x": 0, "y": 0, "item": None}

        # create a movable objects
        self._create_token((200, 200), "black")

        # add bindings for clicking, dragging and releasing over
        # any object with the "token" tag
        self.canvas.tag_bind("token", "<ButtonPress-1>",   self.on_token_press)
        self.canvas.tag_bind("token", "<ButtonRelease-1>", self.on_token_release)
        self.canvas.tag_bind("token", "<B1-Motion>",       self.on_token_motion)
        # Create a bounding circle
        (x,y) = (200, 200)
        self.canvas.create_oval(x-150, y-150, x+150, y+150, 
                    outline='blue', tags="token")

    def _create_token(self, coord, color):
        '''Create a token at the given coordinate in the given color'''
        (x,y) = coord
        self.canvas.create_oval(x-25, y-25, x+25, y+25, 
                                outline=color, fill=color, tags="token")

    def on_token_press(self, event):
        '''Begining drag of an object'''
        # record the item and its location
        self._drag_data["item"] = self.canvas.find_closest(event.x, event.y)[0]
        self._drag_data["x"] = event.x
        self._drag_data["y"] = event.y

    def on_token_release(self, event):
        '''End drag of an object'''
        # reset the drag information
        self._drag_data["item"] = None
        self._drag_data["x"] = 0
        self._drag_data["y"] = 0
        print(event.x, event. y)

        if (250 > event.x > 150) & (250 > event.y > 150):
            self.calc_drive (event.x, event. y)
            self.calc_steering (event.x, event. y)

    def on_token_motion(self, event):
        '''Handle dragging of an object'''
        # compute how much the mouse has moved
        delta_x = event.x - self._drag_data["x"]
        delta_y = event.y - self._drag_data["y"]
        # move the object the appropriate amount        
        if (((event.x - 200)**2 + (event.y - 200)**2)**0.5) < 150:
            self.canvas.move(self._drag_data["item"], delta_x, delta_y)
            # record the new position
            self._drag_data["x"] = event.x
            self._drag_data["y"] = event.y
    def calc_drive(self, x, y):
        if y < 200 :
            print('Move forward')
            print('Move backward')
    def calc_steering(self, x, y):
        angle = 1.1*np.abs(np.abs((180/np.pi)*np.arctan((y-200) / (x-200)))-90)
        if x < 200 :
            print('Move Left, at {}'.format(angle))
            print('Move Right, at {}'.format(angle))

While going through the code, you might notice the following:

  • The controlled oval is created with a tag. We are using this tag to attach (or bund) three functions to it: handling clicking on it, handling releasing it, and moving it. This is an event-driven approach, where functions are executed upon triggering a specific event.

  • When we press on the oval, we record its location. When we drag the oval - changing its location - we redraw the oval and record its new location. Notice that we are changing the oval location only within a specific bound of a circle with a radius of 150 pixels.

  • When the oval is released, we check its location. If its location is roughly at the middle of the bounding circle, we zero all commands (drive and steering). If not, we calculate its drive and steer in dedicated functions.

  • For driving we check if the released oval is above or below the mid-horizontal line, deciding if we should drive the car forward or backward, respectively. For steering, we calculate the angle of the vectorised oval location, related to the mid-vertical line, deciding the steering angle.

We can execute the ‘Round GUI’ using:

from GUI import *
from RPiControl import *
import tkinter as tk

RPinterface = RPiBasic()
root = tk.Tk()
RC_GUI = RC_Round_GUI(root, RPinterface)

These two interfaces work great. However, we would like to remotely control the vehicle. The fastest way to do it is using Bluetooth and the most convenient way to interface with it is using an Android application.

Android-based Graphical Interface

To implement Android-based Bluetooth control, we need two parts: the Raspberry Pi receiving part and the Phone controlling part.

For the Raspberry Pi part we used the bluetooth / BlueZ libraries, with which we can use the on-board Bluetooth chip to communicate with other devices.

from bluetooth import *
import subprocess

class BT_GUI():

    def __init__(self, RPinterface):
        self.RPi = RPinterface

        port = server_socket.getsockname()[1]
        uuid = "00001101-0000-1000-8000-00805F9B34FB"

        advertise_service( server_socket, "MYAPP", service_id = uuid,
                   service_classes = [ uuid, SERIAL_PORT_CLASS ],
                   profiles = [ SERIAL_PORT_PROFILE ])
        # make Bluetooth discoverable
        subprocess.call(['sudo', 'hciconfig', 'hci0', 'piscan'])
        print ("Waiting for connection")
        client_socket, client_info = server_socket.accept()
        print ("Accepted connection from {}".format(client_info))

        while True:          
                data = client_socket.recv(1024).decode('utf-8')
                print ("received {}".format(data))
                x, y = data.split(" ")
                x = int(x)
                y = int(y)

                if (720/2 + 100 > x > 720/2 - 100) & (1280/2 + 100 > y > 1280/2 - 100): 
                    self.calc_drive(x, y)
                    self.calc_steering(x, y)
            except IOError:
    def calc_drive(self, x, y):
        if y < 1280/2 :
            print('Move forward')
            print('Move backward')

    def calc_steering(self, x, y):  
            angle = 1.1*np.abs(np.abs((180/np.pi)*np.arctan((y-1280/2) / (x-720/2)))-90)
        except ZeroDivisionError:
            angle = 0

        if x < 720/2 :
            print('Move Left, at {}'.format(angle))
            print('Move Right, at {}'.format(angle))

While going through the code you might notice the following:

  • First, we initiate the Bluetooth connection, assigning it with a Universally Unique Identifier (UUID). We use the base UUID specified here: https://www.bluetooth.com/specifications/assigned-numbers/service-discovery.

  • Next, we ‘advertise’ a service, calling the terminal using a ‘sub-process’ to make sure the board Bluetooth is discoverable, and then we wait for someone to establish a connection.

  • Once a connection is established, we continuously analyse packets of incoming data. Data is feeding in, in a format of two coordinates (x, y), separated with a space. Here we implemented a specific solution, assuming a phone with a screen size of 720 x 1280 pixels is used. We use the same equations and approach the control of the car, as we used for the ‘round GUI’.

The Bluetooth-based GUI can be executed using:

from GUI import *
from RPiControl import *

RPinterface = RPiBasic()
RC_GUI = BT_GUI(RPinterface)

The second, more complicated part, is the Android application. To program our Android App we used Android Studio (https://developer.android.com/studio/).

Screen Shot 2019-02-16 at 4.55.44.png

App operation streamline is the following: executing the Bluetooth GUI code on the Raspberry Pi - the board is now waiting for connection requests; openning the Android App and pressing ON to enable Bluetooth on the phone; pressing discover to find available connections - the Raspberry Pi should appear on a list at the top left corner of the app; choosing the board in the list will pair the devices; Once the devices are paired, press start connection to establish a communication line between the phone and the Raspberry Pi. The GUI - which is very similar to the ‘Round_GUI’ described above’ - is now operational.

The code of the application is fairly complicated, and the length and breadth of it is not suitable for a blog post. For bluetooth connectivity we used the ‘android.bluetooth’ package. You can read more about the package in: https://developer.android.com/guide/topics/connectivity/bluetooth.

We further extended our application to support voice control. Our current application solely responds to Stop / Go command. This extension is based on Android Speech API. For a great way to start exploring this option check the following tutorial: https://www.developer.com/ws/android/programming/exploring-the-android-speech-api-for-voice-recognition.html.

Executing script on the Raspberry on startup

To allow car operation without having the need to control the board using the Raspberry Pi command line, nor the desktop operating system. There are several ways to execute a script on startup in the Raspberry Pi, and they are succinctly listed in: https://www.dexterindustries.com/howto/run-a-program-on-your-raspberry-pi-at-startup/.

We chose to edit the .bashrc file, adding at the end of the file an execution statement for our script. In addition, to avoid running through the visual system, we used the raspi-config tool (the Raspberry Pi Software Configuration Tool) to define the board Boot to Console Autologin. We used these blog posts to bore our wife to death.

Screen Shot 2019-02-22 at 23.53.03.png
Elishai EzraComment