daily tasks require us to extract data from outside atmosphere, and perform
operations on the basis of useful information obtained from raw data. We
respond in favor of accomplishment of our tasks. There are basically three
steps involved in it (input, processing, and output). The similar task can be
performed by a smart robot.
project idea is to design an autonomous robot which will identify (know there
is an object of interest), recognize (find out the ‘actual’ meaning of the
object by Object Recognition) and respond in accomplishment of pre-specified
task by getting its input with the help of sensor e.g. camera (PiCamera). The
object intended for detection will be signs e.g. a sign which consists of an
arrow which points in the ‘left direction’ or ‘right direction’ or ‘U turn’ or
will a planned arena for the robot to follow in which different signs (say
left, right, U turn) along with a target object (say a circular ball) will be
given. The robot will be able to reach the target object by following the
details extracted from the various signs introduced along its course.
robot will make use of computer vision to extract relevant data from outer
CHAPTER 1: INTRODUCTION
Word robot was coined by a Czech
novelist Karel Capek in a 1920 play titled Rassum’s Universal Robots
(RUR). Robot in Czech is a word for
worker or servant.
In the present era, robots play
crucial role in mass production due to their agility and accurate performance. Future
era of robotics comprises of more intelligent robotic structure, that is much compact
and have even faster processing with wide range of applications from home-based automations
to industrial machinery and from war-crafts to intelligent surveillance robots. Robots
have been widely used in automobile industries by the end of late 20th century
for precise cutting, welding, placing and carrying purposes. Robotics has
achieved its greatest success to date in the world of industrial manufacturing.
Robot arms, or manipulators, comprise a 2 billion dollar industry.
1.2 Problem Statement
The project is based on one of the
fundamental problem that a robot experiences while interacting with the
environment. A robot may require a manual feedback in order to perform its
required task. The manual intervention is somehow a barrier in achieving good
results and accuracy. The human intervention can be minimized by giving visual
features to the robot in order to achieve diverse behavioral response under
different circumstances. We are using CV technique to realize the solution of
this particular problem.
1.2.1 Computer Vision
CV can be briefed as “a field that
involves methods for obtaining, processing, and interpreting images and, in
general, multi-dimensional data from the outside world in order to generate
mathematical or symbolic information, e.g., as a sequence of decisions. The
main idea is the implementation of the abilities of human vision by
electronically processing and understanding an image. This image analyzing can
be seen as the disintegration of numerical information from image data using
analytical methods extracted with the help of geometry, statistics, physics,
and learning theory. Humans employ their eyes and brains to look
and visually understand the world around them. CV is the science that intends
to give a similar compatibility to a machine. CV deals with the automatic
abstraction, processing and comprehension of useful data from a single image or
a set of images (video clip). It involves the implementation of a numerical and
algorithmic method to achieve automatic visual understanding. CV is concerned with the theory behind artificial
systems that extract information from images. The image data can take various
formats, such as video sequences, views from multiple cameras, or
multi-dimensional data from a medical scanner. As a technological discipline, CV
intends to apply its schemes and illustrations for the implementation of CV
CV can be
further categorized in to MV. MV is the approach used to provide imaging-based
automatic inspection and analysis for applications such as automatic handling,
process control, and robot guidance, usually done in industries. MV tends to
focus on utilizations, mainly in manufacturing and inspecting, e.g.,
vision based robots and systems for vision based inspection, measurement, or
picking (robotic arms).
1.3 Project Objectives
The smart robot (able to detect and follow signs) design
implementation is above subsystem level and almost none of the software or
hardware is taken from scratch. The controllers (both master and slave) are
available for purchase along with other periphery (chassis, sensors, actuators)
and OpenCV is released under a BSD license and hence it’s free for both
academic and commercial use. Our goal is to make all the controllers and
peripherals to work simultaneously in order to achieve detection and tracking.
The objectives of the project are as follows:
· Implement OpenCV with Python.
· Compare the raw image to pre-specified images and detect the sign
from a list of signs.
· Choose the necessary action after recognizing a particular sign
(i.e. Turn Left, Right, Backwards or Stop etc.).
· Reach the destination efficiently.
1.4 Project Scopes
The scopes of study are as follows:
· Using Raspberry Pi Model B+ V1.2 as master processor (object
detection and tracking).
· Using Arduino Uno as slave processor (drive motor and obstacle
avoidance). It will have a serial communication with Raspberry Pi and will be
used to send commands to the drive motors. The Arduino will also read inputs
from the ultrasonic sensors and sends that information to the Raspberry Pi.
· Using PiCamera to capture real world image or video (set of images)
· Using Dual-H-Bridge L298 as DC motor driver.
· Using Ultrasonic sensor for obstacle avoidance.
· Using DC motor for robotic movement.
CHAPTER 2: EXPERIMENTAL SETUP
section the specific
components that were
utilized during the experimentation and testing process are
described with their principles of operation in the same order they were
2.1.1 Raspberry Pi
The Raspberry Pi is a series of
computers developed in
Pi Foundation to
promote the teaching of basic computer science in schools and in developing
countries. We are using Raspberry Pi for
image processing purposes. The microprocessor model is RPi B+ V1.2. We use this
model as it has more GPIO pins, lower power
consumption, and neater form factor than other models.
Figure 1: Raspberry Pi (Top View) 1
A brief overview of RaspberryPi Model B+ V1.2 is given bellow:
· Dual step-down (buck) power supply for 3.3V and 1.8V5V supply has polarity
protection, 2A fuse and hot-swap protection.
· New USB/Ethernet controller chip.
· 4 USB ports.
· 40 GPIO pins.
· 3.5mm ‘headphone’ jack.
· MicroSD card socket instead of full size SD.
· Four mounting holes in rectangular layout.
· Many connectors moved around.
Arduino is a brand
that produces development boards, kits and opens source software for the small and commercial projects. It is used widely around
the Globe. We selected
Arduino for the precise
movement of our robot on the basis of signals obtained from RaspberryPi and
ultrasonic sensors. Arduino will control the motion of robot with the help of
motor driver. We selected Arduino because of
our low cost requirement with reusability and market
trust as major factors. There are other project boards available in market like
STM development kits, TI Cortex M3/M4 boards having more IO’s than Arduino Uno
board we used, but since Arduino is meeting our requirements therefore, we
Figure 2: Arduino Uno Board 2
at the board from the top down, this is an outline of what you will see (parts
of the board you might interact with in the course of normal use are
Figure 3: Arduino Uno Elaborative Diagram 3
clockwise from the top centre:
Reference pin (orange)
Ground (light green)
Pins 2-13 (green)
Pins 0-1/Serial In/Out – TX/RX (dark green) – These pins cannot be used for
digital I/O (digitalRead and digitalWrite) if you
are also using serial communication (e.g. Serial.begin).
Button – S1 (dark blue)
Serial Programmer (blue-green)
Analog In Pins 0-5 (light blue)
and Ground Pins (power: orange, grounds: light
Power Supply In (9-12VDC) – X1 (pink)
Power and USB Power (place jumper on two pins closest
to desired supply) – SV1 (purple)
(used for uploading sketches to the board and for serial communication between
the board and the computer; can be used to power the board) (yellow).
which 6 provide PWM output)
Digital I/O Pins
Current per I/O Pin
Current for 3.3V Pin
(ATmega328P) of which 0.5 KB used by bootloader
1: Arduino Board Specification 1
2.1.3 Raspberry Pi Camera
board size: 25mm x 20mm x 9mm
(2592×1944 pixels) Omni vision 5647 sensor in a fixed focus module
1080p30, 720p60 and 640x480p60/90 video record
Figure 4: Raspberry Pi Camera
2.1.4 Ultrasonic Sensors
non-contact distance measurements
within a 2 cm to 3 m range
work in any lighting
condition, making this
a good choice to
supplement infrared object detectors
pulse in/pulse out communication requires just one I/O pin
indicator LED shows measurement in progress
header makes it easy to connect to a development board, directly or with an
extension cable, no soldering required
Figure 5: Ultrasonic Sensor 5
2.1.5 Dual H-Bridge
H-Bridge is derived from the typical graphical representation of such a
circuit. An H bridge is built with four switches (solid-state or mechanical). Since
the IC packaging contains two set of
H- Bridges, hence the name “Dual H-Bridge”. When the switches S1 and S4 are
closed (and S2 and S3 are open) a positive voltage will be applied across the
motor. By opening S1 and S4 switches and closing S2 and S3 switches, this voltage is reversed,
allowing reverse operation
of the motor.
Figure 6: Switches based H-Bridge for driving a DC motor both
ways (FWD/ BWD) 6
Using the nomenclature above, the switches S1
and S2 should never be closed at the same time, as this would cause a short
circuit on the input voltage source. The same applies
to the switches S3 and S4. This condition
is known as shoot-through. We use a
dual H bridge readily available in the market was utilized and the problem we
were experiencing is solved as this H- bridge is capable of providing +5 and -5
volts with a maximum of 2.5 A as load current. Moreover, this Bridge circuit
module is capable of driving two motors simultaneous with a single module based
upon L298 IC.
2.1.6 Mechanical Structure
The mechanical structure is a two layer acrylic fiber.
Figure 7: Mechanical Structure 7
CHAPTER 3: WORKING PRINCIPLE
At first, the robot will start revolving
about itself in search of a sign at the location where it started. The Pi camera will check if it sees any sign, every
time it pauses revolving for a short time. If it won’t find the sign within a certain
amount of rotation it will translate some
finite distance and again start
searching. It will
start approaching the
sign after detecting it.
Once the robot has found a sign, it needs to approach
it. It will do this by ?rst centering the sign in the
view. Once it center’s the sign, it will drive straight towards
it. Every now and then, it will check
again to make sure the sign is still at the center
of the view and adjust
if it isn’t. Also, it will constantly check distance to the sign. Once it reaches
a distance where the sign is close
enough, it will proceed
to the next stage of color detection.
Once the robot
is close enough
to look at the sign,
it will need to ?gure
out the meaning
of the sign. It will use image thresholding and contour detection to determine the shape. Once
it detects the sign, it will
make a move according to the sign.
Search for the new sign
and repeat movements highlighted 2 and 3 until it finds the
After recognizing the last sign (i.e. a sign of a ball),
the bot will move in search of a spherical
ball and when the ball is identified, the bot will be go to the ball and play a buzzer at the end indicating the completion of the route.
CHAPTER 4: PYTHON CODE
The code is under development.
CHAPTER 4: REFERENCES