Arduino Projects

Arduino Camera Tracking System using Image processing vb.net EmguCv

Arduino Camera Tracking Project Description:

 

Arduino Camera Tracking– In this Tutorial, you will learn how to make an image processing based human face tracking system using Arduino and vb.net which is also known as visual basic. This project is based on the image processing application designed in VB.net. This is the 4th  version of the Arduino and vb.net based image processing system.


While version1 of the Arduino and vb.net based image processing system was based on the human face recognition and entrance control system using the Electronic Lock.

While in version2 of the Arduino based image processing system, I designed a CCTV camera system.This image processing application is able to capture pictures whenever a human face is detected, in this project the PIR sensor is also used for the feedback.

While in version 3 of the Arduino and vb.net based image processing system, I demonstrated how to make your own xml file to track a specific eye.

Arduino Camera Tracking Tutorial is entirely based on the version1 which is the human face recognition and entrance control system,  which is a pretty long tutorial covering the EmguCv installation, EmguCv settings which is the most important step, relay circuit making and application designing. So for the best understanding I recommend you should first watch my previous tutorials and then you can resume from here. Because in this tutorial, I will not explain the things which I have already explained in Version1. I did a very little change in the code which I used in version1. So in this tutorial, I will only  discuss the changes. This tutorial covers

  1. Circuit Diagram
  2. Motor H-Bridge explanation
  3. Arduino programming
  4. Application programming
  5. Testing


For the step-by-step explanation watch the video tutorial given at the end of this Article.

Amazon Links:

12v Adaptor:

Arduino Uno

Arduino Nano

Mega 2560:

Car Wiper 12V gear Motor

WebCam:

Other Tools and Components:

Top Arduino Sensors:

Super Starter kit for Beginners

Digital Oscilloscopes

Variable Supply

Digital Multimeter

Soldering iron kits

PCB small portable drill machines

DISCLAIMER:

Please Note: these are affiliate links. I may make a commission if you buy the components through these links. I would appreciate your support in this way!


Circuit Diagram of the Arduino Camera Tracking System:

Arduino Camera Tracking

This is the complete circuit diagram designed in Cadsoft eagle 9.1.0 version. If you want to learn how to make a schematic and PCB then watch my tutorial.

Let’s start with the toy machine trigger. The machine trigger is controlled using a 12v gear motor. This gear motor is controlled using a 12v Relay. This relay is of the type SPDT which stands for the single pole and double throw. This relay has total of 5 pins, two coil pins, common pin, normally open and normally closed. To control this relay we need a driver. The driver simply consists of the 2n2222 NPN transistor and a 10k resistor. The selection of the transistor depends on the relay coil current needed to energize the relay coil. The relay coil resistance can be found using the digital multi-meter and then using the ohm’s law

V = IR


We can find the coil current. You can use any NPN transistor; so far its collector current is greater than the relay coil current. I selected 2n2222 NPN transistor because it has a low price, easily available in electronics shops, you can think of the 2n2222 NPN transistor as the cockroach of the electronics world.

Now let’s have a look at the H-bridge. As you know in the circuit diagram above, an h-bridge is used to control the direction of the dc motor. The relays normally open, normally closed and common contacts are connected with the dc gear motor in such a way that by turning on and turn off the relays the motor direction can be controlled. I have also used the same h-bridge in the tongue controlled wheelchair. Then I tested this circuit using Proteus simulation software. This H-bridge Module will be used to rotate the platform on which the machine is fixed. For the practical demonstration watch Video tutorial given at the end.

Proteus simulation of the H-bridge:

Arduino Camera Tracking

Download Proteus H-Bridge Simulation: h bridge

Arduino Camera Tracking Controller Programming:

int hmw1 = 7; // horizontal motor wire1 
int hmw2 = 8; // horizontal motor wire2

int trigger = 13; // machine gun trigger motor connected here 


void setup()
{


  Serial.begin(9600); 

  pinMode(hmw1, OUTPUT);
   pinMode(hmw2, OUTPUT);
    pinMode(trigger, OUTPUT);
   
  // keep motors off by default
  
  digitalWrite(hmw1, LOW);
  digitalWrite(hmw2, LOW);
  digitalWrite(trigger, LOW);
  
}

void loop()
{

 // data from computer, as per the image processing. signals from computer for the left and right movement of the machine gun and trigger. 
 
  while(Serial.available() > 0) // 
  {
    char received = Serial.read();


 
  if(received == 'd') // if no human
 {
    digitalWrite(hmw1, LOW); // STOP ALL MOTORS
    digitalWrite(hmw2, LOW); 
    digitalWrite(trigger, LOW); 

 }
 
   if(received == 'l') // for human on left
 {
    digitalWrite(hmw1, HIGH);
    digitalWrite(hmw2, LOW);
    digitalWrite(trigger, LOW); 
 }
 
    if(received == 'r') // for human on right
 {
        digitalWrite(hmw1, LOW);
    digitalWrite(hmw2, HIGH);
    digitalWrite(trigger, LOW);
  
 }
   if(received == 'c') // HUMAN FACE ON front
 {
    digitalWrite(hmw1, LOW); // STOP ALL MOTORS
    digitalWrite(hmw2, LOW); 
digitalWrite(trigger, HIGH);

 }
  }
}

Program Explanation:

I started off by defining pins for controlling the h-bridge. As the h-bridge has two relays, so one relay will be controlled using the Arduino’s pin number 7 and the other relay will be controlled using the Arduino’s pin number 8. While the trigger motor will be controlled using the Arduino’s pin number 13.

As the image processing application will send the control commands serially to the Arduino so we will need to activate the serial communication.


Serial.begin(9600); // activates the serial communication while 9600 is the baud rate. All the motors are set to OUTPUT using the pinMode function and keep all the motor off by default…then starts the void loop function.

  while(Serial.available() > 0) // if we have received data or a command from the image processing application, then simply read the serial port and store the received character in variable received.  Then using if conditions we control the motors accordingly.

Computer Application Designed for Arduino Camera tracking system:

Arduino Camera Tracking

Imports Emgu.CV
Imports Emgu.CV.Util

Imports Emgu.CV.Structure
Imports System.Diagnostics

Imports System.IO
Imports System.IO.Ports
Imports System.Threading


Public Class Form1


    Dim facedetected As Integer
    Dim facepresent As Integer
    Dim web As Capture = New Capture(0) ' camera number


    Private Sub Timer1_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer1.Tick

        faces()
    End Sub


    Private Sub wait(ByVal interval As Integer)
        Dim sw As New Stopwatch
        sw.Start()
        Do While sw.ElapsedMilliseconds < interval
            ' Allows UI to remain responsive
            Application.DoEvents()
        Loop
        sw.Stop()
    End Sub




    Sub faces()
        Dim photo As Image(Of Bgr, Byte)
        photo = web.RetrieveBgrFrame
        Dim currentCenter As New Point()

        Dim facedetection As New CascadeClassifier("C:\data\haarcascades\haarcascade_frontalface_default.xml") ' this one is for the face detection

        facedetected = 0


        Try
            Dim image As Image(Of Gray, Byte) = photo.Convert(Of Gray, Byte)()

            For Each face As Rectangle In facedetection.DetectMultiScale(image, 1.1, 8, Size.Empty, Size.Empty) ' default 1.1, 8  ( while best values are 1.2 and 17 after checking)

                photo.Draw(face, New Bgr(Color.White), 4)

                currentCenter.X = CInt(CDbl(face.X + face.X + face.Width) / 2.0)
                currentCenter.Y = CInt(CDbl(face.Y + face.Y + face.Height) / 2.0)

                Label2.Text = (currentCenter.X).ToString
                Label3.Text = (currentCenter.Y).ToString

                Dim f = New MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_COMPLEX, 0.5, 0.5)
                photo.Draw("Human detected", f, currentCenter, New Bgr(0, 255, 0)) ' New Point(10, 80) 

                Dim f2 = New MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_COMPLEX, 0.7, 0.7)
                photo.Draw("human detectioin and tracking system", f2, New Point(10, 20), New Bgr(0, 255, 255)) ' New Point(10, 80)


                Label4.Text = face.ToString

                facedetected = 1  ' we are storing 1 if faces are detected.


            Next

            PictureBox1.Image = photo.ToBitmap


        Catch ex As Exception

        End Try

        If facedetected = 1 Then
            label1.Text = "human detected"

        Else
            label1.Text = "no Human"
            facepresent = 0
            SerialPort1.Open()
            SerialPort1.Write("d")

            SerialPort1.Close()

        End If


        ' new addition
        If currentCenter.X > 350 And currentCenter.X < 550 Then

            SerialPort1.Open()
            Label5.Text = " Human is on Right"

            ' i am sending so many "l" this way i can reduce the speed of motor 
            'and by sending less we can increase the speed. 

            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHT
            SerialPort1.Write("l") ' the human is on RIGHTT
            SerialPort1.Write("l") ' the human is on LEFT
            SerialPort1.Write("l") ' the human is on LEFT
            SerialPort1.Write("l") ' the human is on LEFT
            SerialPort1.Write("l") ' the human is on LEFT
            SerialPort1.Write("l") ' the human is on LEFT
            SerialPort1.Write("l") ' the human is on RIGHTT
            SerialPort1.Write("l") ' the human is on LEFT
            SerialPort1.Write("l") ' the human is on LEFT
            SerialPort1.Write("l") ' the human is on LEFT
            SerialPort1.Write("l") ' the human is on LEFT
            SerialPort1.Write("l") ' the human is on LEFT
            SerialPort1.Write("d") ' the human is on LEFT
            SerialPort1.Close()
        End If

        If currentCenter.X > 0 And currentCenter.X < 250 Then

            SerialPort1.Open()
            Label5.Text = " Human is on left"

            ' i am sending so many "r" this way i can reduce the speed of motor 
            'and by sending less we can increase the speed.
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("d") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("r") ' the human is on LEFT
            SerialPort1.Write("d") ' the human is on LEFT

            SerialPort1.Close()
        End If
        ' new addition end 

        If currentCenter.X > 250 And currentCenter.X < 350 Then  ' HUMAN FACE IN CENTER

            SerialPort1.Open()
            Label5.Text = "Human is in Front"
            SerialPort1.Write("c") ' the human is on front

            SerialPort1.Close()
        End If

    End Sub



    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        SerialPort1.Close()
        SerialPort1.PortName = "com5"
        SerialPort1.BaudRate = "9600"
        SerialPort1.DataBits = 8
        SerialPort1.Parity = Parity.None
        SerialPort1.StopBits = StopBits.One
        SerialPort1.Handshake = Handshake.None
        SerialPort1.Encoding = System.Text.Encoding.Default
    End Sub

End Class

This is the same exact application used in Version1 of the Arduino image processing based entrance control system. This time I did a very little change, that is I find the X and Y co-ordinates which you can clearly see in the program. For the best Understanding watch Version1.



Related Arduino Image Processing Projects:

OCR Optical Character recognition using VB.net and Arduino

Engr Fahad

My name is Shahzada Fahad and I am an Electrical Engineer. I have been doing Job in UAE as a site engineer in an Electrical Construction Company. Currently, I am running my own YouTube channel "Electronic Clinic", and managing this Website. My Hobbies are * Watching Movies * Music * Martial Arts * Photography * Travelling * Make Sketches and so on...

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button