Arduino Camera Tracking System using Image processing vb.net EmguCv
Table of Contents
Arduino Camera Tracking Project Description:
Arduino Camera Tracking– In this Tutorial, you will learn how to make an image processing based human face tracking system using Arduino and vb.net which is also known as visual basic. This project is based on the image processing application designed in VB.net. This is the 4th version of the Arduino and vb.net based image processing system.
While version1 of the Arduino and vb.net based image processing system was based on the human face recognition and entrance control system using the Electronic Lock.
While in version2 of the Arduino based image processing system, I designed a CCTV camera system.This image processing application is able to capture pictures whenever a human face is detected, in this project the PIR sensor is also used for the feedback.
While in version 3 of the Arduino and vb.net based image processing system, I demonstrated how to make your own xml file to track a specific eye.
Arduino Camera Tracking Tutorial is entirely based on the version1 which is the human face recognition and entrance control system, which is a pretty long tutorial covering the EmguCv installation, EmguCv settings which is the most important step, relay circuit making and application designing. So for the best understanding I recommend you should first watch my previous tutorials and then you can resume from here. Because in this tutorial, I will not explain the things which I have already explained in Version1. I did a very little change in the code which I used in version1. So in this tutorial, I will only discuss the changes. This tutorial covers
- Circuit Diagram
- Motor H-Bridge explanation
- Arduino programming
- Application programming
- Testing
For the step-by-step explanation watch the video tutorial given at the end of this Article.
Amazon Links:
Arduino Nano USB-C Type (Recommended)
Other Tools and Components:
ESP32 WiFi + Bluetooth Module (Recommended)
Super Starter kit for Beginners
PCB small portable drill machines
DISCLAIMER:
Please Note: these are affiliate links. I may make a commission if you buy the components through these links. I would appreciate your support in this way!
Circuit Diagram of the Arduino Camera Tracking System:
This is the complete circuit diagram designed in Cadsoft eagle 9.1.0 version. If you want to learn how to make a schematic and PCB then watch my tutorial.
Let’s start with the toy machine trigger. The machine trigger is controlled using a 12v gear motor. This gear motor is controlled using a 12v Relay. This relay is of the type SPDT which stands for the single pole and double throw. This relay has total of 5 pins, two coil pins, common pin, normally open and normally closed. To control this relay we need a driver. The driver simply consists of the 2n2222 NPN transistor and a 10k resistor. The selection of the transistor depends on the relay coil current needed to energize the relay coil. The relay coil resistance can be found using the digital multi-meter and then using the ohm’s law
V = IR
We can find the coil current. You can use any NPN transistor; so far its collector current is greater than the relay coil current. I selected 2n2222 NPN transistor because it has a low price, easily available in electronics shops, you can think of the 2n2222 NPN transistor as the cockroach of the electronics world.
Now let’s have a look at the H-bridge. As you know in the circuit diagram above, an h-bridge is used to control the direction of the dc motor. The relays normally open, normally closed and common contacts are connected with the dc gear motor in such a way that by turning on and turn off the relays the motor direction can be controlled. I have also used the same h-bridge in the tongue controlled wheelchair. Then I tested this circuit using Proteus simulation software. This H-bridge Module will be used to rotate the platform on which the machine is fixed. For the practical demonstration watch Video tutorial given at the end.
Proteus simulation of the H-bridge:
Download Proteus H-Bridge Simulation: h bridge
Arduino Camera Tracking Controller Programming:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 |
int hmw1 = 7; // horizontal motor wire1 int hmw2 = 8; // horizontal motor wire2 int trigger = 13; // machine gun trigger motor connected here void setup() { Serial.begin(9600); pinMode(hmw1, OUTPUT); pinMode(hmw2, OUTPUT); pinMode(trigger, OUTPUT); // keep motors off by default digitalWrite(hmw1, LOW); digitalWrite(hmw2, LOW); digitalWrite(trigger, LOW); } void loop() { // data from computer, as per the image processing. signals from computer for the left and right movement of the machine gun and trigger. while(Serial.available() > 0) // { char received = Serial.read(); if(received == 'd') // if no human { digitalWrite(hmw1, LOW); // STOP ALL MOTORS digitalWrite(hmw2, LOW); digitalWrite(trigger, LOW); } if(received == 'l') // for human on left { digitalWrite(hmw1, HIGH); digitalWrite(hmw2, LOW); digitalWrite(trigger, LOW); } if(received == 'r') // for human on right { digitalWrite(hmw1, LOW); digitalWrite(hmw2, HIGH); digitalWrite(trigger, LOW); } if(received == 'c') // HUMAN FACE ON front { digitalWrite(hmw1, LOW); // STOP ALL MOTORS digitalWrite(hmw2, LOW); digitalWrite(trigger, HIGH); } } } |
Program Explanation:
I started off by defining pins for controlling the h-bridge. As the h-bridge has two relays, so one relay will be controlled using the Arduino’s pin number 7 and the other relay will be controlled using the Arduino’s pin number 8. While the trigger motor will be controlled using the Arduino’s pin number 13.
As the image processing application will send the control commands serially to the Arduino so we will need to activate the serial communication.
Serial.begin(9600); // activates the serial communication while 9600 is the baud rate. All the motors are set to OUTPUT using the pinMode function and keep all the motor off by default…then starts the void loop function.
while(Serial.available() > 0) // if we have received data or a command from the image processing application, then simply read the serial port and store the received character in variable received. Then using if conditions we control the motors accordingly.
Computer Application Designed for Arduino Camera tracking system:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 |
Imports Emgu.CV Imports Emgu.CV.Util Imports Emgu.CV.Structure Imports System.Diagnostics Imports System.IO Imports System.IO.Ports Imports System.Threading Public Class Form1 Dim facedetected As Integer Dim facepresent As Integer Dim web As Capture = New Capture(0) ' camera number Private Sub Timer1_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer1.Tick faces() End Sub Private Sub wait(ByVal interval As Integer) Dim sw As New Stopwatch sw.Start() Do While sw.ElapsedMilliseconds < interval ' Allows UI to remain responsive Application.DoEvents() Loop sw.Stop() End Sub Sub faces() Dim photo As Image(Of Bgr, Byte) photo = web.RetrieveBgrFrame Dim currentCenter As New Point() Dim facedetection As New CascadeClassifier("C:\data\haarcascades\haarcascade_frontalface_default.xml") ' this one is for the face detection facedetected = 0 Try Dim image As Image(Of Gray, Byte) = photo.Convert(Of Gray, Byte)() For Each face As Rectangle In facedetection.DetectMultiScale(image, 1.1, 8, Size.Empty, Size.Empty) ' default 1.1, 8 ( while best values are 1.2 and 17 after checking) photo.Draw(face, New Bgr(Color.White), 4) currentCenter.X = CInt(CDbl(face.X + face.X + face.Width) / 2.0) currentCenter.Y = CInt(CDbl(face.Y + face.Y + face.Height) / 2.0) Label2.Text = (currentCenter.X).ToString Label3.Text = (currentCenter.Y).ToString Dim f = New MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_COMPLEX, 0.5, 0.5) photo.Draw("Human detected", f, currentCenter, New Bgr(0, 255, 0)) ' New Point(10, 80) Dim f2 = New MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_COMPLEX, 0.7, 0.7) photo.Draw("human detectioin and tracking system", f2, New Point(10, 20), New Bgr(0, 255, 255)) ' New Point(10, 80) Label4.Text = face.ToString facedetected = 1 ' we are storing 1 if faces are detected. Next PictureBox1.Image = photo.ToBitmap Catch ex As Exception End Try If facedetected = 1 Then label1.Text = "human detected" Else label1.Text = "no Human" facepresent = 0 SerialPort1.Open() SerialPort1.Write("d") SerialPort1.Close() End If ' new addition If currentCenter.X > 350 And currentCenter.X < 550 Then SerialPort1.Open() Label5.Text = " Human is on Right" ' i am sending so many "l" this way i can reduce the speed of motor 'and by sending less we can increase the speed. SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHT SerialPort1.Write("l") ' the human is on RIGHTT SerialPort1.Write("l") ' the human is on LEFT SerialPort1.Write("l") ' the human is on LEFT SerialPort1.Write("l") ' the human is on LEFT SerialPort1.Write("l") ' the human is on LEFT SerialPort1.Write("l") ' the human is on LEFT SerialPort1.Write("l") ' the human is on RIGHTT SerialPort1.Write("l") ' the human is on LEFT SerialPort1.Write("l") ' the human is on LEFT SerialPort1.Write("l") ' the human is on LEFT SerialPort1.Write("l") ' the human is on LEFT SerialPort1.Write("l") ' the human is on LEFT SerialPort1.Write("d") ' the human is on LEFT SerialPort1.Close() End If If currentCenter.X > 0 And currentCenter.X < 250 Then SerialPort1.Open() Label5.Text = " Human is on left" ' i am sending so many "r" this way i can reduce the speed of motor 'and by sending less we can increase the speed. SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("d") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("r") ' the human is on LEFT SerialPort1.Write("d") ' the human is on LEFT SerialPort1.Close() End If ' new addition end If currentCenter.X > 250 And currentCenter.X < 350 Then ' HUMAN FACE IN CENTER SerialPort1.Open() Label5.Text = "Human is in Front" SerialPort1.Write("c") ' the human is on front SerialPort1.Close() End If End Sub Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load SerialPort1.Close() SerialPort1.PortName = "com5" SerialPort1.BaudRate = "9600" SerialPort1.DataBits = 8 SerialPort1.Parity = Parity.None SerialPort1.StopBits = StopBits.One SerialPort1.Handshake = Handshake.None SerialPort1.Encoding = System.Text.Encoding.Default End Sub End Class |
This is the same exact application used in Version1 of the Arduino image processing based entrance control system. This time I did a very little change, that is I find the X and Y co-ordinates which you can clearly see in the program. For the best Understanding watch Version1.