Arduino Image processing CCTV camera system using VB.net

4,861

Arduino Image Processing CCTV Project Description:

In this Project “Arduino Image processing CCTV camera system”, you will learn how to make your own Arduino based CCTV camera system using image processing, Arduino Uno, and PIR sensor. In this tutorial vb.net “Visual Basic” will be used for making the image processing application. This application works together with the PIR sensor connected with Arduino Uno. Once the human is detected by the PIR sensor and also by the image processing application, then a picture is captured automatically and is saved in a folder.

This is the 2^nd version of the image processing based project, while in version 1 of the image processing I used the image processing for opening and closing the entrance using the electronic lock. If you want to read an Image processing based entranced controlled system?

For the image processing application designing Visual Basic .net will be used. Image processing application designing in vb.net is really simple. In this application, the Frontal face XML file is used for human recognition. Once the human face is detected by the computer application and a signal from the Arduino is also received, then a picture will be taken automatically and will be saved in the desired folder.

Amazon Links:

Arduino Nano USB-C Type (Recommended)

PIR Sensor Arduino:

WebCam:

Arduino Uno

*Disclosure: These are affiliate links. As an Amazon Associate I earn from qualifying purchases.

Arduino Ultrasonic Sensor interfacing, Circuit Diagram:

As you can see the circuit diagram is really easy, all you need is just to connect the PIR sensor with the Arduino Uno pin number 4, and the VCC and ground with the Arduino’s 5v and ground respectively.

Ultrasonic Sensor Arduino Programming:

Arduino Programming is very simple. I started by defining the pin for the PIR sensor. The PIR Sensor is connected with the Arduino’s Pin number 4. If you want you can use any other Pin. Then I defined another variable flag which is of the type integer. This variable is initially set to Zero. The variable flag will be used to stop the unnecessary repetition of code.

<span style="font-size: 14pt;">int pirsensor = 4; // pir sensor connected with 4
int flag = 0; 
</span>

int pirsensor = 4; // pir sensor connected with 4

int flag = 0;

the word void, when used before the function name, means that this function has no return type. If you look you will find the parenthesis are also empty, which means that this function is not taking any argument as the input. So the void setup function has no return type and it also does not take any argument as the input.

To active the Serial communication for this we use a function Serial.begin(), this is a built-in function and it takes one argument as the input, which is the baud rate. The PIR sensor is set to input using the pinMode function. the void setup function is executed only one time when the controller is turned ON.

<span style="font-size: 14pt;">void setup()
{
  Serial.begin(9600); 
  pinMode(pirsensor, INPUT); 
  digitalWrite(pirsensor, LOW); 
  Serial.println("hi how are you"); 
  
}
</span>

void setup()

{

Serial.begin(9600);

pinMode(pirsensor, INPUT);

digitalWrite(pirsensor, LOW);

Serial.println("hi how are you");

}

The void loop function runs infinite times until you stop it. The void loop function also has no return type and it does not take any argument as the input.

As you can see clearly the void loop function consists of only two if conditions. the purpose of these if conditions are to find out if the PIR sensor has detected in movement and send the desired message to the computer application using the Serial.print function. You might have noticed one thing the flag status is changed each time. This is just to stop the unnecessary repetition and to send the message only one time when the human is detected.

<span style="font-size: 14pt;">void loop()
{

  if( (digitalRead(pirsensor) == HIGH)&& (flag == 0))
  {
    Serial.print("intruder detected"); 
    flag = 1; 
  }
  if( (digitalRead(pirsensor) == LOW)&& (flag == 1))
  {
    Serial.println("no Intruder"); 
    flag = 0; 
  }
 


}
</span>

void loop()

{

if( (digitalRead(pirsensor) == HIGH)&& (flag == 0))

{

Serial.print("intruder detected");

flag = 1;

}

if( (digitalRead(pirsensor) == LOW)&& (flag == 1))

{

Serial.println("no Intruder");

flag = 0;

}

Computer Image Processing Application:

For the application designing and program explanation, you can watch a video given at the end of this Article.

<span style="font-size: 14pt;">Imports Emgu.CV
Imports Emgu.CV.Util

Imports Emgu.CV.Structure
Imports System.Diagnostics

Imports System.IO
Imports System.IO.Ports
Imports System.Threading


Public Class Form1
    Dim count As Integer = 0

    Dim facedetected As Integer
    Dim facepresent As Integer
    Dim web As Capture = New Capture(0) ' camera number


    Private Sub Timer1_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer1.Tick

        faces()
    End Sub


    Private Sub wait(ByVal interval As Integer)
        Dim sw As New Stopwatch
        sw.Start()
        Do While sw.ElapsedMilliseconds < interval
            ' Allows UI to remain responsive
            Application.DoEvents()
        Loop
        sw.Stop()
    End Sub




    Sub faces()
        Dim photo As Image(Of Bgr, Byte)

        photo = web.RetrieveBgrFrame

        Dim currentCenter As New Point()

        Dim facedetection As New CascadeClassifier("C:\data\haarcascades\haarcascade_frontalface_default.xml") ' this one is for the face detection

        facedetected = 0


        Try
            Dim image As Image(Of Gray, Byte) = photo.Convert(Of Gray, Byte)()

            For Each face As Rectangle In facedetection.DetectMultiScale(image, 1.1, 8, Size.Empty, Size.Empty) ' default 1.1, 8  ( while best values are 1.2 and 17 after checking)

                photo.Draw(face, New Bgr(Color.White), 4)

                currentCenter.X = CInt(CDbl(face.X + face.X + face.Width) / 2.0)
                currentCenter.Y = CInt(CDbl(face.Y + face.Y + face.Height) / 2.0)

                Label2.Text = (currentCenter.X).ToString
                Label3.Text = (currentCenter.Y).ToString

                Dim f = New MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_COMPLEX, 0.5, 0.5)
                photo.Draw("Human detected", f, currentCenter, New Bgr(0, 255, 0)) ' New Point(10, 80) 

                Dim f2 = New MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_COMPLEX, 0.7, 0.7)
                photo.Draw("human detectioin and tracking system", f2, New Point(10, 20), New Bgr(0, 255, 255)) ' New Point(10, 80)


                Label4.Text = face.ToString

                facedetected = 1  ' we are storing 1 if faces are detected.


            Next

            PictureBox1.Image = photo.ToBitmap


        Catch ex As Exception

        End Try


    End Sub



    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        SerialPort1.Close()
        SerialPort1.PortName = "com5"
        SerialPort1.BaudRate = "9600"
        SerialPort1.DataBits = 8
        SerialPort1.Parity = Parity.None
        SerialPort1.StopBits = StopBits.One
        SerialPort1.Handshake = Handshake.None
        SerialPort1.Encoding = System.Text.Encoding.Default
        SerialPort1.Open()

    End Sub




    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
    
        count = count + 1
        TextBox1.Text = count

        Dim photosave As Image(Of Bgr, Byte)
        photosave = web.RetrieveBgrFrame
        photosave.Save("E:\youtube tutorials\1 image processing security system\pictures\  fahad" + TextBox1.Text + ".jpg")

    End Sub

    Private Sub DataReceived(ByVal sender As Object, ByVal e As SerialDataReceivedEventArgs) Handles SerialPort1.DataReceived


        Try

            Dim mydata As String = ""
            mydata = SerialPort1.ReadExisting()

            If TextBox2.InvokeRequired Then
                TextBox2.Invoke(DirectCast(Sub() TextBox2.Text &= mydata, MethodInvoker))
            Else
                TextBox2.Text &= mydata
            End If
        Catch ex As Exception
            MessageBox.Show(ex.Message)
        End Try
    End Sub

 

    Private Sub Timer2_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer2.Tick
        If (InStr(TextBox2.Text, "intruder detected") And (facedetected = 1)) Then
            count = count + 1
            TextBox1.Text = count

            Dim photosave As Image(Of Bgr, Byte)
            photosave = web.RetrieveBgrFrame
            photosave.Save("E:\youtube tutorials\1 image processing security system\picture pir\  fahad" + TextBox1.Text + ".jpg")
            TextBox2.Text = ""
        End If
    End Sub
End Class
</span>

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

Imports Emgu.CV

Imports Emgu.CV.Util

Imports Emgu.CV.Structure

Imports System.Diagnostics

Imports System.IO

Imports System.IO.Ports

Imports System.Threading

Public Class Form1

Dim count As Integer = 0

Dim facedetected As Integer

Dim facepresent As Integer

Dim web As Capture = New Capture(0) ' camera number

Private Sub Timer1_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer1.Tick

faces()

End Sub

Private Sub wait(ByVal interval As Integer)

Dim sw As New Stopwatch

sw.Start()

Do While sw.ElapsedMilliseconds < interval

' Allows UI to remain responsive

Application.DoEvents()

Loop

sw.Stop()

End Sub

Sub faces()

Dim photo As Image(Of Bgr, Byte)

photo = web.RetrieveBgrFrame

Dim currentCenter As New Point()

Dim facedetection As New CascadeClassifier("C:\data\haarcascades\haarcascade_frontalface_default.xml") ' this one is for the face detection

facedetected = 0

Try

Dim image As Image(Of Gray, Byte) = photo.Convert(Of Gray, Byte)()

For Each face As Rectangle In facedetection.DetectMultiScale(image, 1.1, 8, Size.Empty, Size.Empty) ' default 1.1, 8 ( while best values are 1.2 and 17 after checking)

photo.Draw(face, New Bgr(Color.White), 4)

currentCenter.X = CInt(CDbl(face.X + face.X + face.Width) / 2.0)

currentCenter.Y = CInt(CDbl(face.Y + face.Y + face.Height) / 2.0)

Label2.Text = (currentCenter.X).ToString

Label3.Text = (currentCenter.Y).ToString

Dim f = New MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_COMPLEX, 0.5, 0.5)

photo.Draw("Human detected", f, currentCenter, New Bgr(0, 255, 0)) ' New Point(10, 80)

Dim f2 = New MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_COMPLEX, 0.7, 0.7)

photo.Draw("human detectioin and tracking system", f2, New Point(10, 20), New Bgr(0, 255, 255)) ' New Point(10, 80)

Label4.Text = face.ToString

facedetected = 1 ' we are storing 1 if faces are detected.

PictureBox1.Image = photo.ToBitmap

Catch ex As Exception

End Try

End Sub

Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

SerialPort1.Close()

SerialPort1.PortName = "com5"

SerialPort1.BaudRate = "9600"

SerialPort1.DataBits = 8

SerialPort1.Parity = Parity.None

SerialPort1.StopBits = StopBits.One

SerialPort1.Handshake = Handshake.None

SerialPort1.Encoding = System.Text.Encoding.Default

SerialPort1.Open()

End Sub

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click

count = count + 1

TextBox1.Text = count

Dim photosave As Image(Of Bgr, Byte)

photosave = web.RetrieveBgrFrame

photosave.Save("E:\youtube tutorials\1 image processing security system\pictures\ fahad" + TextBox1.Text + ".jpg")

End Sub

Private Sub DataReceived(ByVal sender As Object, ByVal e As SerialDataReceivedEventArgs) Handles SerialPort1.DataReceived

Try

Dim mydata As String = ""

mydata = SerialPort1.ReadExisting()

If TextBox2.InvokeRequired Then

TextBox2.Invoke(DirectCast(Sub() TextBox2.Text &= mydata, MethodInvoker))

Else

TextBox2.Text &= mydata

End If

Catch ex As Exception

MessageBox.Show(ex.Message)

End Try

End Sub

Private Sub Timer2_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer2.Tick

If (InStr(TextBox2.Text, "intruder detected") And (facedetected = 1)) Then

count = count + 1

TextBox1.Text = count

Dim photosave As Image(Of Bgr, Byte)

photosave = web.RetrieveBgrFrame

photosave.Save("E:\youtube tutorials\1 image processing security system\picture pir\ fahad" + TextBox1.Text + ".jpg")

TextBox2.Text = ""

End If

End Sub

End Class

Watch Video Tutorial:

Other Image Processing Projects:

Arduino image processing Eye Pupil Tracking, how to make Haarcascade XML file

OCR Optical Character Recognition using VB.net and Arduino

Arduino Image Processing based Entrance lock Control System

5 Comments

Lakshmi Vyshnavi says:

July 15, 2019 at 10:43 pm

Can we use this for detecting an ambulance??
Please do reply as soon as possible…Your response matters a lot to me since I am working on a project of detecting an ambulance

1. Engr Fahad says:
  
  July 16, 2019 at 12:30 pm
  
  Yes, of course, you can detect an Ambulance. Watch my Video on EyePupil Tracking, in which I have explained how to make an XML file to track anything.
  
N.Lakshmi Vyshnavi says:

July 16, 2019 at 2:01 pm

Thank you so much for your reply…I have the .xml file ready with me which i have trained earlier using Cascade GUI Trainer.I am little confused where to use that .xml file in your given code.Can you please help me out in sorting my problem?? Thank you..

Lakshmi Vyshnavi says:

July 16, 2019 at 2:04 pm

Thank you for your reply..I have an .xml file ready with me,which I have trained using Cascade GUI Trainer.I am little confused where to use that .xml file in your given code.Can you please help me out in sorting this issue???

Thank you..

1. Engr Fahad says:
  
  July 16, 2019 at 2:08 pm
  
  in my programming replace the front face XML with yours one. Watch my video on eye pupil tracking, in that video, I have explained this. watch that video completely and don’t skip any part. You will completely understand how to add your XML file in my program.