Arduino image processing Eye Pupil Tracking, how to make Haarcascade XML file
Table of Contents
Eye Pupil Tracking, Image Processing based project Description:
eye pupil tracking– A few months back I uploaded a project on “Arduino Image Processing human face recognition and Entrance control using the electronic lock, vb.net and EmguCv“. In this project, we successfully detected the human face using the frontalface XML file and controlled an electronic lock. The face recognition was possible due to the frontalface XML file, I recommend you should watch this tutorial first and then you can resume from here, as today we will be using the same project. You can create your own XML files and track almost anything you want.
In this tutorial, I will practically show you how to train an XML file for tracking the eye pupil, while the same steps can be used to train an XML file for any type of object.
In today’s episode, you will learn how to make your own XML file for eye pupil tracking. Today’s episode covers
- Steps to make your own XML file.
- How to use an XML file in vb.net programming.
For the complete step-by-step explanation, watch video Tutorial given at the end.
Amazon Links:
Arduino Nano USB-C Type (Recommended)
*Disclosure: These are affiliate links. As an Amazon Associate I earn from qualifying purchases.
Eye Pupil Tracking XML file training:
For creating the Haarcascade XML file you will need software.
As I want to track blue eyes, so that’s why I download pictures having blue eyes. If you want to track brown eyes or black eyes then simply download images consisting of brown or black color eyes. Or you can also take pictures of your own eye…
The next step is to change the format of these images, as these images are of the type jpg, let’s convert these images to BMP format using adobe photoshop. This process is really simple, simply open the image and save as the image with a different name and select BMP as the format.
We have a total of 6 images of the type bmp.
This is the software that I will be using form making the XML file.
Open the cascades folder and delete all of these folders…
Now open the negative folder, this folder consists of the negative images. Negative images are those images that do not consist of the object you want to track. You can also add more negative images of the type jpg…currently this folder have 200 negative images… The bg and create_list these two files are not included. Double click the create_list file.
Now open the positive folder then open the raw data folder, simply delete these images
And paste the images consisting of eyes, these are the positive images.
now click on the object maker….now select the region you want to track…then simply press the space bar key on your keyboard and then press the enter key…repeat the same steps for all the images…
After you are done with all the images then Double click the samples creations file, which will make a file in the vector folder, which you can check. By Default the vector folder is empty.
Right click on the haar training and select edit with notepad…the negative images are 200…while the positive images are 6. Update the number of positive images and negative images.
Now double click on the haar training file and wait for a while, this can take some time.
Now the last step is to double click on the converted file to make the XML file…as you can see the XML file with the name myhaar is created…
Rename this file with the name eyepupil…
Now, this XML file is ready and can be used with Arduino, raspberry pi, you can use this XML file with emgucv, opencv, python, etc…
This is the same application which I designed for the image processing based entrance control system…i have already explained this project in very detailed. You can watch this tutorial “Click here link2”.
The changes I made in this application can be watched in the video given below at the end of this article.
Eye Pupil Tracking Image Processing Programming:
For software changes, watch video given below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 |
Imports Emgu.CV Imports Emgu.CV.Util Imports Emgu.CV.Structure Imports System. Diagnostics Imports System.IO Imports System.IO.Ports Imports System.Threading Public Class Form1 Dim facedetected As Integer Dim facepresent As Integer Dim detectTimecount As Integer Dim web As Capture = New Capture(0) Private Sub Timer1_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer1.Tick ' cars() faces() End Sub Sub faces() Dim photo As Image(Of Bgr, Byte) photo = web.RetrieveBgrFrame Dim currentCenter As New Point() ' Dim eyedetection As New CascadeClassifier("C:\data\haarcascades\haarcascade_eye.xml") ' ' this one is for eye detection Dim facedetection As New CascadeClassifier("C:\data\haarcascades\eyepupil.xml") ' this one is for the face detection facedetected = 0 Try Dim image As Image(Of Gray, Byte) = photo.Convert(Of Gray, Byte)() For Each face As Rectangle In facedetection.DetectMultiScale(image, 1.15, 8, Size.Empty, Size.Empty) ' default 1.1, 8 ( while best values are 1.2 and 17 after checking) photo.Draw(face, New Bgr(Color.White), 4) currentCenter.X = CInt(CDbl(face.X + face.X + face.Width) / 2.0) currentCenter.Y = CInt(CDbl(face.Y + face.Y + face.Height) / 2.0) Label1.Text = (currentCenter.X).ToString Label2.Text = (currentCenter.Y).ToString Dim f = New MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_COMPLEX, 0.5, 0.5) photo.Draw("Human detected and scanning....", f, currentCenter, New Bgr(0, 255, 0)) ' New Point(10, 80) Dim f2 = New MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_COMPLEX, 0.7, 0.7) photo.Draw("human detection ", f2, New Point(10, 20), New Bgr(0, 255, 255)) ' New Point(10, 80) Label3.Text = face.ToString facedetected = 1 ' we are storing 1 if faces are detected. Next PictureBox1.Image = photo.ToBitmap Catch ex As Exception End Try If facedetected = 1 Then lblfacedetect.Text = "Scanning..." Timer3.Enabled = True Else lblfacedetect.Text = "no Human..." Timer3.Enabled = False detectTimecount = 0 facepresent = 0 ' SerialPort1.Open() ' SerialPort1.Write("d") ' SerialPort1.Close() End If ' new addition end If detectTimecount >= 3 Then ' HUMAN FACE IN CENTER ' detectTimecount = 0 SerialPort1.Open() Label4.Text = "Human detected" SerialPort1.Write("c") ' the human is on front SerialPort1.Close() End If End Sub Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load SerialPort1.Close() SerialPort1.PortName = "com15" SerialPort1.BaudRate = "9600" SerialPort1.DataBits = 8 SerialPort1.Parity = Parity.None SerialPort1.StopBits = StopBits.One SerialPort1.Handshake = Handshake.None SerialPort1.Encoding = System.Text.Encoding.Default End Sub Private Sub Timer3_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer3.Tick detectTimecount = detectTimecount + 1 Label4.Text = detectTimecount End Sub Private Sub Timer2_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer2.Tick SerialPort1.Open() SerialPort1.Write("d") ' no human SerialPort1.Close() End Sub Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click SerialPort1.Open() SerialPort1.Write("c") ' the human is on front SerialPort1.Close() End Sub Private Sub Button1_Click_1(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click SerialPort1.Open() SerialPort1.Write("c") ' the human is on front SerialPort1.Close() End Sub End Class |
Watch Video Tutorial:
Other Arduino Image Processing based Projects:
OCR Optical Character recognition using VB.net and Arduino
Dear Engr Fahad,
I enjoyed watching your video regarding eye pupil tracking. If you wouldn’t mind, I would like to speak with you about your interest and availability in creating a similar system for my use in an academic research environment. I have some specific objectives, but have no idea if you are at all interested or available to speak about this. At your convenience, sir, please contact me via email to advise. In advance of your reply, I thank you.
-David
here is my WhatsApp number
+923339218213