Arduino Projects

Arduino image processing Eye Pupil Tracking, how to make Haarcascade XML file

Eye Pupil Tracking, Image Processing based project Description:


eye pupil tracking– A few months back I uploaded a project on “Arduino Image Processing human face recognition and Entrance control using the electronic lock, and EmguCv. In this project, we successfully detected the human face using the frontalface XML file and controlled an electronic lock. The face recognition was possible due to the frontalface XML file, I recommend you should watch this tutorial first and then you can resume from here, as today we will be using the same project. You can create your own XML files and track almost anything you want.

In this tutorial, I will practically show you how to train an XML file for tracking the eye pupil, while the same steps can be used to train an XML file for any type of object.

In today’s episode, you will learn how to make your own XML file for eye pupil tracking. Today’s episode covers

  1. Steps to make your own XML file.
  2. How to use an XML file in programming.

For the complete step-by-step explanation, watch video Tutorial given at the end.

Amazon Links:

12v Adaptor:

Arduino Uno

Arduino Nano

Mega 2560:

Other Tools and Components:

Top Arduino Sensors:

Super Starter kit for Beginners

Digital Oscilloscopes

Variable Supply

Digital Multimeter

Soldering iron kits

PCB small portable drill machines

*Please Note: These are affiliate links. I may make a commission if you buy the components through these links. I would appreciate your support in this way!

Eye Pupil Tracking XML file training:

For creating the Haarcascade XML file you will need software.

As I want to track blue eyes, so that’s why I download pictures having blue eyes. If you want to track brown eyes or black eyes then simply download images consisting of brown or black color eyes. Or you can also take pictures of your own eye…

eye pupil tracking

The next step is to change the format of these images, as these images are of the type jpg, let’s convert these images to BMP format using adobe photoshop. This process is really simple, simply open the image and save as the image with a different name and select BMP as the format.

eye pupil tracking

We have a total of 6 images of the type bmp.

This is the software that I will be using form making the XML file.

eye pupil tracking

Open the cascades folder and delete all of these folders…

eye pupil tracking

Now open the negative folder, this folder consists of the negative images. Negative images are those images that do not consist of the object you want to track. You can also add more negative images of the type jpg…currently this folder have 200 negative images… The bg and create_list these two files are not included. Double click the create_list file.

eye pupil tracking

Now open the positive folder then open the raw data folder, simply delete these images

eye pupil tracking

And paste the images consisting of eyes, these are the positive images.

eye pupil tracking

now click on the object maker….now select the region you want to track…then simply press the space bar key on your keyboard and then press the enter key…repeat the same steps for all the images…

eye pupil tracking

eye pupil tracking

After you are done with all the images then Double click the samples creations file, which will make a file in the vector folder, which you can check. By Default the vector folder is empty.

eye pupil tracking

Right click on the haar training and select edit with notepad…the negative images are 200…while the positive images are 6. Update the number of positive images and negative images.

eye pupil tracking

Now double click on the haar training file and wait for a while, this can take some time.

Now the last step is to double click on the converted file to make the XML file…as you can see the XML file with the name myhaar is created…

eye pupil tracking

Rename this file with the name eyepupil…

eye pupil tracking

Now, this XML file is ready and can be used with Arduino, raspberry pi, you can use this XML file with emgucv, opencv, python, etc…

eye pupil tracking

This is the same application which I designed for the image processing based entrance control system…i have already explained this project in very detailed. You can watch this tutorial “Click here link2”.

The changes I made in this application can be watched in the video given below at the end of this article.

Eye Pupil Tracking Image Processing Programming:

For software changes, watch video given below.

Imports Emgu.CV
Imports Emgu.CV.Util

Imports Emgu.CV.Structure
Imports System. Diagnostics

Imports System.IO
Imports System.IO.Ports
Imports System.Threading

Public Class Form1

    Dim facedetected As Integer

    Dim facepresent As Integer

    Dim detectTimecount As Integer

    Dim web As Capture = New Capture(0)

    Private Sub Timer1_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer1.Tick

        ' cars()
    End Sub

    Sub faces()

        Dim photo As Image(Of Bgr, Byte)
        photo = web.RetrieveBgrFrame
        Dim currentCenter As New Point()

        ' Dim eyedetection As New CascadeClassifier("C:\data\haarcascades\haarcascade_eye.xml") '  ' this one is for eye detection
        Dim facedetection As New CascadeClassifier("C:\data\haarcascades\eyepupil.xml") ' this one is for the face detection

        facedetected = 0

            Dim image As Image(Of Gray, Byte) = photo.Convert(Of Gray, Byte)()

            For Each face As Rectangle In facedetection.DetectMultiScale(image, 1.15, 8, Size.Empty, Size.Empty) ' default 1.1, 8  ( while best values are 1.2 and 17 after checking)

                photo.Draw(face, New Bgr(Color.White), 4)

                currentCenter.X = CInt(CDbl(face.X + face.X + face.Width) / 2.0)
                currentCenter.Y = CInt(CDbl(face.Y + face.Y + face.Height) / 2.0)

                Label1.Text = (currentCenter.X).ToString
                Label2.Text = (currentCenter.Y).ToString

                Dim f = New MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_COMPLEX, 0.5, 0.5)
                photo.Draw("Human detected and scanning....", f, currentCenter, New Bgr(0, 255, 0)) ' New Point(10, 80) 

                Dim f2 = New MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_COMPLEX, 0.7, 0.7)
                photo.Draw("human detection ", f2, New Point(10, 20), New Bgr(0, 255, 255)) ' New Point(10, 80)

                Label3.Text = face.ToString

                facedetected = 1  ' we are storing 1 if faces are detected.


            PictureBox1.Image = photo.ToBitmap

        Catch ex As Exception

        End Try

        If facedetected = 1 Then
            lblfacedetect.Text = "Scanning..."
            Timer3.Enabled = True

            lblfacedetect.Text = "no Human..."
            Timer3.Enabled = False
            detectTimecount = 0
            facepresent = 0

            '  SerialPort1.Open()
            '  SerialPort1.Write("d")
            '  SerialPort1.Close()

        End If

        ' new addition end 

        If detectTimecount >= 3 Then  ' HUMAN FACE IN CENTER

            ' detectTimecount = 0
            Label4.Text = "Human detected"
            SerialPort1.Write("c") ' the human is on front

        End If

    End Sub

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        SerialPort1.PortName = "com15"
        SerialPort1.BaudRate = "9600"
        SerialPort1.DataBits = 8
        SerialPort1.Parity = Parity.None
        SerialPort1.StopBits = StopBits.One
        SerialPort1.Handshake = Handshake.None
        SerialPort1.Encoding = System.Text.Encoding.Default
    End Sub

    Private Sub Timer3_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer3.Tick
        detectTimecount = detectTimecount + 1
        Label4.Text = detectTimecount
    End Sub

    Private Sub Timer2_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer2.Tick
        SerialPort1.Write("d") ' no human
    End Sub

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        SerialPort1.Write("c") ' the human is on front
    End Sub

    Private Sub Button1_Click_1(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        SerialPort1.Write("c") ' the human is on front
    End Sub
End Class

Watch Video Tutorial:


Other Arduino Image Processing based Projects:

OCR Optical Character recognition using and Arduino

Arduino Image processing CCTV camera system using

Arduino Image Processing based Entrance lock Control System

Engr Fahad

My name is Shahzada Fahad and I am an Electrical Engineer. I have been doing Job in UAE as a site engineer in an Electrical Construction Company. Currently, I am running my own YouTube channel "Electronic Clinic", and managing this Website. My Hobbies are * Watching Movies * Music * Martial Arts * Photography * Travelling * Make Sketches and so on...


  1. Dear Engr Fahad,
    I enjoyed watching your video regarding eye pupil tracking. If you wouldn’t mind, I would like to speak with you about your interest and availability in creating a similar system for my use in an academic research environment. I have some specific objectives, but have no idea if you are at all interested or available to speak about this. At your convenience, sir, please contact me via email to advise. In advance of your reply, I thank you.

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button