ESP32 Cam, Python OpenCV, Yolov3, & Arduino based Car Parking Barrier/Gate Control

6,252

Table of Contents

ESP32 Cam Car Parking Barrier Control:

ESP32 Cam, Python OpenCV, Yolov3, & Arduino based Car Parking Barrier/Gate Control- Last year, I created a video on a car barrier control system, in which I used an ultrasonic sensor to control the car barrier/gate. That project is suitable for beginners. However, in reality, that project is not practical at all. This is because an ultrasonic sensor can only sense the presence of objects and measure their distance, but it cannot identify those objects. Therefore, if any object comes in front of the ultrasonic sensor, it would open the car barrier/gate, which is not ideal.

So, I thought, why not take that project to a slightly advanced level? So, this time, I will be using the Arduino along with ESP32 Cam for wireless live video streaming, and for the image processing, I am going to use Python OpenCV YoloV3. Now, the barrier/gate will only open for cars. It will not open for any other objects.

This project is entirely based on my previous project, where I explained the most basic things, such as:

How to perform wireless video streaming using the ESP32 Cam module?

How to install Python, OpenCV, and YoloV3? and

How to detect and identify different objects?

In my studio, I detected and identified various objects, and not only did I identify and track birds and cats, but I also displayed alert messages on the screen.

So, I have already explained all of these things, and I won’t repeat them today. Today, I will only explain new things. Including,

Servo motor interfacing with Arduino and its programming.

How to detect and identify a car? And

How to send car barrier/gate opening and closing commands to Arduino from Python OpenCV YoloV3. So, without any further delay, let’s get started!

Amazon Links:

Arduino Nano USB C type (Recommended)

ESP32 Camera Module

ESP32 CAM W-BT Board

MSI Intel Core i7 Laptop check this out.

Disclosure: These are affiliate links. As an Amazon Associate I earn from qualifying purchases.

I am using my Arduino Nano and LoRa based development board. But you can do the same exact connections on a breadboard. Anyway, connect the signal wire to the Arduino pin9, and connect the VCC and GND wires of the Servo to the Arduino 5V and GND pins. You can follow this circuit diagram.

Servo with Arduino, Circuit Diagram:

And let me also tell you. If you are planning on using large servo motors like the ones you can see on the screen.

Then don’t use 5V from the Arduino because it might damage your Arduino board. So, use an external 5V power supply.

On my development board, I have this 5V and 3A power supply and its more than enough for powering these all types of servos. So, you can make yourself this development board, or you can make yourself this 5V and 3A power supply and then you can use it with different microcontroller boards, servos, breakout boards, and even you can use this 5V and 3A power supply to charge your cell phone.

Now, let’s go ahead and take a look at the Arduino programming.

Car Barrier/Gate Control Arduino Programming:

/*
 ESP32 Cam, Python OpenCV Yolo V3, & Arduino based Car Parking Barrier/Gate control system Programming
 https://www.electroniclinic.com/
 */

#include <Servo.h>

Servo servoMotor;
int servoPin = 9;  // Modify this with the appropriate pin number
int Sflag = 0;


void setup() {
  servoMotor.attach(servoPin);
  Serial.begin(9600);
  servoMotor.write(0);  // Move servo to 0 degrees initially
  delay(1000);  // Delay for 1 second
}

void loop() 
{
  if (Serial.available() > 0) 
  {
    int angle = Serial.parseInt();

          if((angle==1) && (Sflag == 0)) 
          {
      servoMotor.write(90);// Move servo to 90 degrees
     Sflag = 1;
      delay(2000);
           }
     
         if((angle==2) && (Sflag == 1))
         {
            servoMotor.write(0);
      Sflag = 0;
      delay(2000);
           }
  }
}

ESP32 Cam, Python OpenCV Yolo V3, & Arduino based Car Parking Barrier/Gate control system Programming

https://www.electroniclinic.com/

#include <Servo.h>

Servo servoMotor;

int servoPin = 9; // Modify this with the appropriate pin number

int Sflag = 0;

void setup() {

servoMotor.attach(servoPin);

Serial.begin(9600);

servoMotor.write(0); // Move servo to 0 degrees initially

delay(1000); // Delay for 1 second

}

void loop()

{

if (Serial.available() > 0)

{

int angle = Serial.parseInt();

if((angle==1) && (Sflag == 0))

{

servoMotor.write(90);// Move servo to 90 degrees

Sflag = 1;

delay(2000);

}

if((angle==2) && (Sflag == 1))

{

servoMotor.write(0);

Sflag = 0;

delay(2000);

}

I started off by adding the Servo.h header file. You can see the Servo signal wire is connected to the Arduino pin 9. The Sflag variable I am going to use as the flag, I could also define it as Boolean but, anyway, I am using it to stop the unnecessary repetition of code.

In the void loop() function, we simply check if any data is available on the Serial port, or in simple words if any data is received from the Python. In my case, I send two numbers from python. So, that’s why I have used parseInt().

So, if the received number is 1 and the Sflag == 0 then move the Servo arm to 90⁰and change the Sflag state to 1. So, these instructions will only execute once.

If the received number is 2 and the Sflag == 1 then move the servo arm to 0⁰.

I have already uploaded this program. And now let’s take a look at the Python OpenCV YoloV3 programming.

Python OpenCV YoloV3 programming:

import cv2
import numpy as np
import urllib.request
import serial

url = 'http://192.168.43.219/cam-hi.jpg'

arduino_port = 'COM11'  # Modify this with the appropriate serial port
baud_rate = 9600  # Modify this with the appropriate baud rate

cap = cv2.VideoCapture(url)
whT = 320
confThreshold = 0.5
nmsThreshold = 0.3
classesfile = 'coco.names'
classNames = []

with open(classesfile, 'rt') as f:
    classNames = f.read().rstrip('\n').split('\n')

carClassId = classNames.index('car')  # Get the class ID for car

modelConfig = 'yolov3.cfg'
modelWeights = 'yolov3.weights'
net = cv2.dnn.readNetFromDarknet(modelConfig, modelWeights)
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_OPENCV)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU)

door_opened = False
door_closed = False
flag=0
flag1=0
Ccounter=0 #for counting cars

line1_x1, line1_y1, line1_x2, line1_y2 = 200, 0, 200, 600  # Coordinates for line 1 (x1, y1, x2, y2)
line2_x1, line2_y1, line2_x2, line2_y2 = 600, 0, 600, 600  # Coordinates for line 2 (x1, y1, x2, y2)

# Establish serial connection with Arduino Nano
arduino = serial.Serial(arduino_port, baud_rate, timeout=1)

def send_servo_command(angle):
    command = f"{angle}\n"  # Append newline character to the command
    arduino.write(command.encode('utf-8'))


def findObject(outputs, img):
    global door_opened, door_closed,flag, Ccounter# Declare the variables as global

    hT, wT, cT = img.shape
    bbox = []
    classIds = []
    confs = []

    for output in outputs:
        for det in output:
            scores = det[5:]
            classId = np.argmax(scores)
            confidence = scores[classId]

            if classId == carClassId and confidence > confThreshold:
                w, h = int(det[2] * wT), int(det[3] * hT)
                x, y = int((det[0] * wT) - w / 2), int((det[1] * hT) - h / 2)
                bbox.append([x, y, w, h])
                classIds.append(classId)
                confs.append(float(confidence))

             

    indices = cv2.dnn.NMSBoxes(bbox, confs, confThreshold, nmsThreshold)

    for i in indices:
        i = i[0]
        box = bbox[i]
        x, y, w, h = box[0], box[1], box[2], box[3]
        center_x = x + w // 2
        
        center_y = y + h // 2
        cv2.rectangle(img, (x, y), (x + w, y + h), (255, 0, 255), 2)
        cv2.putText(img, f'{classNames[classIds[i]].upper()} {int(confs[i] * 100)}%', (x, y - 10),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.6, (255, 0, 255), 2)
        cv2.circle(img, (center_x, center_y), 10, (0, 255, 0), -1)  # Draw a dot at the center of the bounding box
        
        if flag == 0 and center_x > line1_x1 and center_x < line2_x1:
            door_opened = True
            flag=1
            print("Door opened")
            send_servo_command(1)  # Send 90 degree command to servo
            
                    
        if flag==1 and center_x > line2_x1:
            door_closed = True
            flag=0
            print("Door closed")
            send_servo_command(2)  # Send 0 degree command to servo
             
        

    # Draw line 1
    cv2.line(img, (line1_x1, line1_y1), (line1_x2, line1_y2), (0, 255, 0), 2)

    # Draw line 2
    cv2.line(img, (line2_x1, line2_y1), (line2_x2, line2_y2), (0, 0, 255), 2)


while True:
    img_resp = urllib.request.urlopen(url)
    imgnp = np.array(bytearray(img_resp.read()), dtype=np.uint8)
    im = cv2.imdecode(imgnp, -1)
    success, img = cap.read()
    blob = cv2.dnn.blobFromImage(im, 1 / 255, (whT, whT), [0, 0, 0], 1, crop=False)
    net.setInput(blob)
    layerNames = net.getLayerNames()
    outputNames = [layerNames[i[0] - 1] for i in net.getUnconnectedOutLayers()]
    outputs = net.forward(outputNames)
    findObject(outputs, im)
    cv2.imshow('Image', im)

    if door_opened and door_closed:
        door_opened = False
        door_closed = False
        
    if cv2.waitKey(1) == ord('q'):
        break

cv2.destroyAllWindows()

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

import cv2

import numpy as np

import urllib.request

import serial

url = 'http://192.168.43.219/cam-hi.jpg'

arduino_port = 'COM11' # Modify this with the appropriate serial port

baud_rate = 9600 # Modify this with the appropriate baud rate

cap = cv2.VideoCapture(url)

whT = 320

confThreshold = 0.5

nmsThreshold = 0.3

classesfile = 'coco.names'

classNames = []

with open(classesfile, 'rt') as f:

classNames = f.read().rstrip('\n').split('\n')

carClassId = classNames.index('car') # Get the class ID for car

modelConfig = 'yolov3.cfg'

modelWeights = 'yolov3.weights'

net = cv2.dnn.readNetFromDarknet(modelConfig, modelWeights)

net.setPreferableBackend(cv2.dnn.DNN_BACKEND_OPENCV)

net.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU)

door_opened = False

door_closed = False

flag=0

flag1=0

Ccounter=0 #for counting cars

line1_x1, line1_y1, line1_x2, line1_y2 = 200, 0, 200, 600 # Coordinates for line 1 (x1, y1, x2, y2)

line2_x1, line2_y1, line2_x2, line2_y2 = 600, 0, 600, 600 # Coordinates for line 2 (x1, y1, x2, y2)

# Establish serial connection with Arduino Nano

arduino = serial.Serial(arduino_port, baud_rate, timeout=1)

def send_servo_command(angle):

command = f"{angle}\n" # Append newline character to the command

arduino.write(command.encode('utf-8'))

def findObject(outputs, img):

global door_opened, door_closed,flag, Ccounter# Declare the variables as global

hT, wT, cT = img.shape

bbox = []

classIds = []

confs = []

for output in outputs:

for det in output:

scores = det[5:]

classId = np.argmax(scores)

confidence = scores[classId]

if classId == carClassId and confidence > confThreshold:

w, h = int(det[2] * wT), int(det[3] * hT)

x, y = int((det[0] * wT) - w / 2), int((det[1] * hT) - h / 2)

bbox.append([x, y, w, h])

classIds.append(classId)

confs.append(float(confidence))

indices = cv2.dnn.NMSBoxes(bbox, confs, confThreshold, nmsThreshold)

for i in indices:

i = i[0]

box = bbox[i]

x, y, w, h = box[0], box[1], box[2], box[3]

center_x = x + w // 2

center_y = y + h // 2

cv2.rectangle(img, (x, y), (x + w, y + h), (255, 0, 255), 2)

cv2.putText(img, f'{classNames[classIds[i]].upper()} {int(confs[i] * 100)}%', (x, y - 10),

cv2.FONT_HERSHEY_SIMPLEX, 0.6, (255, 0, 255), 2)

cv2.circle(img, (center_x, center_y), 10, (0, 255, 0), -1) # Draw a dot at the center of the bounding box

if flag == 0 and center_x > line1_x1 and center_x < line2_x1:

door_opened = True

flag=1

print("Door opened")

send_servo_command(1) # Send 90 degree command to servo

if flag==1 and center_x > line2_x1:

door_closed = True

flag=0

print("Door closed")

send_servo_command(2) # Send 0 degree command to servo

# Draw line 1

cv2.line(img, (line1_x1, line1_y1), (line1_x2, line1_y2), (0, 255, 0), 2)

# Draw line 2

cv2.line(img, (line2_x1, line2_y1), (line2_x2, line2_y2), (0, 0, 255), 2)

while True:

img_resp = urllib.request.urlopen(url)

imgnp = np.array(bytearray(img_resp.read()), dtype=np.uint8)

im = cv2.imdecode(imgnp, -1)

success, img = cap.read()

blob = cv2.dnn.blobFromImage(im, 1 / 255, (whT, whT), [0, 0, 0], 1, crop=False)

net.setInput(blob)

layerNames = net.getLayerNames()

outputNames = [layerNames[i[0] - 1] for i in net.getUnconnectedOutLayers()]

outputs = net.forward(outputNames)

findObject(outputs, im)

cv2.imshow('Image', im)

if door_opened and door_closed:

door_opened = False

door_closed = False

if cv2.waitKey(1) == ord('q'):

break

cv2.destroyAllWindows()

This is the same exact program from my previous project. You can see I am using the same IP Address.

url = 'http://192.168.43.219/cam-hi.jpg'

1	url = 'http://192.168.43.219/cam-hi.jpg'

Through this IP address, I receive high-quality images from the ESP32 Camera module. I have already explained, how to setup your ESP32 camera module for the live video streaming.

arduino_port = 'COM11'

1	arduino_port = 'COM11'

This is the Arduino_port. My Arduino board is connected to the communication port11 and 9600 is the baud_rate.

This time round, I am using coco.names for the detection and identification of a car. If you open the coco.names file, you will see a long list of the objects. So, you can select any of those objects for controlling the Car barrier/Gate. In my case I am going to continue with a car.

line1_x1, line1_y1, line1_x2, line1_y2 = 200, 0, 200, 600 

line2_x1, line2_y1, line2_x2, line2_y2 = 600, 0, 600, 600

These are the co-ordinates of two lines used to open and close the Car barrier.

        if flag == 0 and center_x > line1_x1 and center_x < line2_x1:

            door_opened = True

            flag=1

            print("Door opened")

            send_servo_command(1)  # Send 90 degree command to servo

        if flag==1 and center_x > line2_x1:

            door_closed = True

            flag=0

            print("Door closed")

            send_servo_command(2)  # Send 0 degree command to servo

line1_x1, line1_y1, line1_x2, line1_y2 = 200, 0, 200, 600

line2_x1, line2_y1, line2_x2, line2_y2 = 600, 0, 600, 600

These are the co-ordinates of two lines used to open and close the Car barrier.

if flag == 0 and center_x > line1_x1 and center_x < line2_x1:

door_opened = True

flag=1

print("Door opened")

send_servo_command(1) # Send 90 degree command to servo

if flag==1 and center_x > line2_x1:

door_closed = True

flag=0

print("Door closed")

send_servo_command(2) # Send 0 degree command to servo

These are the instructions used to send the commands 1 and 2. When the car crosses line1, it sends 1 to the Arduino to open the Barrier and when the Car crosses line2 then it sends 2 to the Arduino to close the Barrier.

Now, let’s go ahead and start our practical demonstration.