Real-Time License Plate Recognition using Raspberry Pi and Python

Real-Time License Plate Recognition using Raspberry Pi and Python

Previously we learned about face recognition using Raspberry Pi and OpenCV. Today we will use these two to build a number plate recognition system using python. Real-Time license plate detection and recognition can be very useful for automating toll booths, finding out traffic rule breakers, and for addressing other vehicle-related security and safety issues.

 

So in this tutorial, we are going to build a Real-Time License Plate Recognition System using Raspberry Pi and Pi Camera. This system automatically recognizes and reads vehicle license plates using OpenCV and Optical Character Recognition. Pi camera module continuously captures the frames, and when a key is pressed on the keyboard, it saves the last frame as a new image. Then it uses the contour function from OpenCV to detect the license plate. And finally, Raspberry Pi crops out that particular area and perform optical character recognition to read the license plate numbers.

 

Components Required

  • Raspberry Pi
  • Pi Camera

 

Here only Raspberry Pi and Pi camera are used to build this Raspberry Pi Plate Recognition System. We previously used Pi camera with Raspberry pi, and built few projects using it like Web Controlled Raspberry Pi Surveillance Robot, IoT based Smart Wi-Fi doorbell, Smart CCTV Surveillance System, etc.

Real-Time License Plate Recognition using Raspberry Pi

To learn more about how to interface Pi camera with Raspberry Pi, follow our previous tutorial.

 

Pre-requisites for Number Plate Recognition OpenCV Python

Here we use the OpenCV library to detect and recognize number plates, and the Tesseract library is used to read the characters. So before proceeding further, first install the OpenCV, Tesseract, and other required libraries. We previously used OpenCV in Face Recognition using the Raspberry Pi project.

 

Installing OpenCV in Raspberry Pi 3

Here OpenCV library is used to detect and recognize faces. To install the OpenCV, first, update the Raspberry Pi.

sudo apt-get update

 

Then use the following commands to install the required dependencies for installing OpenCV on your Raspberry Pi.

sudo apt-get install libhdf5-dev -y 
sudo apt-get install libhdf5-serial-dev –y 
sudo apt-get install libatlas-base-dev –y 
sudo apt-get install libjasper-dev -y
sudo apt-get install libqtgui4 –y
sudo apt-get install libqt4-test –y

 

After that, use the below command to install the OpenCV on your Raspberry Pi.

pip3 install opencv-contrib-python==4.1.0.25

 

Installing Tesseract 

To install the Tesseract, first, configure the Debian Package (dpkg) using the below command:

sudo dpkg - -configure –a

 

After that, install the Tesseract OCR (Optical Character Recognition) using the apt-get option.

sudo apt-get install tesseract-ocr

 

After that, install the pytesseract using the pip.

pip install pytesseract

 

After this, install the PYTTSX3 library for text to speech conversion using the below command:

pip install pyttsx3

 

Installing imutils 

imutils is used to make essential image processing functions such as translation, rotation, resizing, skeletonization, and displaying Matplotlib images easier with OpenCV. Use the below command to install the imutils:

pip3 install imutils

 

SMTP Mail Setup for Raspberry Pi Car Plate Recognition

SMTP (Simple Mail Transfer Protocol) is the standard protocol for providing email services on a TCP/IP network. This server provides the ability to receive and send email messages. We are using SMTP to send a mail when the Raspberry Pi detects and recognizes a license plate.

 

To use the SMTP services on Raspberry Pi, we first have to install the SMTP library on Pi. Use the below command to install the same:

sudo apt-get install ssmtp

 

Now we have to edit the configure file. For that, open Configuration file using the below command:

sudo nano /etc/ssmtp/ssmtp.conf

 

And add the below lines. Don’t forget to add your email and email password.

root=postman
mailhub=smpt.gmail.com:587
AuthUser=youremail@gmail.com
AuthPass=youremailpassword
FromLineOverride=YES
UseSTARTTLS=YES

SMTP Mail Setup for Raspberry Pi

 

Python Programming for Number Plate Recognition using Raspberry Pi

License plate recognition OpenCV python code involves three major steps. The first step is License Plate Detection. The contour function will be used to detect the rectangular objects in the image to find the number plate. The second step is Character Segmentation. Once Contour detects the License Plate, we have to crop it out and save it as a new image. And the final step is Character Recognition. We will perform Optical Character Recognition on the cropped image to detect the number.

 

Complete Number Plate Recognition code in python is given at the end of the page. Here we are explaining the code.

 

At the starting of the code, import the required packages for this project.

import cv2
import imutils
import numpy as np
import pytesseract
from PIL import Image
from picamera.array import PiRGBArray
from picamera import PiCamera
import smtplib

 

Then initialize the camera object and set the resolution at (640, 480) and the frame rate at 30 fps.

camera = PiCamera()
camera.resolution = (640, 480)
camera.framerate = 30

 

Then use the capture_continuous function to start capturing the frames from the Raspberry Pi camera. We are using the keyboard key ‘S’ to capture a particular frame.

for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
        image = frame.array
        cv2.imshow("Frame", image)
        key = cv2.waitKey(1) & 0xFF
        rawCapture.truncate(0)

 

When the keyboard key is pressed, it will take the last frame. After capturing the last frame, it will use a bilateral filter function to remove the unwanted details from the captured image.

if key == ord("s"):
             gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) #convert to grey scale
             gray = cv2.bilateralFilter(gray, 11, 17, 17) #Blur to reduce noise

 

After removing the unwanted details, use the Canny Edge Method to perform edge detection.

edged = cv2.Canny(gray, 30, 200)

 

Now we will look for contours on our image. After that sort the detected contours from big to small.

cnts = cv2.findContours(edged.copy(), cv2.RETR_TREE,              cv2.CHAIN_APPROX_SIMPLE)
             cnts = imutils.grab_contours(cnts)
             cnts = sorted(cnts, key = cv2.contourArea, reverse = True)[:10]
             screenCnt = None

 

The Raspberry Pi can find multiple contours, so we have to filter the license plate contour by searching for a rectangle shape contour with four sides and a closed figure among the obtained results.

   for c in cnts:
                peri = cv2.arcLength(c, True)
                approx = cv2.approxPolyDP(c, 0.018 * peri, True)
                if len(approx) == 4:
                  screenCnt = approx
                  break

Number Plate Recognition

 

After finding the license plate, mask everything except the license plate

mask = np.zeros(gray.shape,np.uint8)
new_image = cv2.drawContours(mask,[screenCnt],0,255,-1,)
new_image = cv2.bitwise_and(image,image,mask=mask)

 

After masking the entire image except the license plate region, we will crop out the license plate area and save it as a new image.

(x, y) = np.where(mask == 255)
             (topx, topy) = (np.min(x), np.min(y))
             (bottomx, bottomy) = (np.max(x), np.max(y))
             Cropped = gray[topx:bottomx+1, topy:bottomy+1]

Real-Time License Plate

 

Now in the final step, read the characters from the license plate image using the Tesseract library and store the recognized characters in the ‘text’ variable. As mentioned earlier, we are also sending a mail when Pi detects a license plate. So the license plate information will be sent to the mentioned email address. The code for sending the mail is given below.

text = pytesseract.image_to_string(Cropped, config='--psm 11')
print("Detected Number is:",text)
server.sendmail("Receiver Mail1","Receiver Mail2",text)

Number Plate Recognition using Raspberry PI

Real-Time License Plate Recognition using Raspberry Pi

Raspberry Pi Number Plate Recognition Working

This is how you can build a Number Plate Recognition using Python and Raspberry Pi. Results from this project will not be 100% accurate. Sometimes it fails to detect the license plate, or sometimes it reads some garbage values. Results largely depend on image quality and image orientation.

 

A working video and complete python code are given below.

Code

import cv2
import imutils
import numpy as np
import pytesseract
from PIL import Image
from picamera.array import PiRGBArray
from picamera import PiCamera
import smtplib
server=smtplib.SMTP('smtp.gmail.com',587)
server.starttls()
server.login("Your Email ID", "Email ID Password")
camera = PiCamera()
camera.resolution = (640, 480)
camera.framerate = 30
rawCapture = PiRGBArray(camera, size=(640, 480))
for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
        image = frame.array
        cv2.imshow("Frame", image)
        key = cv2.waitKey(1) & 0xFF
        rawCapture.truncate(0)
        if key == ord("s"):
             gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) #convert to grey scale
             gray = cv2.bilateralFilter(gray, 11, 17, 17) #Blur to reduce noise
             edged = cv2.Canny(gray, 30, 200) #Perform Edge detection
             cnts = cv2.findContours(edged.copy(), cv2.RETR_TREE,              cv2.CHAIN_APPROX_SIMPLE)
             cnts = imutils.grab_contours(cnts)
             cnts = sorted(cnts, key = cv2.contourArea, reverse = True)[:10]
             screenCnt = None
             for c in cnts:
                peri = cv2.arcLength(c, True)
                approx = cv2.approxPolyDP(c, 0.018 * peri, True)
                if len(approx) == 4:
                  screenCnt = approx
                  break
             if screenCnt is None:
               detected = 0
               print ("No contour detected")
             else:
               detected = 1
             if detected == 1:
               cv2.drawContours(image, [screenCnt], -1, (0, 255, 0), 3)
             mask = np.zeros(gray.shape,np.uint8)
             new_image = cv2.drawContours(mask,[screenCnt],0,255,-1,)
             new_image = cv2.bitwise_and(image,image,mask=mask)
             (x, y) = np.where(mask == 255)
             (topx, topy) = (np.min(x), np.min(y))
             (bottomx, bottomy) = (np.max(x), np.max(y))
             Cropped = gray[topx:bottomx+1, topy:bottomy+1]
             text = pytesseract.image_to_string(Cropped, config='--psm 11')
             print("Detected Number is:",text)
             server.sendmail("Sender's Email ID@gmail.com","Sender's Email ID@gmail.com",text)
             cv2.imshow("Frame", image)
             cv2.imshow('Cropped',Cropped)
             cv2.waitKey(0)
             break
cv2.destroyAllWindows()

Video

21 Comments

Hi, I'm trying for the last 10 hours or so to get your .py working using a uncompressed PNG file stored locally on the SD-Card by a external app. Is there a easy way to get from
filepath = "/home/pi/DCIM/example.png"
to
image = frame.array
? The for loop catching those frames continuously is obviously not needed, because there is only this one. The same goes for the press key thingy further down the lines, no Problem there. But how do I get a PNG file in a shape which cv2.cvtColor() can work with? I could use JPG as well, but I guess this is not a good idea due to quality loss and stuff like that.

Thanks alot for writing this Article, it is really educational.

sorry for the double posting, but there seems to be missing something in the command for

To install the Tesseract, first, configure the Debian Package (dpkg) using the below command:

for the rest, well done. It is always good to see a pro at work.

Jean-sebastien…

1 December 2020

Hello,

Nice work.
Your code is strongly dependant of "picamera" and doesn't allow the use of 3rd party USB camera. Is there any option to use an other USB Camera (or using other library ?)

Kr

Js

Kind regards

Js

Hello,

Nice work i have managed to get it work fine, thank you very much for sharing. Are you still working on improving this as it would be good to get it continuously streaming with an auto capture function.

Hi,

Thanks for the above tutorial. I installed on a fresh Raspberry OS build and had the below problem. I updated .bashrc with the pytesseract location and still found the same problem. Help! (please)

pi@raspberrypi:~ $ python3 numberplate.py
Traceback (most recent call last):
File "numberplate.py", line 4, in <module>
import pytesseract
ModuleNotFoundError: No module named 'pytesseract'

cv2.imshow("Frame", image)
cv2.error: OpenCV(4.5.1) /tmp/pip-wheel-qd18ncao/opencv-python/opencv/modules/highgui/src/window_gtk.cpp:624: error: (-2:Unspecified error) Can't initialize GTK backend in function 'cvInitSystem'

I am having error with installing "E: Unable to locate package \u2013y" why? see commands below:

pi@raspberrypi:~ $ sudo apt-get install libqtgui4 \u2013y
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package \u2013y

Nice project, I'm working on a similar project but with some enhancements like gate control system.
But I'm using a USB Webcam not the pi camera, how can I use the Camera object with a usb webcam or how can I use the video capture method?

Thanks in advance.

great proect
I copied everything , took the same image of the car
but I get this resualt:
you license plate is 7LK-10888
how could it be?

what could be wrong ?

Thanks,

new_image = cv2.drawContours(mask,[screenCnt],0,255,-1,)
(-215:Assertion failed) reader.ptr != NULL in function 'cvDrawContours'
I found this error after run your code. how i can solve this error. Please help me

Is there a Way to make it not send email but just make it add to database with time stamp..
And dods it work on a pi-zero

can anyone guide me that i need to run this code infinitely like code runs in while loop.. so i can detect images without typing "s" again again....