Research and Reviews - International Journals

International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering

Menu

ISSN ONLINE(2278-8875) PRINT (2320-3765)

All submissions of the EM system will be redirected to Online Manuscript Submission System. Authors are requested to submit articles directly to Online Manuscript Submission System of respective journal.

Human Eye Blink Detection using YCbCr Color Model, Haar-Like Features and Template Matching

Sukhwinder Kaur1, Hari Singh2
  1. M. Tech. Student, Dept. of ECE DAV Institute of Engineering and Technology, Jalandhar, India
  2. Assistant Professor, Dept. of ECE, DAV Institute of Engineering and Technology, Jalandhar, India
Related article at Pubmed, Scholar Google

Visit for more related articles at International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering

International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering

Abstract

This paper presents comparison of two image processing algorithms used for eye blink detection. The motivation of this research work is the need of disabled persons who are unable to move their body parts except eyes. The process of blink detection is divided into three parts viz.face localization, eye pair localization and template matching method. In method 1, YCbCr color model and morphological operations are used for the face and eyes localization. In method 2 face and eyes pair localization is performed by using Viola Jones method. After eye pair localization, the concept of template matching is applied for blink detection, in both the methods. A performance comparison is made for both the methods based upon detection accuracy and processing time. It is observed that method 1, gives better accuracy (80.75%) with low processing time (0.38sec.). The overall success rate of method 1 and method 2 is 71% and55% respectively.

Keywords
Face localization, Eye localization, Eye blink detection, YCbCr Color models, Viola Jones, Morphological operations and Template matching.

INTRODUCTION
Paralysis is the complete loss of muscle function for one or more muscle groups. Paralysis can cause loss of feeling or mobility in the affected areas. Paralysis is most often caused by damage to the nervous system, especially the spinal cord [2]. Fully paralysed patients require 24 hour support. But, it is not possible for anyone to be available at all times. In this paper an improved efficiency of eye blink is given. Before starting work on eye blink we have to go through from some steps as shown in fig.1. There are some techniques devised for blink detection as well. This research work is to improve the efficiency of eye blink detection rate. Before doing work on eye blink we have to go through a survey as explained below.

LITRATURE SURVEY
Ijaz Khan, Hadi Abdullah et al [2013]: In this paper the author presents improved algorithms for face, eyes and mouth detection in an image. Viola Jones and skin color pixel detection as face detection techniques are widely used. Viola Jones gives accurate face detection but consumes more time whereas skin color pixel detection technique consumes less time but lacks in accuracy.
Atish Udayashankar et al [2012]: The main aim of this paper is to design a real time interactive system that can assist the paralyzed to control appliances such as lights, fans etc. or by playing pre-recorded audio messages, through a predefined number of eye blinks. Image processing techniques have been implemented in order to detect the eye blinks.
K Takahashi et al [2012]: This paper proposed a practical method for eye blink detection using a monocular system. Eye blink detection is an important technology in many situations such as facial action analysis and signal processing. However, automatic eye blink detection is quite difficult since the eye blink is occurred fast
Kohei Aai and Ronny Mardiyanto et al [2011]: In this paper blink detection is used for a variety of applications such as Human-Computer Interaction, wearable computing, etc. Blink detection method with Gabor filter is proposed to realize high accurate blink detection. The blink detection accuracy is evaluated by several people and compared to the other existing conventional methods. Through the comparison, it is found that the proposed blink detection method with Gabor filter is superior to the other methods.
image
A. Human Face Localization
For human face localization Viola Jones and skin color pixel detection as face detection techniques are widely used. Viola Jones gives accurate face detection but consumes more time whereas skin color pixel detection technique consumes less time but lacks in accuracy. In this research both the techniques which are to be used will increases accuracy while consuming less time. Viola Jones and other methods can accurately detect faces but in case of facial features detection their accuracy decreases [1, 6, 14, 16].
B. Eye Localization
The localization of eyes in facial images has many applications in computer vision such as gaze estimation, pose estimation, face detection, face recognition, human computer interaction, eye blink detection etc. Eyes can be extracted from facial images using different eye features such as color, illumination, shape, geometry, edges etc.[10,12,14].Once two eye blocks appear from the segmented image and spread to a certain size; they will be detected by the determination criterion of eye region[3,19]. However, sometimes, it is found that in the detection process the two detected eyes might not be real eyes, furthermore, it is difficult to seek precise centre of eye [31].
C. Eye Blink Detection
As the users eye closes during the process of a blink, its similarity to the open eye template decreases. Likewise, it regains its similarity to the template as the blink ends and the user’s eye becomes fully open again. This decrease and increase in similarity corresponds directly to the correlation scores returned by the template matching procedure [1, 3, 17].

PROBLEM FORMULATION
Need of Proposed Research Work
The main need of eye blink detection system is to help those people who are unable to move any part of their body excepting their eyes. Fully paralysed patients require 24 hour support. But in present day this is not possible for anyone to be available full time. There are some techniques devised for blink detection as well [2]. The detection of blinking and analysis of blink detection will support the people who are not able to move their body parts.
Significance of Proposed Research Work
Significance of this research work is to improve detection time of eye blink as we know that Human face image analysis, detection and recognition have become some of the most important research topics in the field of image processing and pattern classification. The potential applications involve topics such as face detection, face identification and recognition, and facial expression analysis. Among these research topics, one fundamental but very important problem to be solved is automatic eye detection. The eye is the most significant and important feature in a human face, as extraction of the eyes are often easier as compared to other facial features.

METHODOLOGY/PLANNING OF WORK
Two methods are used for eye blink detection in this research. In both the methods different techniques are applied to localize human face and eye and a common method (Template Matching) is used for eye blink detection. Finally, processing time and accuracy of both the methods is compared. Because, processing time is very important parameter when eye blink detection is to be implemented in real-time (in future). Steps we followed in both the methods are as:
Method 1
a) Human Face Localization and Eye Localization using YCbCr Color Model.
b) Template Matching for Blink Detection.
Method 2
a) Haar - like features for Face Localization and Eye Localization.
b) Template Matching for Blink Detection.

YCbCr COLOR MODEL
YCbCr is a family of color spaces used as a part of the color image pipeline in video and digital photography systems. Y is the luma component and CB and CR are the blue-difference and red difference chrome components. Y (with prime) is distinguished from Y which is luminance, meaning that light intensity is non-linearly encoded using gamma correction. YCbCr color model wedge the digital image into two parts, luma or luminance component and chrome or chrominance component. Due to luminance and chrominance effect, it becomes most important in digital video camera to handle video information [9, 23, 25] The YCbCr color space is widely used for digital video. In this paper, luminance information is stored as a single component (Y), and chrominance information is stored as two colordifference components (Cb and Cr). Cb represents the difference between the blue component and a reference value Cr represents the difference between the red component and a reference value.

VIOLA-JONES METHOD
The Viola Jones object detection framework proposed in by Paul Viola and Michael Jones is the first object detection framework to provide competitive object detection rates in real-time .Viola Jones and skin color pixel detection as face detection techniques are widely used.[1,21,27] They proposed a face detection method based on the adaboost learning algorithm using haar features that detected the face successfully with high accuracy.[26] Viola Jones algorithm are:
A. Integral images
B. Haar Features
C. Cascade of boosted classifier
A. Integral images
The simple rectangular features of an image are calculated using an intermediate representation of an image, called the integral image [9]. The integral image is an array containing the sums of the pixels’ intensity values located directly to the left of a pixel and directly above the pixel at location (x, y) inclusive. So if A[x, y] is the original image and AI[x, y] is the integral image.
B. Haar Classifier
The core basis for Haar classifier object detection is the Haar-like features. These features, rather than using the intensity values of a pixel, use the change in contrast values between adjacent rectangular groups of pixels. The contrast variances between the pixel groups are used to determine relative light and dark areas. Two or three adjacent groups with a relative contrast variance form a Haar-like feature. Haar-like features, as shown in Fig.3 are used to detect an image [8, 16, 27]. Haar features can easily be scaled by increasing or decreasing the size of the pixel group being examined. This allows features to be used to detect objects of various sizes.
image
C. Cascade of boosted classifier
The overall form of the detection process is that of a degenerate decision tree, what we call a “cascade”. A positive result from the first classifier triggers the evaluation of a second classifier which has also been adjusted to achieve very high detection rates. A positive result from the second classifier triggers a third classifier, and so on Stages in the cascade are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize false negatives as shown in fig.4 has also been adjusted to achieve very high detection rates. A positive result from the second classifier triggers a third classifier, and so on Stages in the cascade are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize false negatives as shown in fig.4.A series of classifiers are applied to every sub-window. The initial classifier eliminates a large number of negative examples with very little processing. Subsequent layers eliminate additional negatives but require additional computation. After several stages of processing the numbers of sub-windows have been reduced radically.
image

MORPHOLOGICAL OPERATIONS
Morphological techniques are used for boundary detection. Dilation, followed by erosion and the calculation of differences between the two produces an image with boundaries. The structuring element used in dilation and erosion has a large matrix, so that clear and thick boundaries are detected [5, 25].

TEMPLATE MATCHING
For eye blink detection template matching is used here.In the template matching method, segments of an input image are compared to previously stored images, to evaluate the similarity of the counterpart using correlation values[2, 5].
Diff; Image
D = (Tc-Ts)
s = sum (sum (d));
Tc = current frame
Ts = stored frame
*Threshold is set by empirical method.
Frame differencing is applied to find correlation between current template and stored template. Hence the value of ‘s’ less than threshold value indicates template matching.
Method 1
(a) Human Face and Eye Localization using YCbCr Color Model: In this part following steps are to be followed:
A. I/P frame
B. Convert RGB to YCbCr
C. Morphological operations
D. Face localization
E. Eye pair localization
F. Eye blink detection(Template Matching Method)
A. I/P frame: This is the very first step in which an input frame are to be given to the YCbCr Color Model for the purpose of whole procedure.
B. Conversion RGB to YCbCr: As we know that RGB frame is acquired for the process but we have to convert RGB frame to YCbCr because this transformation is useful in detecting blobs and reduces the computational complexity.
C. Morphological operations
Dilation: The value of the output pixel is the maximum value of all the pixels in the input pixel's neighbourhood. In a binary image, if any of the pixels is set to the value 1, the output pixel is set to 1.
Erosion: The value of the output pixel is the minimum value of all the pixels in the input pixel's neighbourhood. In a binary image, if any of the pixels is set to 0, the output pixel is set to 0.
Structuring Elements: An essential part of the dilation and erosion operations is the structuring element used to probe the input image.
D. Face localization: After applying morphological operation on an input frame at the end face is detected as shown.
E. Eye pair localization: This is the next step i.e. eye pair detection and edge detection is involved in it. Morphological techniques are used for boundary detection. Dilation, followed by erosion and the calculation of differences between the two produces an image with boundaries. Diamond structuring element is used.
image
Template matching is a technique for finding areas of frame that match (are similar) to a template frame (patch) it has two primary components:
Source frame (I): The frame in which we expect to find a match to the template frame as shown below:
Template frame (T): The patch frame which will be compared to the template frame as shown below:
image
Method 2
(a) Haar - like features for Face Localization and Eye Localization using Viola Jones Algorithm.
A. I/P frame
B. Face localization
C. Eye pair localization
D. Eye blink detection (Template Matching Method)
image
Template matching is a technique for finding areas of frame that match (are similar) to a template frame (patch) it has two primary components:
Source frame (I): The frame in which we expect to find a match to the template frame as shown below:
Template frame (T): The patch frame which will be compared to the template frame as shown
image

RESULTS AND DISCUSSION
Detection Accuracy (DA) =TP/TP+FN…………. (1)
False Alarm Rate (FAR) =FP/TP+FP……………… (2)
Success Rate (%) =DA/DA+FAR…… (3)
TP: Correctly detected blinks
FP: False detection of blinks
FN: Missed blink
image
Average success rate: 71%
Average Accuracy: 80.75%
Processing Time (Per blink): 0.38 sec.
image
Average success rate: 55%
Average Accuracy: 60.37%
Processing Time (Per blink): 0.65 sec
image

CONCLUSION
Two different methods are implemented in this research for Blink Detection, which follows face and eye pair localization in a frame and template matching to check the status of eyes(opened or closed).In Method 1, face and eyes pair localization is performed usingYCbCr skin color model and morphological operations. Then template matching is applied for blink detection. In Method 2, face and eyes localization is performed using Viola Jones Computer-Vision Toolbox and template matching for blink detection. From result analysis, it is observed that, average detection accuracy and processing time for Method 1 and 2 are 80.75%, 60.37% and 0.38 sec., 0.65sec., respectively. Therefore, it can be concluded that the accuracy of Method 1 is better as compared to Method 2 and method one require less processing time too. The fall in accuracy is due to inability of the method to detect face and eye pair area, especially is case of tilted faces. Further, the overall success rate is 71% and 55% for method one and two, respectively. The fall in the success rate is due to large no. of “False Positives” i.e. false detection of blinks. Finally, it can be concluded that, Method 1 is more suitable choice for blink detection for subjects with tilted faces too.

References