Journal of Global Research in Computer Sciences
MenuISSN: 2229-371X
ISSN: 2229-371X
| Rajneesh Singla* and Sahil Bansal Computer Engineering Section, Yadavindra College of Engineering, Talwandi Sabo |
| Corresponding Author: Rajneesh Singla, E-mail: eng.singla85@gmail.com |
| Related article at Pubmed, Scholar Google |
Visit for more related articles at Journal of Global Research in Computer Sciences
Natural facial expressions commonly occur in social interactions between people, and are useful for providing an emotional context for the interaction, and for communicating social intentions. This paper depicts an idea regarding detecting an unknown human face from input imagery and recognise his/her current mood. The objective of this paper is that psychological state giving information about some disorders helpful with diagnosis of depression, mania or schizophrenia. The elimination of errors due to reflections in the image has not been implemented but the algorithms used in this paper are computationally efficient to resolve errors. In this paper we have accepted five different moods to be recognized are: Joy, Fear, Contempt, Sad, Disgust and Astonished. Principal Component Analysis (PCA) is implemented with Fisher face Algorithm to recognize different moods.The main part of this paper is an emotional database which will contain images of faces, their corresponding Action Units and their labels. The contribution of this database to the problem stated above is that it can be used by systems in order to recognize emotional facial expressions given one of the database data i.e. action units’ combination
Keywords |
| Feature Extraction, Facial Expression Detection, Principal Component analysis (PCA), Fisher face Algorithm, |
INTRODUCTION |
| A human-computer interface of the future will have methods both for recognizing emotional context from facial expression, and for communicating emotions in an understandable way. Of course, such an interface will surely make choices based on many factors, but we posit that facial expressions can provide a great deal of useful information, and we propose to focus our study on this type of observation. Furthermore, our work will focus primarily on the recognition problem, although we believe that a strong emotional communication method is a necessity for this type of interaction. That is, a computer system which expresses emotions will be more likely to elicit strong emotional expression from its user. Therefore, an expression component can actually make the recognition task much simpler.. Our research will focus primarily on the mood detection and recognising different facial expressions. Facial expression is a visible manifestation of the affective state, cognitive activity, intention, personality, and psychopathology of a person ; it plays a communicative role in interpersonal relations. Facial expressions, and other gestures, convey non-verbal communication cues in face-to-face interactions. These cues may also complement speech by helping the listener to elicit the intended meaning of spoken words. The facial expressions have a considerable effect on a listening interlocutor; the facial expression of a speaker accounts for about 55 percent of the effect, 38 percent of the latter is conveyed by voice intonation and 7 percent by the spoken words. As a consequence of the information that they carry, facial expressions can play an important role wherever humans interact with machines. Automatic recognition of facial expressions may act as a component of natural human machine interfaces (some variants of which are called perceptual interfaces or conversational interfaces). Such interfaces would enable the automated provision of services that require a good appreciation of the emotional state of the service user, as would be the case in transactions that involve negotiation, for example. some robots can also benefit from the ability to recognise expressions [3][2]. Automated analysis of facial expressions for behavioural science or medicine is another possible application domain. In everyday life such prototypic expressions occur relatively infrequently.Instead,emotion more often is communicated by subtle changes in one or a few discrete facial features, such as tightening of lips in anger or obliquely lowering the lips corner in sadness[7].Change in isolated ,especially in the area of the areas of eyebrows or eyelids, is typical of paralinguistic displays; for instance raising the brows signals greeting[11].To capture such subtlety of human emotion and paralinguistic ,automated fine grained facial expressions is needed. |
LITERATURE REVIEW |
| Lee et.al [8] approached a method of expression-invariant face recognition that transforms input face image with an arbitrary expression into its corresponding neutral facial expression image. To achieve expression-invariance, first extract the facial feature vector from the input image using AAM. Next, transform the input facial feature vector into its corresponding neutral facial expression vector using direct or indirect facial expression transformation. Finally, perform the expression-invariant face recognition by distance-based matching techniques nearest neighbour classifier, linear discriminant analysis (LDA) and generalized discriminant analysis (GDA). Mandeep Kaur et. al. [10] paper presents a new idea for detecting an unknown human face in input imagery and recognizing his/her facial expression . the objective of this project is to implement highly intelligent machines or robots that are mind implemented. Fasel et. al. [7] fulfills the recognition of facial action units, i.e., the subtle change of facial expressions, and emotion-specified expressions. The optimum facial feature extraction algorithm, Canny Edge Detector, is applied to localize face images, and a hierarchical clustering-based scheme reinforces the search region of extracted highly textured facial clusters. Peter et. al. [8] proposed a method is based on Fisher’s Linear Discriminant and produces well separated classes in a low-dimensional subspace, even under severe variation in lighting and facial expressions. The Eigen face technique, another method based on linearly projecting the image space to a low dimensional subspace, has similar computational requirements. Yet, extensive experimental results demonstrate that the proposed “Fisher face” method has error rates that are lower than those of the Eigen face technique for tests on the Harvard and Yale Face Databases. Bartlett et. al. [5] explores and compares techniques for automatically recognizing facial actions in sequences of images. These techniques include analysis of facial motion through estimation of optical flow; holistic spatial analysis, such as independent component analysis, local feature analysis, and linear discriminant analysis; and methods based on the outputs of local filters, such as Gabor wavelet representations and local principal components. |
FACIAL EXPRESSION DATABASE |
| The Database used in my research paper for facial mood detection is Real Time Database. This Database contains( 48 images of which 7 facial expressions including neutral images). The Database contains different images of an individual which represents different moods according to different situations. For the implementation the Database contains 48 coloured face images of individual. There are 4 images per subject, and these 4 images are respectively, under the following different facial expressions or configuration. In this implementation, all images are resized to a uniform dimension of 256 x 256. Following Figure shows the database images considered for face Expression recognition could be used. |
 |
DIMENSION REDUCTION TECHNIQUES |
| In statistics, dimension reduction is the process of reducing the number of random variables under consideration, and can be divided into feature selection and feature extraction. |
| Feature Selection |
| Feature selection approaches try to find a subset of the original variables (also called features or attributes). Two strategies are filter (e.g. information gain) and wrapper (e.g. search guided by the accuracy) approaches. See also combinatorial optimization problems. In some cases, data analysis such as regression or classification can be done in the reduced space more accurately than in the original space. It is mainly considered as first step in face recognition process. |
| Feature Extraction |
| Feature extraction transforms the data in the high-dimensional space to a space of fewer dimensions. The data transformation may be linear, as in principal component analysis (PCA), but many nonlinear dimensionality reduction techniques also exist. The main linear technique for dimensionality reduction, principal component analysis, performs a linear mapping of the data to a lower dimensional space in such a way that the variance of the data in the low-dimensional representation is maximized. In practice, the correlation matrix of the data is constructed and the eigenvectors on this matrix are computed. The eigenvectors that correspond to the largest eigenvalues (the principal components) can now be used to reconstruct a large fraction of the variance of the original data. Moreover, the first few eigenvectors can often be interpreted in terms of the large-scale physical behavior of the system. The original space (with dimension of the number of points) has been reduced (with data loss, but hopefully retaining the most important variance) to the space spanned by a few eigenvectors. Principal component analysis can be employed in a nonlinear way by means of the kernel trick. The resulting technique is capable of constructing nonlinear mappings that maximize the variance in the data. The resulting technique is entitled Kernel PCA. Other prominent nonlinear techniques include manifold learning techniques such as locally linear embedding (LLE), Hessian LLE, Laplacian eigenmaps, and LTSA. These techniques construct a low-dimensional data representation using a cost function that retains local properties of the data, and can be viewed as defining a graph-based kernel for Kernel PCA. More recently, techniques have been proposed that, instead of defining a fixed kernel, try to learn the kernel using semi definite programming. The most prominent example of such a technique is maximum variance unfolding (MVU). The central idea of MVU is to exactly preserve all pair wise distances between nearest neighbors (in the inner product space), while maximizing the distances between points that are not nearest neighbors. An alternative approach to neighborhood preservation is through the minimization of a cost function that measures differences between distances in the input and output spaces. Important examples of such techniques include classical multidimensional scaling (which is identical to PCA), Isomap (which uses geodesic distances in the data space), diffusion maps (which uses diffusion distances in the data space), t-SNE (which minimizes the divergence between distributions over pairs of points), and curvilinear component analysis. A different approach to nonlinear dimensionality reduction is through the use of auto encoders, a special kind of feed-forward neural networks with a bottle-neck hidden layer. The training of deep encoders is typically performed using a greedy layer-wise pre-training (e.g., using a stack of Restricted Boltzmann machines) that is followed by a finetuning stage based on back propagation. Principal Component Analysis: Principal component analysis (PCA) is a mathematical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of uncorrelated variables called principal components. The number of principal components is less than or equal to the number of original variables. This transformation is defined in such a way that the first principal component has as high a variance as possible (that is, accounts for as much of the variability in the data as possible), and each succeeding component in turn has the highest variance possible under the constraint that it be orthogonal to (uncorrelated with) the preceding components. Principal components are guaranteed to be independent only if the data set is jointly normally distributed. PCA is sensitive to the relative scaling of the original variables. Depending on the field of application, it is also named the discrete Karhunen–Loève transform (KLT), the Hostelling transform or proper orthogonal decomposition (POD). PCA was invented in 1901 by Karl Pearson. Now it is mostly used as a tool in exploratory data analysis and for making predictive models. PCA can be done by eigenvalue decomposition of a data covariance matrix or singular value decomposition of a data matrix, usually after mean centering the data for each attribute. The results of a PCA are usually discussed in terms of component scores (the transformed variable values corresponding to a particular case in the data) and loadings (the weight by which each standardized original variable should be multiplied to get the component score). PCA is the simplest of the true eigenvector-based multivariate analyses. Often, its operation can be thought of as revealing the internal structure of the data in a way which best explains the variance in the data. If a multivariate dataset is visualized as a set of coordinates in a high-dimensional data space (1 axis per variable), PCA can supply the user with a lower-dimensional picture, a "shadow" of this object when viewed from its (in some sense) most informative viewpoint. This is done by using only the first few principal components so that the dimensionality of the transformed data is reduced. PCA is closely related to factor analysis; indeed, some statistical packages (such as Stata) deliberately conflate the two techniques. True factor analysis makes different assumptions about the underlying structure and solves eigenvectors of a slightly different matrix. Fisherface Algorithm: Fisher's linear discriminant are methods used in statistics, pattern recognition and machine learning to find a linear combination of features which characterize or separate two or more classes of objects or events. The resulting combination may be used as a linear classifier, or, more commonly, for dimensionality reduction before later classification. In the other two methods however, the dependent variable is a numerical quantity, while for LDA it is a categorical variable (i.e. the class label). Logistic regression and probit regression are more similar to LDA, as they also explain a categorical variable. These other methods are preferable in applications where it is not reasonable to assume that the independent variables are normally distributed, which is a fundamental assumption of the LDA method. |
| LDA is also closely related to principal component analysis (PCA) and factor analysis in that both look for linear combinations of variables which best explain the data]. LDA explicitly attempts to model the difference between the classes of data. PCA on the other hand does not take into account any difference in class, and factor analysis builds the feature combinations based on differences rather than similarities. Discriminant analysis is also different from factor analysis in that it is not an interdependence technique: a distinction between independent variables and dependent variables (also called criterion variables) must be made. LDA works when the measurements made on independent variables for each observation are continuous quantities. When dealing with categorical independent variables, the equivalent technique is discriminant correspondence analysis. |
EXPERIMENT |
| We have experimented on The Real Time Database. Database contains ( 48 images of which 7 facial expressions including neutral images). The Database contains different images of an individual which represents different moods according to different situations. For the implementation the Database contains 48 coloured face images of individual. There are 4 images per subject, and these 4 images are respectively, under the following different facial expressions or configuration. In this implementation, all images are resized to a uniform dimension of 256 x 256. Vigorous experimentation is done by selecting proper number of epochs, number of runs, step size on randomize data set to generalize the problem. Input image forms the first state for the face recognition module. To this module a face image is passed as an input for the system. The input image samples are considered of non-uniform illumination effects, variable facial expressions, and face image with glasses. In second phase of operation the face image passed is transformed to operational compatible format, where the face Image is resized to uniform dimension; the data type of the image sample is transformed to double precision and passed for Feature extraction. In Feature extraction unit runs both Fisher face and PCA algorithms for the computations of face for extraction. These features are passed to classifier which calculates the minimum Euclidean distance from the neutral image and the image having minimum distance is selected for output. For the implementation of the proposed recognition architecture the database samples are trained for the knowledge creation for classification. During training phase when a new facial image is added to the system the features are calculated and aligned for the dataset formation. Comparing the weights of the test face with the known weights of the database is found by calculating the norm of the differences between the test and known set of weights, such that a minimum difference between any pair would symbolize the closest match. |
| Methodology |
| The methodology used to detect different facial moods is described as follows:- |
| Step 1: Two folders were created |
| (1) Training images |
| (2) Input image |
| Step 2: Created Loop through the training images for reading images data in T- matrix. (Preprocessing) |
| Step 3: Read the definition of moods/face expressions for each image and index them in matrix.(Knowledge). |
| Step 4: Calculated the eigenfaces. |
| Step 5: Calculated the fisherfaces.(Feature extraction) |
| Step 6: Identified the neutral images.(Euclidean distance is calculated) |
| Step 7: Used the classifier to identify image close to input image. |
| Step 8: Recognized the image with expressions |
RESULTS AND DISCUSSION |
| The optimally design PCA and Fisher face algorithms tested on the training set. The results obtained are brilliant. The recognition rate for all five principal moods namely Sad, Contempt, Fear, Joy, Disgust and Astonished along with Neutral is obtained which is more than previous existing techniques. Finally the network is tested on real time dataset with excellent recognition rate. The finding that there was a significant main effect for emotion is not surprising. In this way, these results support past research indicating that some emotions are recognized more readily than others. In the current study, Joy was the most accurately identified emotion, followed by anger, sadness, surprise, disgust, and finally fear. The finding that both groups performed poorly at recognizing fear is surprising, as this emotion is often rated as a more primary emotion, and the percentage correct for 86%. |
 |
 |
 |
CONCLUSIONS |
| The aim of this research paper was to explore the area of facial moods. A wide variety of image processing techniques was developed to meet the facial expression recognition system requirements. However, there are still many challenges and problems to solve in such systems, especially in the area of their performance and applicability improvement. In this research paper we proposed PCA and Fisher face methods for dimension reduction of different types of facial moods. The proposed algorithm is successfully implemented on Real time database. Experiments results show that algorithm can effectively recognize different facial moods by indentifying different features. |
References |
|