International Journal of Innovative Research in Science, Engineering and Technology
MenuISSN ONLINE(2319-8753)PRINT(2347-6710)
ISSN ONLINE(2319-8753)PRINT(2347-6710)
| Miss. Leena Mahajan, Prof. G.A.kulkarni Department of E&C Engineering S.S.G.B.C.O.E&T, Bhusawal N.M.U, Jalgaon India. |
| Related article at Pubmed, Scholar Google |
Visit for more related articles at International Journal of Innovative Research in Science, Engineering and Technology
In this review paper we are going to discuss a systematic trajectory recognition algorithm framework that can construct effective classifiers for hand writing & gesture identification. Review of Digital Pen for Handwritten Digit and Gesture Recognition using Trajectory Recognition Algorithm based on Accelerometer is discuss for the identification of 2-D handwriting digits and 3-D hand gestures. for this implementation we are going to use triaxial accelerometer, a microcontroller, and an RF wireless transmission module for sensing and collecting accelerations of handwriting and gesture trajectories. The proposed trajectory recognition algorithm composes of the procedures of acceleration acquisition, signal reprocessing, feature generation, feature selection, and feature extraction. The algorithm is capable of translating time-series acceleration signals into important feature vectors. Users can use the pen to write digits or make hand gestures, and the accelerations of hand motions measured by the accelerometer are wirelessly transmitted to a computer for online trajectory recognition. Basically this algorithm is divided in two parts. First part is discussion about the implemented for the Handwritten Digit Recognition while in second part is discussion about the implemented for the Gesture Recognition.
Keywords |
| Human–Computer Interaction (HCI), Micro Inertial Measurement Unit (μIMU), Optical Tracking System (OTS), Hidden Markov Model (HMM),linear discriminate analysis (LDA), probabilistic neural network (PNN), Ubiquitous Digital Writing Instrument (UDWI). |
INTRODUCTION |
| Volatile growth of efficient technologies in electronic circuits and components has greatly decreased the dimension and weight of consumer electronic products, such as smart phones and handheld computers, and thus made them more handy and convenient. Due to the rapid development of computer technology, human–computer interaction (HCI) techniques [1]–[3] have become an indispensable component in our daily life. Recently, an attractive alternative, a portable device embedded with inertial sensors, has been proposed to sense the activities of human and to capture his/her motion trajectory information from accelerations for recognizing gestures or handwriting. A significant advantage of inertial sensors for general motion sensing is that they can be operated without any external reference and limitation in working conditions. However, motion trajectory recognition is relatively complicated because different users have different speeds and styles to generate various motion trajectories. Thus, many researchers have tried to narrow down the problem domain for increasing the accuracy of handwriting recognition systems. Recently, some researchers have concentrated on reducing the error of handwriting trajectory reconstruction by manipulating acceleration signals and angular velocities of inertial sensors [4]–[8]. However, the reconstructed trajectories suffer from various intrinsic errors of inertial sensors. Hence, many researchers have focused on developing effective algorithms for error compensation of inertial sensors to improve the recognition accuracy. To name a few, Yang et al. [9] proposed a pen type input device to track trajectories in 3-D space by using accelerometers and gyroscopes. An efficient acceleration error compensation algorithm based on zero velocity compensation was developed to reduce acceleration errors for acquiring accurate reconstructed trajectory. An extended Kalman filter with magnetometers (micro inertial measurement unit (μIMU) with magnetometers), proposed by Luo et al. [10], was employed to compensate the orientation of the proposed digital writing instrument. If the orientation of the instrument was estimated precisely, the motion trajectories of the instrument were reconstructed accurately. Dong et al. [11] proposed an optical tracking calibration method based on optical tracking system (OTS) to calibrate 3-D accelerations, angular velocities, and space attitude of handwriting motions. The OTS was developed for the following two goals: 1) to obtain accelerations of the proposed ubiquitous digital writing instrument (UDWI) by calibrating 2-D trajectories and 2) to obtain the accurate attitude angles by using the multiple camera calibration. However, in order to recognize or reconstruct motion trajectories accurately, the aforementioned approaches introduce other sensors such as gyroscopes or magnetometers to obtain precise orientation. This increases additional cost for motion trajectory recognition systems as well as computational burden of their algorithms. In order to reduce the cost of systems and simplify the algorithms, much research effort has been devoted to extract important features from time-series inertial signals. To name a few, Lim et al. [12] computed correlation coefficients of the absolute value of acceleration and the absolute value of the first and second derivatives of acceleration to form feature vectors. They then applied principal component analysis (PCA) and Fisher linear discriminates to reduce the dimension of the feature vectors. With the reduced features, a time-lagged feed forward network was trained to recognize 2-D handwriting gestures and the best performance with an overall accuracy of 95%. In [13], the acceleration, velocity, and position features were generated from raw acceleration signals, and then, the PCA was utilized to reduce the feature dimension size. They successfully employed a hidden Markov model (HMM) with dynamic time warping algorithms to recognize 3-D handwriting digits with a recognition rate of 90%. Krishnan et al. [14] calculated the time- and frequency-domain features (such as mean, variance (VAR), correlation, spectral entropy, and spectral energy) of the acceleration signals measured from the accelerometers worn simultaneously on different positions of a participant’s hand. Subsequently, the AdaBoost, HMM, and k-NN classifiers were utilized to classify hand motions, and the AdaBoost classifier achieved the best performance with an overall accuracy of 86%. |
| In this paper, we developed a pen-type portable device and a trajectory recognition algorithm. The pen-type portable device consists of a triaxial accelerometer, a microcontroller, and an RF wireless transmission module. The acceleration signals measured from the triaxial accelerometer are transmitted to a computer via the wireless module. Users can utilize this digital pen to write digits and make hand gestures at normal speed. The measured acceleration signals of these motions can be recognized by the trajectory recognition algorithm. The recognition procedure is composed of acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction. The acceleration signals of hand motions are measured by the pen-type portable device. The signal preprocessing procedure consists of calibration, a moving average filter, a high-pass filter, and normalization. First, the accelerations are calibrated to remove drift errors and offsets from the raw signals. These two filters are applied to remove high frequency noise and gravitational acceleration from the raw data, respectively. The features of the preprocessed acceleration signals of each axis include mean, correlation among axes, interquartile range (IQR), mean absolute deviation (MAD), root mean square (rms), VAR, standard deviation (STD), and energy. Before classifying the hand motion trajectories, we perform the procedures of feature selection and extraction methods. In general, feature selection aims at selecting a subset of size m from an original set of d features (d > m). Therefore, the criterion of kernel-based class reparability (KBCS) with best individual N (BIN) is to select significant features from the original features (i.e., to pick up some important features from d) and that of linear discriminate analysis (LDA) is to reduce the dimension of the feature space with a better recognition performance (i.e., to reduce the size of m). The objective of the feature selection and feature extraction methods is not only to ease the burden of computational load but also to increase the accuracy of classification. The reduced features are used as the inputs of classifiers. In this paper, we adopted a probabilistic neural network (PNN) as the classifier for handwritten digit and hand gesture recognition. The contributions of this paper include the following: 1) The development of a portable digital pen with a trajectory recognition algorithm, i.e., with the digital pen, users can deliver diverse commands by hand motions to control electronics devices anywhere without space limitations, 2) An effective trajectory recognition algorithm, i.e., the proposed algorithm can efficiently select significant features from the time and frequency domains of acceleration signals and project the feature space into a smaller feature dimension for motion recognition with high recognition accuracy. The rest of this paper is organized as follows. In Sections II and III, we survey related work on digital pen trajectory recognition and similar HCI applications and introduce the hardware components of the digital pen in detail, respectively. The proposed trajectory recognition algorithm consisting of acceleration acquisition, signal preprocessing, feature generation, feature selection, feature extraction, and a PNN is presented in Section V. In Section VI, simulation results are presented to validate the proposed approach. Finally, conclusions are given in the last section. |
RELATED WORK |
| Recently, some studies have focused on the development of digital pens for trajectory recognition and HCI applications. For instance, an alternative method of conventional tablet- based handwriting recognition has been proposed by Milner [15]. In his system, two dual-axis accelerometers are mounted on the side of a pen to generate time-varying x- and y-axis acceleration for handwriting motion. The author employed an HMM with a band pass filtering and a down-sampling procedure for classification of seven handwritten words. The best recognition rate is 96.2% when the number of states of the HMM is equal to 60. Oh et al. [16] presented a wand like input device embedding a triaxial accelerometer and a triaxial gyroscope for online 3-D character gesture recognition. Fisher discriminate analysis was adopted, and different combinations of sensor signals were used to test the recognition performance of their device. When all six axes raw signals were used as inputs of the recognition system, the recognition rate was 93.23%. In addition, they proposed an ensemble recognizer consisting of three sub recognizers with the following signals as inputs: acceleration, angular velocity, and estimated handwriting trajectory. The recognition rate of the recognizer was 95.04%. Similarly, a gesture recognition system consisting of a gesture input device, a trajectory estimation algorithm, and a recognition algorithm in 3-D space was proposed by Cho et al. [17]. The trajectory estimation algorithm based on an inertial navigation system was developed to reconstruct the trajectories of numerical digits and three hand gestures, and then, a Bayesian network was trained to recognize the reconstructed trajectories. The average recognition rate was 99.2%. Zhou et al. [18] proposed a μIMU for 2-D handwriting applications. They extracted the discrete cosine transform features from x- and y-axis acceleration signals and one angular velocity and used an unsupervised self-organizing map to classify 26 English alphabets and ten numerical digits. The recognition rate of 26 English alphabets and ten numerical digits achieved 64.38% and 80.8%, respectively. |
 |
| Our digital pen consists of capacitive triaxial accelerometer (LIS3L02AQ3, STMicroelectronics), The triaxial accelerometer measures the acceleration signals generated by a user’s hand motions. Capacitor formed between field electrode and mobile electrode. Acceleration Changes the capacitance which tends to Change in output voltage. LIS3L02AQ3 is a low-cost capacitive micro machined accelerometer with a temperature compensation function.The microcontroller collects the analog acceleration signals and converts the signals to digital ones via the A/D converter. The C8051F206 integrates a high-performance 12-b A/D converter and an optimized signal cycle 25-MHz 8-b microcontroller unit (MCU) (8051 instruction set compatible) on a signal chip.The wireless transceiver transmits the acceleration signals wirelessly to a personal computer (PC). The output signals of the accelerometer are sampled at 100 Hz by the 12-b A/D converter. Then, all the data sensed by the accelerometer are transmitted wirelessly to a PC by an RF transceiver at 2.4-GHz transmission band with 1-Mb/s transmission rate. This module is a direct line in replacement for your serial communication it requires no extra hardware and no extra coding to turn your wired communication into wireless one. It works in Half Duplex mode i.e. it provides communication in both directions, but only one direction at same time (not simultaneously). This switching from receiver to transmitter mode is done automatically. The overall power consumption of the digital pen circuit is 30 mA at 3.7 V. Therefore, if a typical AA battery (2000 mAh at 1.5 V) is used as the power of the system, the system requires three batteries simultaneously, and the lifetime is about 67 h. |
| III. TRAJECTORY RECOGNITION ALGORITHM |
 |
| Trajectory recognition algorithm consisting of acceleration acquisition, signal pre-processing, feature generation, feature selection, and feature extraction. The recognition procedure is composed of acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction. The acceleration signals of hand motions are measured by the pen-type portable device. |
| A. signal pre-processing |
| The signal pre-processing procedure consists of calibration, a moving average filter, a high-pass filter, and normalization. First, the accelerations are calibrated to remove drift errors and offsets from the raw signals. These two filters are applied to remove high frequency noise and gravitational acceleration from the raw data, respectively. |
| B. Feature Generation: |
| The features of the pre-processed acceleration signals of each axis (three axis) include mean, correlation among axes, interquartile range (IQR), mean absolute deviation (MAD), root mean square (rms), VAR, standard deviation (STD), and energy. The characteristics of different hand movement signals can be obtained by extracting features from the pre-processed x, y, and z-axis signals, and we extract eight features from the triaxial acceleration signals, including mean, STD, VAR, IQR[19], correlation between axes [20],MAD, rms, and energy [21] .Before classifying the hand motion trajectories, we perform the procedures of feature selection and extraction methods |
| C. Feature selection |
| The objective of the feature selection and feature extraction methods is not only to ease the burden of computational load but also to increase the accuracy of classification. The reduced features are used as the inputs of classifiers. |
| D. Feature extraction |
| For pattern recognition problems, LDA [23] is an effective feature extraction (or dimensionality reduction method) which uses a linear transformation to transform the original feature sets into a lower dimensional feature space. |
| E. PNN |
| PNN is a probabilistic neural network used as a classifier for handwritten digit and hand gesture recognition. The PNN is guaranteed to converge to a Bayesian classifier, and thus, it has a great potential for making classification decisions accurately and providing probability and reliability measures for each classification. The most important advantage of using the PNN is its high speed of learning. Typically, the PNN consists of an input layer, a pattern layer, a summation layer, and a decision layer as shown in Fig. 4. The function of the neurons in each layer of the PNN is defined as follows. |
| Layer 1: The first layer is the input layer, and this layer performs no computation. |
| Layer 2: The second layer is the pattern layer, and the number of neurons in this layer is equal to NL. |
| Layer 3: The third layer is the summation layer. The contributions for each class of inputs are summed in this layer to produce the output as the vector of probabilities. Each neuron in the summation layer represents the active status of one class. |
| Layer 4: The fourth layer is the decision layer |
 |
| TABLE I RECOGNITION RATES FOR DIFFERENT COMBINATIONS OF FEATURE SELECTION AND EXTRACTION METHODS WITH PNN AND FNN. |
 |
| TABLE II TOTAL CPU RUNTIMES (IN SECONDS) FOR TRAINING AND TESTING PHASES USING COMBINATIONS OF FEATURE SELECTION AND EXTRACTION METHODS WITH PNN AND FNN. |
 |
EXPECTED RESULTS |
| For both the recognition i.e. 1) handwritten digit recognition and 2) gesture recognition. We are going to collect about ten acceleration signals. Some from females & some from male. So as to calculate the effectiveness of proposed algorithm. The trajectory recognition algorithm consist of the following procedures: acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extractions explain in early section. we also expect result towards the Total CPU time comparatively less than other algorithms also as the recognition rate of tablet digitizer pen is 97.2% while IMUPEN gives the recognition rate about 90.4?% [5]. our expectation is to improve this in all aspect. We are going to used different combinations of feature selection and extraction methods and employed PNN to recognize handwritten digits and hand gestures. In addition, we are training for the better results of the recognition using PNN in case of all features than different feature engineering methods with those of feed forward neural networks (FNNs). |
CONCLUSION |
| In this paper we are going to use trajectory recognition algorithm consists of acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction. PNN can be quickly trained as an effective classifier. In this paper we used 2-D handwriting digits and 3-D hand gestures to validate the effectiveness of the proposed device and algorithm. |
References |
|