ISSN ONLINE(2320-9801) PRINT (2320-9798)

Survey on Context Based Input of Microenvironment through Sensors and Inference of Activity to Alert Smartphone User

Monika Sharma1, S.R.Idate2
  1. P. G. Student, Dept. of I.T., Bharati Vidyapeeth Deemed University College of Engineering, Pune, India
  2. Associate Professor, Dept. of I.T., Bharati Vidyapeeth Deemed University College of Engineering, Pune, India
Related article at Pubmed, Scholar Google

Visit for more related articles at International Journal of Innovative Research in Computer and Communication Engineering

Abstract

Basic functionalities of Mobile phones have changed from mundane activities like call, SMS to performing the tasks like navigation, tracking etc. which has made mobile units smart. This has become possible due to the introduction of sensors. Sensors enable a mobile phone to be context aware i.e.; aware of the environment and activities of phone and its user. Sensors are of various types and each of them has a different functionality e.g.: magnetometer, accelerometer, proximity, camera, Wi-Fi, gyroscope etc. They can be used individually or in group to detect the microenvironment (a few centimeters around a unit) activities. At the lowest level sensors take as input the context of user and then at the next level these are analyzed using various techniques and algorithms for the purpose of inferring the correct activity that is being performed. At the application level according to the activities the various tasks are executed. For example when a call activity occurs then by inferring the proximity of user the task of automatically call receiving is executed. These types of functionalities help the user in using their mobile unit in an efficient manner without much interference from user’s side.as all this requires the sensors to be activated for long intervals of time which in turn results in battery consumption and some performance issue. These can be dealt by stopping unnecessary processes and activating sensor only when its related activity is being performed.

KEYWORDS

Microenvironment sensing; pervasive computing; context recognition; mobile sensors

I. INTRODUCTION

Sensors are being increasingly used in devices like mobile phones, laptops, I pads, etc. to increase the usability of system and to enhance the user’s experience. Without the active participation of user sensors are able to collect all the data related to the activities of user and utilize such data for execution of various tasks.
Sensors are used for pervasive computing or also known as ubiquitous computing. It requires that the sensors run continuously in the background once an application related to a particular sensor has started (i.e. switched on). Then these sensors subsequently monitor the changes occurring in the environment of user and storing all the data related to those changes. This data then can be used for finding out the exact kind of activities (in this case activities like a phone call, a noise alert etc.) And take action on those activities (like automatic call picking on receiving a phone call). As in a mobile unit at a particular time there are many processes running at the same time. When the sensors also run simultaneously it adds to load of the system and can hinder the performance of the machine. Also when various sensors are always running in background to record the activities of user consumes battery life. So when dealing with sensors two issues are there i.e. performance and battery consumption. Here we are using techniques through the applications to overcome these two problems like killing processes which have not been active for a long period of time and saving the battery life by not letting the system use the functionalities like blinking, ringing, and vibration mode simultaneously when there is no need. For example when the phone is in pocket it does not need to use the functionality of blinking a light when a new message, call or a notification is received.
Each sensor has parameter which changes when there is some change in the context of the usage. Every parameter is related to an event and these events bind themselves in the operating system of the mobile phone. Sensor listeners are responsible for the continuous assessment of and storing the data of the environment changes as input. Algorithms are used to manipulate the collected data for an output.

II. RELATED WORK

Zheng Yang [1] SHERLOCK consists of a smartphone equipped with sensors to record and infer from that data the context in which the phone is placed at any given time. In this paper a prototype was run on an android based platform and over a period of time readings were taken as input and the efficiency of the system is concluded as having high sensing accuracy, being energy efficient and providing a rapid development system. The various activities that are detected using this system include: Moving &Walking Detection, Local Placement Recognition, Phone-in-hand Identification, On-body Placement Recognition, Phone Interaction Detection, Backing Material Detection.
Derick A. Johnson [2]: MIROAD considers the use of smartphones to find out the driving style which can be put into two categories: Aggressive and Non-aggressive
This system employs a non-invasive method with which it can record and sense the activities of user while driving to determine whether or not they are driving safely. The sensors used here are: Camera (often multiple), Microphone (often multiple), and 3-axis Accelerometer, 3-axis Gyroscope, Proximity, Ambient Light, Touch, Magnetometer (compass), and GPS.
The system is named as MIROAD: A Mobile-Sensor-Platform for Intelligent Recognition of Aggressive Driving. Two methods are used here: Driving Event Recognition, Gesture Recognition. When the MIROAD application is started, it can be in one of two modes: Active: data is only stored in an event when potentially-aggressive activity is detected or passive: all the data is recorded all the time for analysis which is done at some other time than current time. Algorithms used here are detection and recognition also endpoint detection is used to detect the start and end point of maneuvers like turning hard left or right etc.
Jilong Liao [3] which focuses on the concept of context based input. The source of such input is through the smartphones which include the sensors as part of their internal devices which are used to read the microenvironment. This collection of data is called raw data collection which is the first step in converting such type of data into the form of sentences which are highly usable and human readable. Various machine learning algorithms are used to understand and learn the various contexts and thus infer the events accordingly. The various sensors used are accelerometer and gyroscope which collect contextual data. Classification and event generation is done then the recorded data in digital diary is used to identify the patterns and behavior of the user. This is a user-centric application which allows a person to log their daily activities which does not require their involvement. This results in a digital diary which is able to record all the data non-invasively and without any mistakes (i.e. in concise and precise manner). Two types of models are introduced here to enhance the ability of a smartphone sensor to infer and predict the activity’s type efficiently. These algorithms are namely sustainable mining model and narrative structured sentence model. Steps in this methodology used: System Framework, Raw Data Collection, Context Analysis, Event Personalization, Diary Generation.
Xing Su [4]: There are many data mining based applications in this paper which are available like health management, elder care etc. These all types of applications need to know about the user’s activities that occur in his life on a daily basis. So we need a human activity recognition system to know the tasks that are performed which will result into identifying the behavioral pattern of that user. The sensors like gyroscope and accelerometers are used to record data of activities like walking and sitting. This application has - Inputs: Sensors, Outputs: Activities. The core techniques used here are: Raw data collection, Preprocessing: De-noising and segmentation, Feature computation, Classification. The challenges are: Subject sensitivity, Location sensitivity, Activity complexity, Insufficient training set, Energy and resource constrains. The applications here are: Daily life monitoring, Elderly and youth care, Personal biometric signature, Localization.
Jennifer R. Kwapisz [5]: Various types of sensors are present in a mobile device like camera, microphones and accelerometers which are used to perform task of video, audio and acceleration sensing. In this paper only one sensor i.e. an accelerometer is studied thoroughly and its working, implementation and usefulness is studied. The only specification for the success of this application is the presence of a accelerometer which is nowadays available in all kind of smartphone which might be a iPhone or some simple smartphone contain android as its operating system. Accelerometers are used to measure: All three dimensions, Detection of mobile phone’s orientation, They can detect earth gravity. Initially this sensor was used for screen rotation and game playing but over time more of its usefulness was discovered in realizing human activity recognition. Six activities namely are: standing, ascending stairs, jogging sitting, walking and descending stairs are considered here and accelerometer is used to measure an identifying them. Different types of graphs were procured related to all these activities and then used the results in application of various functions. The various tasks are: Data Collection, Feature Generation & Data Transformation, and The Activities.
Stefan Dernbach [6]: Mostly sensor application in smartphones are used to identify simple activities such as walking sleeping etc. on the basis of locomotion of the human body, extensive inference algorithms use many different body sensors to recognize a certain human activity. This paper stresses on finding out the complex tasks like cooking and uses only two sensors namely accelerometer and gyroscope to find out the activity instead of body sensors. Supervised learning algorithms are used to recognize the complex user activities and how they differ from simple activities. When the experiment was completed it was discovered that prediction on complex activities is very poorly done compared to simple ones. But on the other hand these algorithms were able to work on complex activities to some measure and were also able to easily identify simple tasks. There are various activities namely: Data Collection, Activities, Feature Extraction, and Classification.
Zhibo Wang: [8] Recommendation system is one of the important methods of information retrieval. Environmental sensors are used to gather user’s data from their daily activities. Earlier social networks used social graphs to find friends but this recommender system uses mobile sensor data to infer the life style and habit to recommend them to other people with similar choices. Text and data mining activities are performed on raw data collected from user activities which is called here as life documents. Algorithm named is a data mining and extraction tool used in this system. The process in this system helps find people friends who cannot be easily searched by basic search methods and queries. Friend matching graphs are used to match and evaluate the likeness and feedback methods help increase the recommendation accuracy.

III. PROPOSED ALGORITHM

While configuring the Smart Sensing System (SSS) it is required to maintain the security of the application so that it cannot be changed by any unauthorized person. SSS uses RSA algorithm for encryption of the details so that they cannot be accessed by anyone. It includes the following phases:
STEP 1: Key generation: First of all in RSA we have to choose two prime numbers: (p, q) at random with same length which are not known to anyone (i.e. are secret).
STEP 2: Calculate: m=p*q and Ø (m) = (p-1)*(q-1); m is modulus for public and private key and Ø is Euler’s totient function (secret).
STEP 3: Take an integer e s.t: 1<e<Ø (n); e is public key exponent.
STEP 4: Calculate d ≡ e−1 (mod Ø (n)); i.e., d is the modular multiplicative inverse of e (modulo Ø (n))
STEP 5: For encryption c = m ˄ e % n
STEP 6: For decryption m = c ˄ d % n

IV. PROPOSED METHOD

For recognizing the task and activities sensors have to analyse the various events and the corresponding changes in them that occur over intervals of time.
Steps for developing User contextual data sensing system are given as below:
STEP 1: Configuring the settings of various applications according to which threshold of different sensors can be set if required.
STEP 2: Initiating the application by turning them on such that they run continuously in the background of system and collect data as input.
STEP 3: Sensors are used to collect user contextual data which is in physical form initially and then converted into parameters that can be manipulated for deduction of events.
STEP 4: By analysing the collected data, events are recognized and these events help in defining the activity.
STEP 5: After inferring a task an application assists user to efficiently perform activities.
STEP 6: The performance of the system is increased by a evaluating and eliminating unnecessary tasks.
STEP 7: When a certain activity occurs it is recognized by the system and accordingly the action is taken
In this project we are going to develop and integrate various functionalities into single application. These functionalities are:
A. Increasing efficiency of user and system by
• Proximity call
• WIFI-Webcam
• WIFI-Mouse
B. Increasing Security of handset and user
• Unauthorized access
• Touch pressure
• Noise alert
• Morse code
C. Increasing performance of the system by
• Close environment
• Back surface
• Process kill
Proximity call: whenever a user receives a call, the change in values of parameters occurs and an event is identified by the system. Whenever users hand or ear comes in close proximity of mobile unit the call is received automatically. This functionality requires the use of proximity sensors.
Wi-Fi webcam: It helps the user to convert and use the mobile unit as a webcam for the laptop or computer if one is not available. This is done when the system allows both the machine (i.e. computer and mobile unit) to be connected via Wi-Fi to each other. Both front camera and rear camera can be used for such functionality.
Wi-Fi mouse: In the absence of a mouse or such pointing device user can easily convert the mobile unit as an external device (i.e. mouse) they are connected to each other through Wi-Fi. The camera sensor is used for this purpose.
Unauthorized access: Here the security of mobile phone is considered. Whenever a wrong pattern is entered by the person who is using the mobile phone it is detected by the input from the touch sensors and results in activation of front camera which captures the photograph of user and sends it to an email address provided by the authenticated user of the mobile phone. Also an alert is send to an alternative number provided by the user.
Touch pressure: It concerns with the security of the mobile phone user. During the configuration of the application user provides a time duration which is utilized in this functionality. If the user is in danger they can press on the screen for the pre specified duration which results in sending their GPS location to the appropriate authorities.
Noise Alert: It aids user as a health care system. As the user can activate this functionality in ICUs, where a beforehand set threshold is used by the system to recognize certain noise level and alert the authorities if such situation is detected. For example if a patient is in need then if no help is available they can easily be alerted. It requires microphone sensors.
Morse code: It helps the user to receive and send text messages in the form of code language called Morse code. It utilizes flash or screen blink functionalities of the system to convey the message.
Close environment: Whenever the system detects that the mobile unit is placed inside a closed environment like inside a bag then functionalities like blinking of mobile light whenever a new message or notification is received are automatically switched off. When the system detects the change from close to open environment the functionality of light blink is switched on again. It helps system in saving the extra and unnecessary efforts and thus increases its performance.
Back surface: It detects the type of surface on which the mobile phone is placed i.e. soft or hard surface. If the surface is soft then it is unnecessary for the phone to be on vibration mode because it is not felt so it only rings. If the surface is hard the mobile only turns its vibration mode on. This increases the performance of system and saves battery life. Process Kill: In a system there are three type of activities information provided by system itself i.e. Active services, active activities and processes. The processes are a list of inactive and unused processes which take unnecessary space of system such processes are killed in this functionality using magnetic sensors.
Different types of alarms are sounded when sensors find a change in events which alert the user to the situation at hand.

V. APPLICATIONS

1. For health care system
2. For user’s mobile safety
3. For assistance of user in danger
4. For automatically detecting mundane activities like an incoming phone call and assisting the mobile phone user by performing the corresponding task related to that activity like automatic call picking.
5. It helps in increasing the efficiency of user and the performance of the mobile unit.

VI. CONCLUSION AND FUTURE WORK

Microenvironment sensing system is used to collect the user’s contextual data. This data is collected using the sensors which are inbuilt in the smartphones. These smartphone are android based and the sensors use the data they get as input to infer various factors like phone placement, backing material. After the detection of the environment is done then the various modules of the system provide various functionalities to the user. For example according to the backing surface the mobile will ring or vibrate depending upon whether the surface is soft or hard. These activities help the user work efficiently and also increase the performance of the system as a whole.

REFERENCES

1. Zheng Yang, Longfei Shangguan, Weixi Gu, Zimu Zhou, Chenshu Wu and Yunhao Liu , “Sherlock: Micro-environment Sensing for Smartphones”, IEEE Transactions on Parallel and Distributed Systems , pp.2-5, 2013

2. Derick A. Johnson and Mohan M. Trivedi , “Driving Style Recognition Using a Smartphone as a Sensor Platform”, Laboratory for Intelligent and Safe Automobiles (LISA)University of California, San Diego La Jolla, 14th International IEEE Conference on

3. Intelligent Transportation Systems Washington, DC, USA. pp.1609-1615, 2011

4. Jilong Liao, Zhibo Wang, Lipeng Wan, Qing Cao and Hairong Qi, “Smart Diary: A Smartphone-based Framework for Sensing, Inferring and Logging Users’ Daily Life”, pp.4-12, 2013

5. Xing Su, Hanghang Tong, and Ping Ji*, “Activity Recognition with Smartphone Sensors” , Volume 19, pp.236-248, 2014

6. Jennifer R. Kwapisz, Gary M. Weiss, Samuel A. Moore, “Activity Recognition using Cell Phone Accelerometers”, 2013

7. Stefan Dernbach, Barnan Das, Narayanan C. Krishnan, Brian L. Thomas, Diane J. Cook, “Simple and Complex Activity Recognition Through Smart Phones”, 2012

8. Zhibo Wang, Jilong Liao, Qing Cao, Hairong Qi and Zhi Wang, “Friendbook: A Semantic-based Friend Recommendation System for Social Networks”, IEEE, 2013

9. Kiran K. Rachuri, “Smartphones based Social Sensing: Adaptive Sampling, Sensing and Computation Offloading”, pp.5-46, 2013

10. Patrice C. Roy, Newres Al Haider,William VanWoensel Ahmad Marwan Ahmad and Syed SR Abidi, “Towards Guideline Compliant Clinical Decision Support System Integration in Smart and Mobile Environments: Formalizing and Using Clinical Guidelines For Diagnosing Sleep Apnea”, NICHE Research Group, Faculty of Computer Science Dalhousie University, Halifax, Canada, 2014

11. Jens Gerken, Stefan Dierdorf, Patric Schmid, Alexandra Sautner*, Harald Reiterer, “Pocket Bee - a multi-modal diary for field research by”, ACM , NordiCHI ,University of Konstanz, 2010

12. A Survey of Mobile Phone Sensing by Nicholas D. Lane, Emiliano Miluzzo, Hong Lu, Daniel Peebles, Tanzeem Choudhury, and Andrew T. Campbell, Dartmouth College, 2010

13. Jerald Jariyasunant, Raja Sengupta, and Joan Walker, “Overcoming battery life problems of smartphones when creating automated travel diaries”, UC Berkeley April, 2014

14. Robert LiKamWa, Yunxin Liu, Nicholas D. Lane, Lin Zhong, “MoodScope: Building a Mood Sensor from Smartphone Usage Patterns” Rice University, Houston, TX Microsoft Research Asia, Beijing, China, 2013