Information Systems Frontiers

, 11:539

Monitoring user activities in smart home environments

Authors

    • Jodrey School of Computer ScienceAcadia University
  • Senol Zafer Erdogen
    • Faculty of EngineeringMaltepe University
  • Jong Hyuk Park
    • Department of Computer Science and EngineeringKyungnam University
Article

DOI: 10.1007/s10796-008-9124-1

Cite this article as:
Hussain, S., Erdogen, S.Z. & Park, J.H. Inf Syst Front (2009) 11: 539. doi:10.1007/s10796-008-9124-1

Abstract

Wireless sensor networks (WSNs) enable smart environments to create pervasive and ubiquitous applications, which give context-aware and scalable services to the end users. In this paper, we propose an architecture and design of a web application for a sensor network monitoring. Further, the variation in received signal strength indicator values is used for knowledge extraction. Experiments are conducted in an in-door room environment to determine the activities of a person. For instance, a WSN consisting of Moteiv’s Tmote Sky sensors is deployed in a bedroom to determine the sleeping behavior and other activities of a person.

Keywords

Smart home environmentsWireless sensor networkPervasive computingUbiquitous computingIntelligent monitoring

1 Introduction

A smart environment can be defined as “a physical world that is richly and invisibly interwoven with sensors, actuators, displays, and computational elements, embedded seamlessly in the everyday objects of our lives, and connected through a continuous network”. Edwards and Grinter (2001) describe some challenges for smart homes by considering technical, social, and pragmatic domains. They discuss the problems arising from the smart homes deployment such as technical questions of interoperability, manageability, and reliability; social concerns about the adoption of domestic technologies and the implications of such technologies; and design issues that arise from considering just how smart the smart home must be.

Wireless sensor networks (WSNs) consist of a large number of smart sensors that have limited computing, storage, communication, and energy resources. The smart devices can interact with one another to create ad hoc, self-organized, and scalable networks that can provide intelligent, pervasive, and ubiquitous applications, Akyildiz et al. (2002), Ye et al. (2002). For example, sensor networks are used in military, security, health-care, Gao et al. (2005), environment and habitat monitoring. Further, as computing, storage, and communication resources are very limited for current available sensors, there is a need for energy efficient algorithms and techniques to provide scalable solutions, Kahn et al. (1999). A smart environment can be created by using dense deployment of sensors to provide in-situ and precise monitoring, as well as using actuators to enable real-time, adaptive, and context-aware control mechanisms.

The term smart home covers a variety of practical and theoretical approaches dealing with ideas of life, living and working today, and in the future. For instance, it automates and addresses problems in different areas such as home entertainment and health care.  Ghorbel et al. (2004) develop a framework of intelligent environment that incorporates various aspects such as a freedom of moving using wheelchairs, a compensation of upper arm impairments using robots, and environmental control systems for home appliances, without the intervention of another person.

WSNs are commonly used in assisted living and monitored care for the elderly and persons in need of such care. WSNs enable a high quality health care, where the person can live independently and can receive timely care when needed. For instance, approximately 30% of the people over 65 years of age and living in the community face an accidental fall each year. Further, although less than 10% falls results in a fracture, 20% of fall incidents require medical attention, Tabar et al. (2006), Gillespie et al. (2003). The real-time and intelligent monitoring could provide timely care for these urgent situations.

In pervasive computing, intelligent environments which identify and track the behavior of occupants in a room are gaining immense attention. The methods can be applied in homes for the elderly and disabled as an enabling technology for monitoring as well as in surveillance systems. While hidden video cameras are being used to identify people and their motion, the feeling of big brother watching is not that pleasant for patients in a medical facility or shoppers in a departmental store. Hampapur et al. (2005) explore the concepts of multiscale spatio-temporal tracking through the use of real-time video analysis, active cameras, multiple object models, and long-term pattern analysis to provide comprehensive situation awareness.

Received signal strength indicator (RSSI) technique measures the power of the signal at the receiver. Based on the known transmit power, the effective propagation loss can be calculated. Either theoretical or empirical model is used to translate this loss into a distance estimate. Further, as RSSI values vary because of mobility and obstruction, the RSSI variation can be used in the investigation of localization and mobility.

We present an architecture and design of a web-based application to provide continuous and intelligent monitoring for smart homes. The objective is to provide scalable application with an easy maintenance. Further, several experiments are conducted using Moteiv’s Tmote Sky sensors to study the behavior of RSSI values for localization and mobility of a person in a room environment. The remaining paper is organized as follows: Section 2 gives a brief description regarding RSSI related research. Section 3 describes an architecture and design of a web-based application. Section 4 provides details of the web application. Section 5 provides the experiment details and discussions for monitoring user activities in a room environment. Finally, Section 6 concludes the paper.

2 Related work

There is an active research in estimating the values of RSSI, link quality indicator (LQI), and packet reception rate for realistic radio communication Zhou et al. (2004), Gu et al. (2005), Cerpa et al. (2003), Ganesan et al. (2002), Woo et al. (2003). Zhou et al. (2004) investigate the degree of variation in RSSI values and propose a non-circular radio irregularity model (RIM) for sensor networks. Gallais et al. (2006) discuss the effect of a realistic radio channel on area coverage protocols. Scott et al. (2006) use transmit and receive signal strengths to investigate propagation patterns. Halkidi et al. (2006) use an online mechanism to determine the overall network status and to reduce the communications costs.

Erdogan and Hussain (2007) investigate radio irregularity with respect to distance, transmission power level, direction, and alignment of the sensor node from the base station. The experimental results show that proper alignment and transmission power level can reduce the energy consumption, in order to increase the network lifetime. In this paper, however, the focus is not on reduced energy consumption but the objective is to extract knowledge and context information from the RSSI values. The variance in RSSI values can be used to identify user behavior, mobility, and environment.

Recently, there has been an increasing interest in indoor localization systems. The RF based RADAR Bahl and Padmanabhan (2000) system can track users within a building, while the Cricket  Priyantha et al. (2000) location support system uses ultrasound instead for indoor localization. Directional approaches considering steerable directional antennas have been explored in  Savarese et al. (2001),  Nasipuri and Li (2002). In  Estrin et al. (2003), experimental evaluation of localization systems with beacon density is considered an important parameter in determining localization quality. In general, RSSI based positioning includes two phases: a) the training phase where the wireless map of the environment is determined using field measurements, and b) the positioning phase where position estimation is performed based on the wireless map.

RADAR system  Bahl and Padmanabhan (2000) is a typical in-door localization system based on RSSI technology. The main advantage of RSSI method is its low communication overhead and no additional hard ware requirement. As shown in an empirical study of RF technology, however, problems such as multi-path fading, background interference and irregular signal propagation characteristics make range estimates inaccurate. Furthermore, RSSI method needs an empirical radio propagation model beforehand. Erdogan and Hussain (2007) investigate radio irregularity with respect to distance, transmission power level, direction, and alignment of the sensor node from the base station.

RFID tags are commonly used for precise object tracking and inventory control Vogt (2002), De et al. (2004); however, these tags could be inconvenient for general human usage because RFID tags must be attached to a person. Sensor motes, on the other hand, can be used to detect mobility or user behavior without any wiring or a tag attached to a person. Certainly, the sensor motes cannot provide the exact identification accuracy as of RFID tags; however, in several applications, we do not need the exact identification. The sensor motes can provide unobtrusive monitoring at common places such as hospitals, restaurants, homes, schools, and offices. Further, the sensor motes can also help in maintaining person’s privacy. Although the information about the behavior of a person would be known, it would be relatively easy to hide the person’s identity.

3 Architecture and design

The architecture of a smart home application would contain the following entities: base station, sensor nodes, service agent, and a web application, as shown in Fig. 1.
  1. 1.

    Base Station: A base station is a gateway between sensor network and traditional networks (Internet).

     
  2. 2.

    Sensor nodes: The sensor nodes form a WSN for in-situ monitoring.

     
  3. 3.
    Service Agent: The service agent is responsible to extract knowledge and non-trivial information from the sensor data. The service agent would contain the following entities:
    • communication component: abstracts the communication details of the application

    • database: stores the data retrieved from the WSN

    • knowledge component: extracts knowledge from the data stored in a database

    • context-aware services: provide customized services for various applications

    As service agent needs extensive computing and storage resources, it must be located on a computer or a gateway node.
     
  4. 4.
    Web Application It provides customized web pages to the end users. The details are as follows:
    • MessageViewer: It reads data from the base station and transmits it over an open transmission control protocol (TCP) socket to the web application.

    • Listener Component: A component which reads the data from the MessageViewer and make it available for the web application.

    • Database Engine: All the sensor data is stored in a relational database such as PostgreSQL.1 A database (backend) engine maintains the database connection and enforces secure queries. For instance, Hibernate2 Query service/framework can be used for database engine. A PostgreSQL database is used because of its ease of use, no cost and support for large tables. The Hibernate framework also supports authorization, permissions, and views for multiple users.

    • Graphical User Interface: A front-end graphical interface is required to allow users to easily navigate the site and view data. For instance, the website provides a layout for the smart house, which is automatically updated when new monitoring facilities are created.

    • Data Visualization Framework: The application should provide graphs and layouts for queries and sensor positions on-the-fly. For instance, the data visualization is implemented in such a way that a user can specify the deviation and the number of readings to check for anomalous data. Cewolf chart framework,3 can be used as a data visualization framework. It can easily handle large amounts of data and dynamically generated graphs. Cewolf is commonly used because of its extensive features and simple implementation.

     
https://static-content.springer.com/image/art%3A10.1007%2Fs10796-008-9124-1/MediaObjects/10796_2008_9124_Fig1_HTML.gif
Fig. 1

Agent-based architecture for smart applications

3.1 Reading data

The MessageViewer component is developed outside of the web application. It receives data read from the base station using a TCP Socket. The data is collected in the following format:
$$ <node>~<time>~\{<comp>~<value>\} $$
The reading gives the sensor node id, timestamp of the reading, and a sequence of pairs of component id and component value. For instance, if sensor node 7 has three sensors, the sensor reading could be as follows:
$$ 7~2007-05-23~14:20:21.000~1~27~2~1.5~3~-26 $$
For easy maintenance and scalability, the data delimiter (tab by default) can be changed in the properties file settings.properties. On the server-side, an open socket listens for data from MessageViewer, which then enters into the PostgreSQL database. Figure 2 shows the pseudo code of the algorithm used to enter readings into the database.
https://static-content.springer.com/image/art%3A10.1007%2Fs10796-008-9124-1/MediaObjects/10796_2008_9124_Fig2_HTML.gif
Fig. 2

Pseudo code for SensorReadSocketTask

3.2 Event detection

An event is detected by considering the variation in received signal strength. Due to mobility or physical obstruction, the signal quality between two nodes is affected, which results in lower RSSI values. For instance, when RSSI value falls below a certain threshold, there is a possibility of an event. The event is detected as follows:
$$ Event=\left\{ \begin{array} {r@{\quad:\quad}l} TRUE & RSSI \le RSSI_{th} + (\mu - \sigma) \\ FALSE & otherwise \end{array} \right. $$
where μ is mean and σ is standard deviation. The mean and standard deviation are computed over a given n number of samples. In the preliminary investigation stage, the number of samples (n) and RSSI threshold (RSSIth) are determined based on the deployment environment and a few given test cases.

4 Website framework

The website allows two different types of users: admin and viewer. Any user in the viewing group will be able to do data queries, view graphs and view a sensor’s position and range. A user in the admin group has access to all viewing elements, as well as access to the administrative menu, which allows for the addition, deletion and modification of rooms, sensors, components and users.

To allow for easy navigation, a building layout is automatically generated by the web application. Further, by clicking on a room within the image, a browsing user can then view all sensor’s within a room, including their position on an X-Y eagles-eye graph. By selecting a sensor, the user is taken to the query page where they can set the appropriate graphs to display, the data-set range, minimum and time period. Figure 3 shows the main screen of the application.
https://static-content.springer.com/image/art%3A10.1007%2Fs10796-008-9124-1/MediaObjects/10796_2008_9124_Fig3_HTML.gif
Fig. 3

Main screen of the web application

4.1 Database

The PostgreSQL database is a relational database that not only records readings, but also holds information on sensors, components, and users. It has been designed in such a way that sensors, components and users can be added and removed without affecting other tables. The database can be split into two sections: users and sensors. The users section has two tables: users that contains the username and password information for a user, and roles that contains the different level of privilege a user has (viewing or admin). The second section is larger, containing five tables. First, the sensorpacks table, which holds the name, description, status and location of each sensor. Linked directly to sensorpacks is sensorpack_status, which contains different status types a sensor can have; rooms, which holds the name, description, size and location of a room; sensors which contains all information pertaining to sensor components; and readings, which contains all data read from the base station, as shown in Fig. 4.
https://static-content.springer.com/image/art%3A10.1007%2Fs10796-008-9124-1/MediaObjects/10796_2008_9124_Fig4_HTML.gif
Fig. 4

Relational database layout

The database is designed using a number of many-to-one relations. There can be many readings linked to one sensorpack, and many sensorpacks linked to one room. The database has been designed this way to allow for a more modular-based approach, wherein we can limit the number of hard connections within the database, and thus data is less likely to become corrupt when an insert or delete occurs.

4.2 Room generation

To automatically generate the building layout on the fly, DrawRoomLayout class determines the total size of the image, and then builds each individual room into the image using the algorithm given in Fig. 5. First, the dimensions and coordinates are computed based on the room’s actual dimensions. Then, for each sensor, it’s location and type is obtained. As the locations of sensor nodes are known at the deployment stage, there is no need of any localization algorithm. Further, there is no need for precise sensor location, an approximate location will be sufficient for room visualization. The sensor is drawn as an ellipse, where the choice of color depends on the current sensor reading. Once each room has been generated, the image is printed using the BufferedImage class. Finally, an image map is generated using a similar algorithm within the Rooms.jsp file, so that a room can be selected by clicking on the image.
https://static-content.springer.com/image/art%3A10.1007%2Fs10796-008-9124-1/MediaObjects/10796_2008_9124_Fig5_HTML.gif
Fig. 5

Algorithm for drawing a room layout

4.3 Data visualization

Users can also directly go to the position and query pages by clicking the appropriate link in the navigation bar. The Position view allows users to see all sensors within a specific room that have a certain component active. An X-Y eagle-eye graph is displayed with small colored circles representing the sensors within the room, as well as their current status. For each sensor, the range of the selected component is displayed in a shaded circle. This feature is very useful when it comes to placing sensors within a test-bed because it allows for the maximum coverage of an area using as few sensors as possible. The position-view graphs are created on the fly using the Cewolf chart framework as the base. Then, each sensor is represented with a circle using java.awt.geom.Ellipse2D class, which is then colored Green if fully functional, red if an error was detected or black if the battery is dead.

The Query view, as stated above, allows browsing users to select a sensor, component, time period, range and reading value (which can control what data is considered). Once the properties are set, the Sensor Monitor web application initiates the Cewolf chart software with the required dataset, and the appropriate graph(s) are generated. Below the graph, a hidden table contains all elements within the data set. By selecting show, a user can see the raw data used to generate the graph. For instance, the query view is used for a TMote Sky wireless sensor, which has readings for RSSI, LQI, Battery Voltage, and Power, as well as sensor components that read temperature and humidity. The graphs are generated using a user-determined range, which allowed for the smoother image, as shown in Fig. 6 and Fig. 7.
https://static-content.springer.com/image/art%3A10.1007%2Fs10796-008-9124-1/MediaObjects/10796_2008_9124_Fig6_HTML.gif
Fig. 6

Sending a query through a web form

https://static-content.springer.com/image/art%3A10.1007%2Fs10796-008-9124-1/MediaObjects/10796_2008_9124_Fig7_HTML.gif
Fig. 7

Humidity variation for a sensor

Another small application which sits on top of the Sensor Monitor web application is the data monitor. As data is read from MessageViewer and input into the database, the data monitor will read the entries. After x readings, the monitor will compare the difference in range between the elements with y. If the data set contains anomalous information (i.e. data outside of the range), the website admin is emailed with a message detailing which sensor/component produced the irregular data as well as what time it occurred. Although the application is currently built only to send emails to the administrator, one could easily implement a component that would allow for the website to send a SMS message to a cell phone, send a fax or print off a report of the anomalous data.

4.4 System robustness

Since the Sensor Monitor Web Application is built to read data from clusters of sensors in multiple rooms, both the database and the graphical interface must be able to handle large numbers of users as well as millions of sensor readings. The results generated by the experiment showed that the website framework and the database are both stable enough to handle such large amounts of data. The details of experiment are as follows:
  • The MessageViewer application was modified so that data is randomly generated, instead of being collected from the base station. This allowed for data to be sent to the web application at a much faster rate, for testing purposes.

  • Next, 10,000 users were added to the database, using a simple SQL script

  • After all the new data was added to the database, the position and query charts were generated for samples using different ranges, domains, and components. Several users were also tested by logging in, and submitting queries.

It was found that, even with a million sensor readings and tens of thousands of users, the website operated within usual parameters. There was no noticeable delay in waiting for data to be queried or graphs to be generated. As SQL query filters the data according to date, time, room, and other parameters, there was no scalability issue in terms of managing the data. However, the performance of SQL query will be limited by the actual hardware and database engine at the back end. Further, it must be noted that although 10,000 users were added in the database, there were only a few (20–30) concurrent connections. If there are a large number of concurrent users, there will be a delay in the responsiveness of the application, which could be improved by the efficient high performance computing (HPC) techniques; the discussion of HPC techniques for scalable web and database applications is beyond the scope of this paper.

4.5 Easy maintenance

Since the entire web application was written in Java, and was built in a modular way, adding new components, removing unneeded ones, or editing current components is simple. Many of the components (Rooms, Cewolf Charts, User Administration, etc) are fairly stand alone, in that with a limited number of changes these components can be altered or removed without affecting the overall performance of the application or its ability to read and present data. A good instance of this is the Room interface. Currently, sensors’ are categorized in a hierarchy of Room→Sensor→component. However, a developer could easily add another level to this hierarchy, or change it to better reflect the usage of the website.

As the entire application is written in Java using Javabeans and Java Server Pages, it is platform independent and can be executed using any Servlet Container (e.g. Tomcat). As stated above, the system also uses the Hibernate query service to manage its database. This allows for users and developers to use a number of different database applications (i.e. PostgreSQL) with relatively little changes to the application code itself.

5 Home monitoring

Experiments are conducted in an in-door environment to investigate localization and mobility of a person. The variance in RSSI values is used to identify the current location and activities of a person.

5.1 Monitoring human behavior in a bedroom

In this experiment, sensors are deployed in a bedroom to monitor the behavior, mobility or lifestyle of a person. For instance, the sensors can be used to determine the time spent on the study table or a bed.

Figure 8 shows the room plan of a bedroom. The sensors are deployed as follows: 4 sensors on a study table (Mote 1, Mote 2, Mote 3, and Mote 4), 2 sensors near the window (Mote 5 and Mote 6), 1 sensor on the corner of bed (Mote 7), and 2 sensors near the door (Mote 8 and Mote 9). The base station is attached to a laptop on the study table. The sensor data is collected for several days.
https://static-content.springer.com/image/art%3A10.1007%2Fs10796-008-9124-1/MediaObjects/10796_2008_9124_Fig8_HTML.gif
Fig. 8

Room plan of a bedroom

Figure 9 shows RSSI values for table, window and door sensors. RSSI variation confirms with the actual behavior of a person. For instance, as recorded by the person, the RSSI variation confirms the behavior observed by the sensors. The student was working at the study table for the initial 3 h, which is confirmed by RSSI variation for Table motes 1 and 3, as shown in Fig. 9a and Fig. 9c respectively. Further, the small variation for Table motes 2 and 4 (Fig. 9b and Fig. 9d) indicates that the person’s sitting position did not affect these values. By comparing the RSSI values of Table motes 1, 2, 3, and 4, the sitting posture or inclination on a table can also be determined, although it is not investigated in this paper.
https://static-content.springer.com/image/art%3A10.1007%2Fs10796-008-9124-1/MediaObjects/10796_2008_9124_Fig9_HTML.gif
Fig. 9

RSSI variation for sensors in a bedroom. a Table: Mote 1. b Table: Mote 2. c Table Mote 3. d Table: Mote 4. e Window: Mote 5. f Window: Mote 6. g Door: Mote 8. h Door: Mote 9

After 3 h of working, the person slept for 7–8 h. The sleeping time is also evident from RSSI values of window and door motes. Further, the RSSI variation for door motes are not consistent. For instance, Mote 8 (Fig. 9g) near the bed confirms the sleep behavior; however, the mote near the closet (Mote 9) shows some unexpected RSSI variation. Finally, the RSSI variation in the last couple of hours shows the morning activity, which is confirmed by most of the motes.

The above results can be used in maintaining a daily activity diary for a person. The results obtained at the base station can be logged in a database and can be retrieved through a web-based application. Consequently, a life style of a person, active or sedentary, can be estimated by these results.

Figure 10 shows the humidity variation for the same experiment as described above. The humidity variation confirms the presence of a person. Initially, the humidity level was low; however, due to presence of a person in a small room, the humidity level reaches to a higher value. Further, in the morning, when the door was opened, there is a sharp decrease in the humidity level, as shown around 5am in the graphs of Fig. 10. These rapid changes in humidity level can be used to determine the opening and closing of the door.
https://static-content.springer.com/image/art%3A10.1007%2Fs10796-008-9124-1/MediaObjects/10796_2008_9124_Fig10_HTML.gif
Fig. 10

Humidity variation. a Window: Mote 5. b Window: Mote 6. c Bed: Mote 7. d Door: Mote 8

6 Conclusion and future work

In this paper, an agent-based architecture is proposed for knowledge discovery. The variation in RSSI values is used for knowledge extraction. Several experiments are conducted in an in-door environment such as: a) a WSN is deployed in a bedroom to determine the sleeping behavior and other physical activities of a person, b) a WSN is used to identify the occupied chairs in a room, as well as the mobility of a person, and c) humidity sensor of Moteiv’s Tmote Sky is used for knowledge extraction.

In future, the fuzzy-based logic and other machine learning techniques will be used for knowledge discovery and data mining. Further, the context-aware services and end-user applications will be provided.

Copyright information

© Springer Science+Business Media, LLC 2008