Pattern Analysis and Applications

, Volume 12, Issue 3, pp 237–249

Soft authentication using an infrared ceiling sensor network

Authors

  • T. Hosokawa
    • Graduate School of Information Science and TechnologyHokkaido University
    • Graduate School of Information Science and TechnologyHokkaido University
  • H. Nonaka
    • Graduate School of Information Science and TechnologyHokkaido University
  • J. Toyama
    • Graduate School of Information Science and TechnologyHokkaido University
Theoretical Advances

DOI: 10.1007/s10044-008-0119-9

Cite this article as:
Hosokawa, T., Kudo, M., Nonaka, H. et al. Pattern Anal Applic (2009) 12: 237. doi:10.1007/s10044-008-0119-9

Abstract

Person identification is needed to provide various personalized services at home or at the office. We propose a system for tracking persons identified at the entrance to a room in order to realize “soft authentication.” Our system can be constructed at low cost and works anytime and anywhere in a room. Through experiments, we confirmed that the system could track up to 5 persons with a high probability of correct identification, though precise identification is difficult.

Keywords

Soft authenticationInfrared sensorSensor-attached ceilingHuman tracking

1 Introduction

Commercial authentication systems using various biometric technologies, such as identification by fingerprint, speech, iris, face, palm vein and finger vein, have recently been developed. Such systems are useful for maintaining a high level of security, but a high level of security is not needed in daily life. At home or at the office, we need authentication to know who wants services and where he/she is. In this situation, misidentification does not cause a serious problem.

The important feature required for such a system is that there is no psychological or physical disturbance of usual life. The usefulness of continuous surveillance by video cameras for monitoring single elderly residents has been extensively discussed [13], but the use of such a system might have an adverse psychological impact. Requiring people to carry a special ID device such as an infrared ID badge [4] might also cause problems. Recording of a person’s behavior can also cause a privacy problem. It is also annoying to show biometrics whenever, we need a personalized service. In these respects, many systems using biometrics are not appropriate for authentication to provide personalized services. To solve these problems, an authentication process that does not require any cooperation of users and in which sensing devices are unnoticeable is desirable.

We distinguish our motivation from the traditional motivation by calling authentication for security “hard authentication” and by calling authentication for personalized services “soft authentication.” The differences are summarized in Table 1.
Table 1

Differences between “soft authentication” and “hard authentication”

Item

Soft authentication

Hard authentication

Goal

Personalized service

Security

Process/Device

Unnoticeable

Noticeable

Population size

Small (2–20)

Large (10–100,000)

Place to use

Home or office, anywhere in a room

Building or security-controlled room, fixed place

Level of accuracy

Low

(Very) High

Strength of evidence

Weak (behavioral evidence, gait, etc.)

Strong (iris, DNA, finger vein, etc.)

User’s cooperation

Unnecessary

Necessary

Psychological barrier

Weak/None

Strong

Sensors

Infrared, pressure, etc.

Camera, special devices

Necessary environment

No special condition, day and night, movable obstacles

Controlled condition, no obstacles

Establishment cost

Low and flexible

High and fixed

Stolen damage

Low

Crucial

In soft authentication, we can only obtain several pieces of weak evidence because user’s cooperation is not expected. Therefore, a high rate of correct identification by a single piece of evidence is difficult to obtain. A combination of several pieces of weak evidence should therefore be used. The number of persons to be identified in a soft authentication system should be less than 20. In such a small group, a reasonable rate of correct identification would be achievable. The performance of a soft authentication system should not be disturbed by obstacles that can be moved. For example, furniture such as chairs, sofa, desks and tables are sometimes moved. In addition, change in light conditions during the day and at night should not greatly affect performance. The psychological aspect is also an important issue. A video camera cannot be used because it gives people an uncomfortable feeling of being observed and recorded.

The weakness of evidence would be useful in a different sense. Typical biometrics such as iris or fingerprint provide very strong evidence, but they are almost impossible to be changed. Therefore, if they are stolen or completely copied, there is nothing we can do. On the other hand, pieces of weak evidence are informative only in a limited condition/situation and a combination of them is necessary for authentication. Thus, even if a single piece of evidence is stolen, it would not cause a serious problem.

Possible biometrics for soft authentication are divided into three categories according to the degree of restriction:
  1. 1.

    For the most general (almost no restriction) cases, several kinds of natural behavior are taken into consideration. Motions and movements of the body, hands or legs can be used to provide evidence. It is desirable that such movements can be sensed in a large area, both indoors and outdoors.

     
  2. 2.

    For somewhat restricted cases, a person’s gait or sitting posture can be used. In these cases, the sensing area is limited to a hall/passage or to the chair’s position

     
  3. 3.

    For the most restricted cases, key-typing or operations using special devices such as a mouse can be used. The place where such devices are used can be specified.

     

Many studies on the above categories have been carried out. For the first category, studies and analyses of the user’s behavior by video cameras or by infrared sensors have been carried out [1, 3, 58]. For the second category, analyses of behavior of a sitting person [9] and analyses of walking patterns by multiple footprints [10] have been carried out. For the third category, studies on the use of key stroke dynamics for detecting individuals [11] have been carried out.

The focus of this study is between the first and second categories, closer to the first category. We developed a system for tracking users continuously by the use of infrared sensors attached to the ceiling. That is, the biometrics are motions and movements of the body. In this system the users are identified at the entrance to a room by their finger veins. Tracking persons is, in general, not authentication in the exact sense, but tracking once-identified persons is one of the most desirable authentication methods. We can know anytime both who the person is and where he/she is.

In this system, due to the characteristics of infrared sensors, the only information we obtain is the fact that (moving) someone is under the active sensor. Therefore, when there are two persons in one place, they cannot be distinguished. Accordingly, in a situation where many persons are walking around in a room, after a while, we can only know their location with some ambiguity.

In a tracking system using only a single piece of weak evidence, it is natural to assume that the precision of prediction will decrease as time passes. Therefore, we assume that other stronger pieces of evidence are sometimes available. In our system, such a strong piece of evidence is obtained when someone enters or leaves the room, because he/she is necessarily (hard) authorized to open the door.

As an alternative system for realizing soft authentication, we have been developing a chair system with pressure sensors placed on the seating face [9]. The system can identify the person sitting on the chair with a correct identification rate of over 70% if the number of people is limited to 10. Complementary usage of the proposed ceiling network and the sensor chair would be promising for soft authentication.

1.1 Related works

Multiple video cameras have been used in many studies for tracking people [1, 1216]. These studies include studies on tracking two persons by multiple cameras in an indoor environment [15] and tracking with panoramic vision sensors in an indoor environment [12]. Zhao [13] succeeded in tracking persons by their modes of movement (e.g. walking, running) using multiple outdoor cameras. In another study, using a human shape model for gait, human tracking was achieved by the use of a single camera in an outdoor environment [16]. In that study, up to 20 persons could be distinguished with a correct identification rate of about 90%. These systems do not require any user cooperation, but vision is sometimes not obtained due to the existence of obstacles. In addition, video cameras are greatly affected by light conditions.

Monitoring residents by video cameras is also popular [13]. However, cameras might violate privacy. There has been a trial on monitoring residents by several electrical sensors attached to doors, kitchenware such as a microwave oven, and a washing machine [3]. The aim of that trial was to determine whether an abnormal state of a single resident can be detected by a long period (two and a half hours) of sensor inactivation. However, such a system cannot provide information on a person’s current position in a room.

There have also been trials on tracking systems using pressure sensors on the floor [10, 1719]. However, there are some problems in such tracking systems: many sensors are necessary (about 25 sensors per 1 m2 in [10]) and there is a wide area occluded by obstacles such as tables and chairs.

There have also been studies on the usefulness of infrared sensors for tracking [48, 1719]. In two of those studies [4, 18], however, the subjects had to wear an ID badge. Many people would hesitate to wear such sensing devices. There has been a series of studies using pyroelectric infrared sensors and Fresnel lens arrays [7, 8]. In those studies, a pyroelectric infrared detector with Fresnel arrays was developed for human identification and human tracking. In the above studies, the resolution of tracking was not sufficient to specify a person’s position in a relatively large room with furniture. The closest study to ours is a study in which sensors attached to the ceiling were used to detect a single person’s position [19]. In the system used in that study, 9 sensors were attached to a 2.5 m × 2.5 m area of ceiling and position detection was carried out by reading the analog output of sensors. However, the detective space is very limited, and there has been no study on performance of this system in more general environments in which more than one person walk around a large space such as a room. In addition, there has been no discussion on tracking of once-identified persons, which is our goal.

2 Infrared sensing system

2.1 Sensor module

We used a passive infrared sensor called a “pyroelectric infrared sensor” or an “infrared motion sensor.” The pyroelectric infrared sensor detects change in the motion of a person or an object with a different temperature from the surrounding temperature. Owing to its low price and wide sensing area, it has been used for many applications, such as automatic security lights, burglar alarms, visitor acknowledgement, light switch control and door opener.

We used NaPiOn (AMN11111, Matsushita Denko Co.) as the sensor module [20]. The specifications and characteristics are shown in Table 2.
Table 2

Standard performance of a sensor

Rated detection distance

5 m (max)

Horizontal detection range

100°

Vertical detection range

82°

Detection zone

64 zones

Motion speed

0.5 m/s (min) –1.5 m/s (max)

Motion amplitude

0.3 m (max)

This sensor module is a standard type of motion detection sensor equipped with a built-in amplifier and a digital output circuit. It also includes 16 lenses for gathering infrared radiation to 4 quadrants on the surface of the pyroelectric infrared detector. Thus, 64 (=4 × 16) detection zones and corresponding detection area are formed in front of the sensor module. The detection area is up to 7.42 m × 5.66 m on a plane at a distance of 2.5 m from the sensor. In our system, the detection area of each sensor was narrowed by a hand-made cylindrical lens hood.

2.2 Layout of infrared sensor network

Figure 1 shows a block diagram of the sensor network. The sensor units and the controller are interconnected with a bus architecture, which is occupied by only one of them at a time. The traffic is controlled centrally by the controller so that no collision of the packet occurs. All of the units are scanned during collection of data from the units in the way of time-out detection. A break down of any unit is detected at once. The units can be detached and substituted even while the system is in operation.
https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Fig1_HTML.gif
Fig. 1

Interconnection of sensor units

Fifty sensors were attached to the ceiling of our research room (15.0 m × 8.5 m). Figure 2 shows the layout of the room and the arrangement of sensors. A binary response was read every 0.5 s from each sensor (sampling rate of 2 Hz).
https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Fig2_HTML.gif
Fig. 2

Layout of infrared sensors

For obtaining another strong piece of evidence for authentication, we used a hard authentication door. At the door, a finger vein must be presented to enter or leave the room [21]. When a person enters or leaves the room through the door, our system knows who he/she is by the registered name.

2.3 Detection area of a sensor

We investigated the basic performance of a single sensor. The actual detection area of each sensor was narrowed by a hand-made cylindrical lens hood with a diameter of 11 mm and length of 20 mm. The detection area on the floor was first examined in the setting shown in Fig. 3. One subject approached from eight different directions to the position just below the sensor. The measurement was repeated three times. The results are shown in Fig. 4. The average detection distance was 43.2 cm, the minimum detection distance was 25.0 cm and the maximum detection distance was 65.5 cm. From these observations, we set the distance between two sensors shown in Fig. 2 to 1.2 m on average.
https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Fig3_HTML.gif
Fig. 3

Detection area adjusted by a paper cylinder

https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Fig4_HTML.gif
Fig. 4

Detectable area measured from eight directions

A geometrical model of a human is shown in Fig. 5 as an ellipse with a long axis of 50 cm and short axis of 30 cm. In the above arrangement of sensors, a person cannot move without being detected by sensors.
https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Fig5_HTML.gif
Fig. 5

Detection area and a human model

2.4 Characteristics of infrared motion sensor system

The use of such infrared motion sensors has the following merits:
  • They are easy to set up (The voltage unit is one and the network connection is made by modular (phone) cables.).

  • They can be set up at low cost ($20/unit).

  • They are not affected by light conditions.

  • They do not require user’s cooperation.

  • Movable obstacles do not greatly affect the detective area.

  • Persons do not feel that they are being observed.

All of these are necessary conditions for soft authentication. In our layout of sensors, multiple sensors can react to one motion of a person. It should be noted that there is a “delay” in response to one motion.

3 Basic evaluation of the infrared sensing network

3.1 Basic performance of the system

We investigated the basic performance of our system. We examined (1) if there is a difference between the ability of detection when walking in a straight line and that when turning left/right, (2) if every sensor is activated when a person passes under the sensor, and (3) how sensors react when a person does not move.

We examined the sensor status when a subject walked along two different routes at a natural speed (Fig. 6). The actual routes that the subject walked were determined subjectively from image sequences of two video cameras differently located only for the purpose of evaluation.
https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Fig6_HTML.gif
Fig. 6

Actual walking routes

The routes and instructions given to each subject beforehand are as follows:
  • Route 1 (Fig. 6a): Enter the door and go to No. 8 according to the indicated route. Stay at No. 8 and type a document on the PC. Then stand up and go to the refrigerator at No. 19 and open the refrigerator door. Walk immediately under the sensors as much as possible.

  • Route 2 (Fig. 6b): Enter the door and walk along the route shown in Fig. 6b. Stay at No. 18 for a while. Walk between the sensors as much as possible. Walk across the area between No.2 and No.3 and the area between No.12 and No. 13 (where there are no sensors).

The sensor responses and the actual paths that the subject walked are shown in Fig. 7. On the vertical axis of Fig. 7, the sensor nodes are renumbered according to the route. The solid line is the actual path. In Fig. 7a, the person after time 60 is lost from both cameras due to the dead angles.
https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Fig7_HTML.gif
Fig. 7

Responses of sensors when a person walks along route 1 or 2. The order of sensors is reordered according to the route

The following can be observed in Fig. 7.
  • The sensors respond correctly in accordance with the actual path followed.

  • A sensor starts reacting 0.5 s before the subject passes immediately under the sensor and remains active for 3 to 4 sec. after passing under the sensor.

  • The activated sensor becomes inactive when the subject stays at one position and does not move (Sensor No. 18 in Fig. 7b).

  • The sensors react correctly even if the subject turns left or right.

  • When the subject is doing a light job without a large motion, the sensor becomes inactive (Sensor No. 8 in Fig. 7a) but is reactivated when the subject performs a large motion (opening the door of the refrigerator) (Sensor No. 19 in Fig. 7a).

  • There is no undetectable area even if the subject walks in such a way to avoid sensors (Nos. 2, 3, 12 and 13 in Fig. 7b).

  • At each sampling time, 2.66 sensors become active on average.

All of these guarantee that the performance of this system is sufficient for detecting a walking person.

4 Tracking algorithm

We propose an algorithm for tracking persons on the basis of previous observations. In principle, two persons cannot be distinguished when they meet at one place. When this occurs, we divide their personalities. For example, when persons A, B and C get together at the same place, we replace them with three virtual persons who each have multiple personalities consisting of 1/3 of A, 1/3 of B, and 1/3 of C. By this division of personality, the total number of persons is kept as three. In this way, we have ambiguity but can keep tracking them. In the following, the position of a person is discritized to one of sensor positions. In addition, the symbol ′ denotes one time step before and the symbol ″ denotes two time steps before.

4.1 Terminology

Before describing the algorithm, we present terminology. In the following, we consider a connected region as the set of active sensors that are connected in one of eight directions.

(Terminology)
sS, zs = (xs, ys)

sensor set S and position zs = (xs, ys) of sensor s

TS

set of active sensors

T′ ⊂ S

set of sensors that were active one time step before

p ∈ P = {1,2,...,n}

set P of registered persons p

L

minimum time step to judge as long stay

zp = (xp, yp)

estimated position of person p at the present time

zp = (xp, yp)

estimated position of person p one step before

zp′′ = (xp′′, yp′′)

estimated position of p two steps before

lp

number of time steps of p staying at zp

Q = {1,2,..., m}⊂ P

set of persons in the room

Q

set of persons who were in the room one step before

4.2 Algorithm

The tracking algorithm is described using the above terminology. The outline is as follows. First, the initialization step is carried out for every person to be outside the door. When a person enters, tracking starts. The basic strategy at time t is as follows: (1) find active sensors and cluster them as non-connected regions, (2) for an existing person p, choose the nearest cluster Tv of active sensors, (3) according to the past trajectory of the person, zp′′zp, at times t − 2 and − 1, choose the most probable position zp ∈ Tv at time t, referring to the location-dependent trajectory probabilities conditioned by zp′′ → zp. When there are more than one person at the same position, a “multiple personality process” is carried out. In this process, each person is divided into the same number of personalities as the number of the persons at the position. After this process, some persons have multiple personalities. When someone has left the room, that person’s personality is removed from all of the remaining persons. In addition to this basic strategy, some auxiliary processes are carried out: one process is for assigning the nearest person to an empty sensor cluster, and another process is for clearance of multiple personalities by a stronger piece of information (e.g. personal desk information).

The concrete algorithm is as follows.

Algorithm
  1. (0)
    Initialization
    https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Fige_HTML.gif
     
  2. (1)
    Initialization of the person entering through the door
    https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Figf_HTML.gif
     
  3. (2)
    Update process
    https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Figg_HTML.gif
     
  4. (3)
    Multiple personality process
    https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Figh_HTML.gif
     
  5. (4)
    Recovery process
    https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Figi_HTML.gif
     
  6. (5)
    Update of record
    https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Figj_HTML.gif
     
  7. (6)
    Process with other pieces of evidence
    https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Figk_HTML.gif
     
  8. (7)

    Go back to (1)

     

Here, in Step 6, there is an option when personal desk information is available. In this algorithm, judgement of the position of a person at a specified time is carried out by taking into account the person’s multiple personalities. That is, for a question such as “Where is p now ?”, we have an answer such as “p is at position z1 with probability of 1/2, at z2 with probability of 1/4, and at z3 with probability of 1/4.”

4.3 Prediction probability \(P_{c(z^\prime)}(z|z^\prime, z^{\prime\prime})\)

On the basis of position z′ at one step before and z′′ at two steps before, we change the prediction probability \(P_{c(z^\prime)}(z|z^\prime, z^{\prime\prime})\) to predict the current position z. We also exploit knowledge about the obstacle location. We change the probability depending on the obstacle location c(z′) at z′. It is noted that the values of z, z′ and z′′ are discretized on one of the sensor positions.

In the preliminary experiment, we confirmed that a person’s walking speed ranges from 1.0 to 1.5 m/s, and we therefore concluded that a person moves at most 0.75 m (=1.5 m/s × 0.5 s) at one time step (0.5 s). Since the distance between sensors is more than 1.0 m, it is sufficient to consider nine neighbor positions of z′ as candidates of next position. Therefore, we consider only 9 neighbor positions around z′.

Adopted empirical knowledge in the probability is as follows:
  1. 1.

    A person tends to walk straight rather than turn.

     
  2. 2.

    A person seldom turns suddenly at a right angle or turns back.

     
  3. 3.

    When a person stays for a long time at one position (maybe on a chair), the person tends to stay longer at the position.

     
Considering these pieces of knowledge, we determined the prediction probability P(z|z′, z′′) as shown in Fig. 8. In Fig. 8, the trajectory z′′ → z′ is fitted to one of the three possible directions (solid arrows). The three kinds of prediction probabilities correspond to the three cases: (1) coming from a vertical or horizontal direction, (2) coming from a diagonal direction, and (3) staying at a position.
https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Fig8_HTML.gif
Fig. 8

P(z|z′, z′′) according to trajectory z′′→ z′ (solid arrow). The trajectory is arranged to one of three directions: a comes straight (from bottom to center), b comes diagonally (from left behind to center) and c stays (z′′ = z′)

When obstacles (e.g. desks, tables) exist near position z′, we reset by zero the probability values of the positions where obstacles exist and normalize the remaining. This process is carried out depending on the position z′, so that P(z|z′, z′′) comes with suffix c(z′). It is noted that the absolute values of the probability have no meaning and only the ranking is meaningful. If a tie happens at the highest value of probabilities, we break it by a random choice.

5 Experiments

5.1 Trackablity in different situations

We investigated in which cases our system can correctly track persons. The results were divided into three cases: (1) trackable cases, (2) trackable cases with some ambiguity of multiple personalities, and (3) untrackable cases in principle (Fig. 9).
  1. 1.
    Correctly trackable cases (Fig. 9a):
    • Some persons are moving while keeping a distance ((a)-1).

    • One person passes behind another sitting with a motion ((a)-2).

    • Two persons move in opposite directions and cross in the middle ((a)-3).

     
  2. 2.
    Trackable with ambiguity cases (Fig. 9b):
    • Some persons meet at one place and leave, after a while, in order or at the same time ((b)-1).

    • Two persons are sitting at a close location and are working at their desks with large motions ((b)-2).

    • More than one person move together ((b)-3)).

     
  3. 3.
    Untrackable cases (in principle) (Fig. 9c):
    • A person passes through a group of persons ((c)-1).

    • Two persons come close, one stops, and then they move in different directions ((c)-2).

    • One person passes another who is working motionlessly at a desk. ((c)-3).

     
The limitation of this system is seen in “untrackable cases.” In case (c)-1, a person passes by a group. The movement of the person is correctly detected by the sensors. However, the person is judged to have joined the group because the group has a large active region. As a result, the tracking fails. In this case, the system chooses a wrong direction in P(z|z′, z′′). In case (c)-2, two persons approach one place by different routes. A problem occurs when one person remains standing in one position for a while. That person is not detected by the sensors. Thus, the lost person is judged to have moved to another active region. Case (c)-3 shows a typical situation in which the system fails to track the person. In this case, a person is working at a desk. However, the person is sitting still almost motionlessly and the sensor above the person therefore does not become active. When another person passes behind that person, the system judges that the sitting person has moved to the region activated by another. As a result, the sitting person is completely lost. In the algorithm, a person is judged to have moved to the nearest active region when the person is not detected by the sensors above the person. Pieces of knowledge, such as a sitting person tends to stay rather than leave, are embedded in the algorithm (Fig. 8c). However, it does not work when the person is motionless.
https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Fig9_HTML.gif
Fig. 9

Three cases of a trackable, b trackable with multiple personalities and c untrackable. A gray region indicates a connected region of active sensors and a capital letter indicates a person. A person enclosed by a solid circle is the predicted position of the person. When a person is not enclosed by a solid circle, the person is lost by this system. (a)-1 Two persons move apart, (a)-2 one person is working at a desk and another passes behind the person, (a)-3 two persons cross. (b)-1 Two persons meet and leave, (b)-2 two persons are in motion at closely located desks, (b)-3 two persons walk together. (c)-1 Three persons meet and another person passes them, (c)-2 two persons come close, one has a break (no active sensor), and they move independently, (c)-3 one person passes behind another who is motionless at a desk

Most of the faults arise from the fact that we cannot distinguish two different situations from the sensor reading: (1) a sensor is inactive because no one is there and (2) a sensor is inactive because the person under the sensor is still.

5.2 Tracking in general cases (Practical cases)

Finally, we tested the system in a real situation in which up to five persons spent a few hours in the room without any instructions. They behaved naturally because they were not informed that the system was in operation.

The experiment was conducted 1 day at the end of a semester. During a 2-h period around noon, 7 persons (including the same persons) entered and left the room. For about half of the period, they walked around, moved or left/(re)entered, while, in the other half, they stayed at their desks. The “long stay” parameter L was set to L = 8 (4 s).

The results are summarized in Figs. 10, 11 and Table 3 (The data were taken every 0.5 s). Figure 10 shows (a) the number of persons at each time, (b) the highest score (personality rate) of the correct personality of persons who were correctly tracked (Best), and (c) the lowest score of the correctly tracked persons (Worst).
https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Fig10_HTML.gif
Fig. 10

Number of persons and tracking accuracy. “Best” line (solid line) shows the highest score of the personality scores of the correctly tracked persons and “Worst” line (dotted line) shows the lowest score of the personality score of the correctly tracked persons. Interpretation of A–F is given in Table 3. Trajectory of people in some intervals is shown in Fig. 11

https://static-content.springer.com/image/art%3A10.1007%2Fs10044-008-0119-9/MediaObjects/10044_2008_119_Fig11_HTML.gif
Fig. 11

Trajectory of movements of people. Different colors show different persons

Table 3

Relation between tracking/identification accuracy and actual events

Point or Interval

Identification Precision

Actual Event or Observation (observed by cameras)

A

Best < 1.0

Two persons enter in succession.

B

Precision up

One person leaves.

Int. C

Worst = 0.0

At least one person is lost.

D

Instantaneously Worst = 0.0

A person is lost but is recovered soon.

Int. E

Worst = 0.5

Every person is tracked, but they have a half personality.

F

Best becomes 1.0

Increase in precision by personal desk information

Int. G

Worst = 1.0

Success in perfect tracking

Int. H

Worst ≃ 1.0

Almost perfect tracking

In Identification precision, “Best” shows the highest score of the personality scores of the correctly tracked persons and “Worst” shows the lowest score of the personality score of the correctly tracked persons

For example, when at least one person is successfully tracked as a single personality, then score Best is one. When person A is successfully tracked but A has multiple personalities including 1/3 of A’s personality, the score is 1/3. Best becomes 1/3 when A is tracked with the best score. Worst shows the worst score among them. If there is a person who the system fails to track even if multiple personalities are taken into account, then score Worst is zero. The predicted position is judged as “correct” when it is located within 9-neighbors of the true position observed by video cameras. In addition, a short delay (1 step–2 steps) for prediction is allowed.

In Fig. 10, two different cases are shown: (a) when only enter/leave information at the door is used as well as sensor responses (Step 6 in the algorithm: process with other pieces of evidence) and (b) when private desk information is added (“optional” in Step 6).

We confirmed the following:
  1. 1.

    For two thirds of the observation time period, the system does not loose any person for up to five persons even if only enter/leave information is used (in Fig. 10b, the time length accumulated except for C (time interval with lost persons) is 82 min/120 min). Furthermore, the system succeeded in tracking persons almost perfectly for up to three persons (in Fig. 10b, the time length accumulated except for C is 84 min/101 min for at most three persons.).

     
  2. 2.

    If private desk information is used in addition to enter/leave information, the system almost perfectly tracks all persons (in Fig. 10c, the time length of interval H (almost perfect tracking) is 115  min/120 min.). In this case, the ambiguity of personalities is almost 1/2.

     
What happened during tracking is shown in Fig. 11a–d. The trajectory for the entire time period is shown in Fig. 11a. Almost all persons enter through the door and walk straight to their own desks through the central path. Sometimes a person approaches a refrigerator, a printer, etc. and sometimes visits other persons. In Fig. 11b (time frames 2583–2805), one person visits three persons to talk in order. In our system, a sitting person is taken away by another in this situation, especially when the sitting person is sitting still. In Fig. 11c (time frames 2955–3093), one person walks around and visits another. After a while, two persons leave for lunch. In Fig. 11d (time frames 4311–5007), two persons enter together and, after a while, another two persons enter, indicating friends act together. In these situations, the tracking system is easily confused and looses the perfect personalities of these persons.

Figure 10 shows that degradation of precision occurs impulsely several times. This means that some persons were lost but were soon discovered.

6 Discussion

In the proposed infrared ceiling sensor system, the tracking/identification precision decreases with passage of time due to the multiple personality process. This type of degradation of precision is unavoidable in principle as long as infrared sensors are used. However, as has been shown, such degradation can be recovered if some other pieces of evidence, such as enter/leave information, private desk information, and identification information by sensor-attached chairs, are occasionally available.

The current precision level of our system is not high enough. However, the precision level can be considered as a standard level as long as such an infrared ceiling sensor network is used. It might seem a better idea to deliver the personality of everyone in the room evenly to all of the active sensor regions. Indeed, in this way, we could avoid the worst case in which all persons are lost, as seen in interval C of Fig. 10b. However, there are cases for which this strategy would not work. Let us assume that there are m active sensor regions for n persons. Unfortunately, in general, mn holds. This is because some sensors can be inactive for motionless persons and a sensor region can include more than one person meeting at one place. In the former case, it is inevitable that some persons are lost in tracking. In addition, it is difficult to treat the case in which two persons cross. In this naive assignment of 1/n personalities of everyone, we cannot obtain a score of Best higher than 1/n.

In the early stage of our study, we tried to use a Bayesian network for prediction of the user’s current position [22]. However, we noticed that it was difficult to embed into the algorithm several kinds of knowledge about the difference in trajectories and the difference in layout of obstacles. To learn the probability of possible movements depending on the locations, many training data are necessary. Therefore, we decided to use a simpler knowledge-based probability.

In this study, we have used knowledge about a single person’s walking trajectory. One way to improve the tracking/identification rate is to use knowledge about more than one person. Indeed, an untrackable case (c)-3 is solvable by taking into account knowledge such as “If a person is judged as (motionlessly) stay at one step before and another person passes behind the person, then the first person is judged to be staying still and the second person passes.” However, such a strategy is not easy to realize because there are many cases to be considered as knowledge. Another way to improve the precision is to use the volume and configuration of an active region. The size and shape of an active region may change according to the number of persons and their movements.

It is possible to use more sensors with narrowed beams to raise the detection resolution. However, the set-up cost is one of the important issues in soft authentication. In this sense, we used as few sensors as possible for detection in a large room. The cost of this network system is about $1,000 (=$20 × 50 units).

Obviously, as long as infrared sensors are used for tracking, it is impossible to distinguish two persons standing in one place. As a result, the ambiguity of multiple personalities is inevitable. Therefore, as described in the introduction, we implicitly assume the usage of multiple pieces of evidence. For this goal, we have been developing a chair with pressure sensors for identifying the person sitting on the chair, and we have achieved a correct identification rate of over 70% for up to 10 persons (over 90% for 10 relaxed persons) [9]. Such a system is effective for identifying persons at a certain place with a high rate of correct identification, while our network system is effective for identifying persons anywhere with a low rate of correct identification. Therefore, complementary usage of these systems would be more effective. In this study, we used private desk information as a complementary piece of evidence. In a house, we could use private room information and/or personal time schedule. At an office, information on individual conference schedule, regular arrival/leaving time and personal holidays could be used.

7 Conclusion

As a system that realizes “soft authentication” with the aim of providing personalized services instead of maintaining a high level of security, we have proposed an infrared ceiling sensor network. We have shown that it is possible to track once-identified persons almost perfectly for up to five persons in a room. Our main goal is to provide services specialized for each user at an appropriate place, e.g. in front of a TV set, a refrigerator, and a microwave oven. With this system, we can know where a specified person of a small group is, although some candidate positions might be presented. The tracking/identification precision could be improved by using some other pieces of evidence such as private desk/room information and personal schedule. This system can also be used for monitoring persons continuously or for detecting an invader.

Copyright information

© Springer-Verlag London Limited 2008