Inhabitant Prediction in Smart Houses

Rachid Kadouche
Universit e de Sherbrooke
Sherbrooke, Qu ebec, Canada
Rachid.Kadouche@
usherbrooke.ca
Bessam Abdulrazak
Universit e de Sherbrooke
Sherbrooke, Qu ebec, Canada
bessam.abdulrazak@
usherbrooke.ca
ABSTRACT
This paper addresses the inhabitant prediction issue in smart
houses based on daily life activities. We use data provided
by non intrusive sensors and devices to predict the house
occupant. Support Vector Machines (SVM) classier was
applied to build a Behavior Classication Model (BCM) and
learn the users habits when they perform activities for pre-
dicting and identifying the house occupant. The model was
tested using data coming from the Washington State Univer-
sity smart apartment tesbed and data from experiment held
with six users at the DOMUS apartment. The BCM model
results was also compared with a frequency based approach.
Author Keywords
Inhabitant Recognition, Smart Houses, Support Vector Ma-
chines
ACM Classication Keywords
I.5.0 Computing Methodologies: PATTERNRECOGNITION.
General Terms
Algorithms, Design, Human Factors, Performance, Security
INTRODUCTION
Smart houses [4], [17], [3] are becoming a viable option for
people with special needs (PwSN
1
) who would prefer to stay
in the comfort of their homes rather than move to a health-
care facility. This support includes facilities for environmen-
tal control, information access, communication, monitoring,
etc., and built over various existing and emerging technolo-
gies. However, PwSN have a large variability in their needs
and levels of cognitive and/or motor handicap. Hence, they
call for adapted services, especially when interacting with
their environment. The task of environment personalization
becomes more complicated in the case of multiple inhabi-
tants in the house. According to [21], over half of the el-
derly live with their spouse, and one third live alone assisted
by residential care. In both cases, the house should be able
1
PwSN: People with disabilities and elderly
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for prot or commercial advantage and that copies
bear this notice and the full citation on the rst page. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specic
permission and/or a fee.
SAGAware11, September 18, 2011, Beijing, China.
Copyright 2011 ACM 978-1-4503-0926-4/11/09...$10.00.
to correctly distinguish between multiple inhabitants before
any other operation. The system should predict and identify,
among a group of people, who is currently present in the
house in order to provide him a personalized service.
In case of PwSN, the identication system should take into
account the omission errors of this population particularly
when they are involved by themselves in the process (using
a badge, introducing a password...). In this case, the risk of
missing the identication task is high. However, the identi-
cation should be automatic. It should not intrude or disturb
the user, but should quietly support them. In addition, sev-
eral constraints should be considered to build such a system:
Occupants privacy has to be preserved. For instance,
the use of video cameras and microphone are not recom-
mended.
The sensors used should be economic, designed to be quickly
and ubiquitously installed and easy to dissimulate in the
house to provide a familiar environment.
Avoid the use of wearable tags. This reduces anxiety and
the feeling of being constantly monitored.
In this paper, we present a smart house occupant prediction
model based on basic activities that people undertake each
day, such as having breakfast, grooming, etc. The particular-
ity of our approach remains in the use of non intrusive sen-
sors and devices and apply a supervised learning technic to
classify users by their habits when they are performing their
daily activities. In this work, we prove that different users
can have their own way of carrying out an activity. Thus,
they can be easily identied using a prediction technique.
BACKGROUND AND RELATED WORK
Blind users prediction is under-researched area of work. It
uses an effective means to predict the users without per-
sonal information. It denes an alternative method of au-
thentication in smart house systems. Floor sensor settings
have been used for user prediction in smart environments.
The authors in [16] and [5], respectively use the nearest-
neighbour and hidden Markov models (HMM) methods, for
a footstep prediction based on a small area of ground re-
action force sensors. UbiFloor [10], based on neural net-
work classier, use binary type ON/OFF sensors to detect
the characteristic walking styles of the user based on both
single footsteps and walking from consecutive footsteps on
the oor. In these systems, to perform a high resolution of
the prediction process, the house should be equipped with
27
a large number of sensors. Thus, the cost of the predic-
tion should be high. Other prediction systems are based on
gait recognition. Gait-based methods use the modelling se-
quence of walking using cameras [14]. Continuous hidden
Markov models [7] and eigenspace transformation combined
with canonical space transformation [14] have been applied
to predict a user among a group. These technics suffer from
occlusion, differences in background movements, and light-
ing conditions.
Physiological measurements such us biometrics are also used
for user prediction in smart environments. They refer to
automatic recognition of people based on their distinctive
anatomical (e.g., face, ngerprint, hand shapes, iris, retina,
hand geometry)[11, 12]. These systems have not yet carried
out automatic human recognition. They are sensitive to the
environmental factors like shadow, light intensity and obsta-
cles.
Although prediction systems presented above present a ex-
ible and natural way to identify the inhabitant without any
wearable devices in smart environments, these solutions have
many disadvantages. They are either costly or extracted fea-
tures are sensitive to environmental factors and the record-
ing devices are not always hidden from the user (e.g. cam-
era) which can compromise the individuals privacy. Our
approach uses behavioral modelling based on users habits
inside the house. The key advantage of our approach is that
it can uses any imbedded sensors already mounted in the
smart house to detect the inhabitant.
OUR APPROACH
We use supervised learning technics for predicting a person
from inhabitant group (classication). This task predicts the
house occupants after they have performed, many times, par-
ticular daily activities (e.g. grooming, eating, having break-
fast, ...) inside the house. We rst dene the activity tar-
geted for evaluation, and then learn the habits of each user
by training a Support Vector Machines (SVM) classier over
the set of data received from sensors and actuators provided
by the users when they perform the activity (see gure 1).
The SVM classier builds a learning model, called Behavior
Classication Model (BCM), which will be able to separate
one user from others. BCM will class each user by his/her
behavior to perform a particular activity and be able to pre-
dict the house occupant (see gure 2).
To evaluate the BCM, we analyzed the early morning routine
activity. Six users at the DOMUS apartment [1] have par-
ticipated to the experiment. In addition and to conrm the
efciency of our approach, we analyzed the dataset of the
Washington State University smart apartment tesbed, which
is part of the CASAS smart home project [18].
Support Vector Machines (SVM)
SVM are powerful classication systems based on regular-
ization technics with excellent performances in many practi-
cal classication problems[20]. SVM deliver state-of-the-art
performance in real-world applications such as text catego-
rization, hand-written character recognition, biosequences
analysis, image classication, etc. It is now established as
one of the standard tools for machine learning and data min-
ing. The SVM decision function is dened as follows:
f(y) =
N

i=1

i
K(x
i
, y) +b (1)
Here y is the unclassied tested vector, x
i
are the support
vectors and
i
their weights and b is a constant bias. K(x, y)
is the kernel function introduced into SVM to solve the non-
linear problems by performing implicit mapping into a high-
dimensional feature space [19, 8].
We have applied SVM to build the BCM for learning the
users habits when they perform their activities inside the
house. The BCM data input is a matrix dened by a set of
vectors, each vector, called pattern, is composed of n com-
ponents which we name features. Features are the apartment
sensors states. Patterns correspond to the users actions and
movement in the smart house. It denes the involved sensors
when the user was performing the activity. Features have bi-
nary values (1 and 0). The value 1 (resp 0) means that the
corresponding sensor have been involved (resp not been in-
volved). We have to note that patterns do not give any tem-
poral order of the actions performed by the user. Particulary
in this work, the temporal constrain is not taken into account.
Each vector was labelled by the feature Class which denes
the user who is performing the activity.
EXPERIMENTATION
In this work we have focused only on the early morning
habits. Particularly, we assume that we are monitoring a
group of peoples living together where the morning wake
up time is different from one another. Each person at a time
does the early morning routine alone in the apartment. This
choice is due to being unable to detect the activity of one
person among a group performing altogether activities in-
side the house. Thus, we rst conduct a series of experi-
ments at DOMUS apartment and evaluated six users. Then
analyzed the publicly available dataset from the Washington
State University smart apartment tesbed.
DOMUS
The DOMUS laboratory [1] includes a standard apartment
(kitchen, living room, dining room, bedroom and bathroom)
located within the computer science department of the Uni-
versity of Sherbrooke and equipped with a set of infrared
sensors, pressure detectors, smart light switches, electrical
contacts on doors, audio and video systems, as well as smart
tags (RFID) to obtain, in the apartment, the real-time posi-
tion of the user and objects.
The Sensors
In this work we have considered 36 imbedded and unobtru-
sive sensors which are already mounted in DOMUS apart-
ment, they allow us to cover a large space of potential sen-
sors. Each sensor have two states (open and close) that makes
a total of 72 features. Six zones are dened to cover the dif-
ferent apartment area (see gure 3). The number of installed
sensors varies depending on the interested zone. Table 1 de-
nes their disposition per zone. The following list gives the
details of each sensor.
28
Figure 1. Behavior Classication Model(BCM)
Figure 2. Users prediction
Entrance hall Living room Dining room Kitchen Bathroom Bedroom Total
IR 0 1 1 3 0 0 5
Pressure Detector 1 0 0 0 0 0 1
Lamps 0 1 1 1 1 1 5
Door contacts 0 0 0 0 1 1 2
Switch contacts 0 0 0 19 0 0 19
Flow meters 0 0 0 0 2 2 4
Table 1. List of the DOMUS sensors used per zones
29
Figure 3. Domus apartment
30
Infrared (IR) movement detectors: they provide the users
location in a zone. They cover a zone or a part of a zone.
For example there is only one IR detector that covers the
entire zone in the dining room and living room (salon),
whereas three are installed in the kitchen covering oven,
sink and toaster.
Pressure detector: in form of tactile carpets placed on en-
trance hall, it detects the user moving between the bed-
room and living room. The users can use two paths to
move around these two zones are through the kitchen or
through the entrance hall.
Lamps Light switches: these sensors send an event every
time the occupant turns the lights on or off.
Door contacts: these sensors are placed on the doors. They
send an event related to the door state (open or close).
Switches contacts: the same as door contact, they are placed
on the lockers and fridge. They provide an event when
their state is changed either opened or closed.
Flow meter: they provide the taps and the ush toilet stats,
two are mounted on the cold and hot water taps of the
kitchen sink, one is mounted on the washbasins cold wa-
ter tap and another in the ush toilet. They send an event
when the tap is opened or closed and the ush toilet used.
The Experiment Scenario
Six adults have participated in the experiment to evaluate
the early morning habits (wash up, having breakfast). The
experience held in 2 times at DOMUS apartment. In the rst
time (serie 1), the user was asked to perform the early morn-
ing routine as he is used to do at home (see gure 4). In
the second time (serie 2), he was asked to repeat the same
routine where a constraint was introduced during the experi-
ment. This constraint which joined another study conducted
by colleagues in the DOMUS laboratory, consist on learning
a tea recipe which takes at most 10 minutes. In serie 1 the
user cames 10 times to the laboratory, ideally in two consec-
utive weeks. After 2 weeks break, the user starts the serie 2,
he was asked to come 5 days, ideally in one week. In both
series, the user was free to use any equipment available in
the apartment. The experiment time was about 45 minutes.
The experiment starts with the same apartment conditions
for all the users, all doors closed and lights switched off. The
user is asked to stay in the bedroom for one minute (time re-
quired to start data recording). Each user experiment denes
a sample data which represents a pattern, this makes a total
of 60 samples for serie 1 and 30 samples for a serie 2. The
data are available to download here [2]
Data Preprocessing
A preprocessing step is used to select, from the original val-
ues, the DOMUS data subset that is used to construct the
BCM matrix. We faced many problems during the experi-
ment providing incorrect samples. These is due to technical
(blackout during the experiment, server shutdown, ...) and
experimental problems (user begins the experiment before
start recording the data). We reported 16 incorrect samples
which are eliminated in this phase. To recover this data, we
need missing data technics [15] which is another issue that
have not been addressed in this paper.
In this paper we used the data of serie 1 as a training set (to
learn a classier) and a test data set (to estimate the perfor-
mance of the classier).
BCM matrix example
Table 2 represent ve patterns dening the activities that
user U1 performed in the DOMUS apartment during ve
days. Each sensor have two states. For visibility, in this
example we show only 4 sensors states from 72. Thus, the
sensor bedroom-lamp have the states open and close. We
can interpret the rst vector (day1) as followed: the user
U1, switched on the bedroom light (Lamp-Bedroom-Open),
opened and closed the bedroom door (Door-Bedroom-Open,
Door-Bedroom-Close).
CASAS Smart Home Project
The CASAS
2
Smart Home project is a multi-disciplinary re-
search project at Washington State University. The CASAS
project is focused on the creation of an intelligent home en-
vironment aiming at minimizing the cost of maintaining the
home and maximizing the comfort of its inhabitants. The
testbed smart apartment which is part of the CASAS smart
home project [6] is a three bedroom apartment that includes
three bedrooms, one bathroom, a kitchen, and a living / din-
ing room. The testbed apartment is equipped with different
kinds of sensors and actuators to sense the environment and
give back information to inhabitants accordingly.
Sensors description
The testbed smart apartment is equipped with the following
sensors categories.
Motion sensors: motion sensors are positioned on the ceil-
ing approximately 1 meter apart throughout the space. Mo-
tion sensors are labeled from M01 to M51, and have two
states: ON, OFF.
Temperature sensors: these sensors are installed to pro-
vide temperature readings, and are labeled Txx. These
sensors have numeric value.
Door sensors: these sensors are used to detect the state of
doors, and are labeled Dxx. They have two states: OPEN,
CLOSE.
Item sensors: are used for selected items in the kitchen,
and have two states: PRESENT, ABSENT.
Light controllers: are used to control lights in the testbed
apartment, and are labeled Lxx. These controllers have
two states: ON, OFF.
In addition to the above sensors categories, the testbed
apartment is equipped with a burner sensor labeled AD1-
A, a hot water sensor AD1-B, a cold water sensor AD1-C,
and electricity usage sensor labeled P001. All these sen-
sors have numeric values.
2
http://ailab.eecs.wsu.edu/casas/
31
Lamp-Bedroom-Open Lamp-BedroomClose Door-Bedroom-Open Door-Bedroom-Close Class
Day 1 1 0 1 1 U1
Day 2 0 0 0 1 U1
Day 3 1 1 1 1 U1
Day 4 0 0 0 1 U1
Day 5 1 1 0 1 U1
Table 2. BCM matrix example of DOMUS sensor data
Figure 4. User performing the early morning routine at the Domus appartment
Date Time Sensor Name State / value
2009-02-02 12:18:44 M16 ON
2009-02-02 12:18:46 M17 OFF
2009-02-02 12:28:50 D12 OPEN
2009-02-02 12:29:55 I03 PRESENT
2009-02-05 08:05:52 AD1-B 0.0448835
2009-02-05 12:21:51 D09 CLOSE
2009-02-10 17:03:57 I03 ABSENT
Table 3. Example of the CASAS Smart Home Project sensor data
32
Each sensor data has associated features such as sensor name,
value or state, and temporal information which indicates when
the sensor is triggered. An example of sensors data collected
during this experiment is given in table 3.
Sensors are installed in well dened zones according to their
types and the experiments being performed. The general dis-
position of the testbed sensors is shown in gure 6.
Data preprocessing
The dataset collected from the testbed smart apartment can
not be used directly in our approach. Hence, a preprocess-
ing step is required to analyze these data. All the data col-
lected in the CASAS testbed apartment are recorded in one
le. We rst classed the data by user, which provide us one
le for each user. Then we extracted the data of the eval-
uated activities from each le. We then converted CASAS
sensors states to features, taking into account the different
states value ( ON, OFF, CLOSE, OPEN, etc). Ajava applica-
tion is implemented for this purpose to construct the patterns
and build the BCM matrix.
Testbed activities description
Activities performed during this study were achieved in the
testbed smart apartment during spring of 2009 over three
months. The apartment housed two residents at this time
and they performed their normal daily activities (see gure
6), which correspond to the basic everyday tasks of life such
as sleeping, eating, bathing, dressing, cleaning and toileting.
The list of activities involved during this study are: Bed to
toilet, Grooming and Having breakfast.
BCM matrix example
Table 4 represents an example of the processed data for the
Grooming activity with some light controllers states. In
this example, each day represents a pattern. In line 2 (Day
1), the user U1 has switched on the lights L01, L02 and
L03 and switched off the light L02. The light L04 was not
activated. Here, in this example, we can remark that, for the
ve days, the user U1 switches on the light L01 an does
not switch it off. This habit belongs to the users behavior
when he performs the grooming activity. It is the kind of
information which are extracted by the BCM to perform user
classication for prediction.
RESULT AND DISCUSSION
BCM Accuracy Results
We trained the SVM classier to build the behavior classi-
cation model (BCM). The accuracy of the BCM was tested
using 10-fold cross-validation [13]. Figures 7, 8 shows re-
spectively the results of the BCM accuracy over various ex-
periment time period for DOMUS data and the three activi-
ties tested using the CASAS Smart Home Project data. For
the DOMUS data the accuracy is over then 80%. The best
score is recorded at the tenth minute (90%).
Dataset Correctness (%)
Episode Length
E=3 E=4 E=5
Domus 84.16 82.69 72.72
Breakfast(CASAS) 62.50 66.66 60
Grooming(CASAS) 47.50 55 59
Table 5. Frequency based approach for Inhabitant identication ac-
curacy per activities using DOMUS and CASAS Smart Home Project
data
Regarding the CASAS Smart Home Project data, for the
Bed to Toilet activity, the accuracy is 100%fromthe fourth
minute of the experiment time. For the grooming activity,
the best score is recorded at the twentieth minute (99%). Fi-
nally, for the having breakfast activity, the users are cor-
rectly predicted for an accuracy of 79% at the tenth minute.
Although the morning routine, tested in this work, is a com-
plicated activity, and usually, several users have the same
habits to perform this activity, the behavior classication
model (BCM) succeeds, with a high precision, to correctly
predict and discriminate the users. This results prove that the
users tested in this work have different behaviors and each of
them have his own manner to perform the morning routine
activity.
Frequency based approach Accuracy
The data of both testbeds (DOMUS and CASAS Smart Home
Project) were tested using a frequency based approach [9]
with three episode lengths. The results of these tests are
shown in table 5.
Fore the three episode lengths tested, we can observe that
frequency based approach achieve relatively less accuracy
than our model (less then 90% for all activities evaluated for
both testbeds)
This implies that sequential information about the occur-
rence of events is not relevant to predict inhabitant.
CONCLUSION
In this paper, we present our ongoing work on the house oc-
cupant prediction issue based on daily life habits in smart
houses. Our approach is based on supervised learning tech-
nics. We used Support vector machines (SVM) to build be-
havior classication model (BCM) for learning the users
habits. We have analyzed the early morning routine with
six users at the DOMUS apartment and two users from the
publicly available dataset of the Washington State University
smart apartment tesbed. The results showed that the user can
be recognized with a hight precision which means that each
user have his own way to perform this activity. Our approach
is signicantly accurate than the frequency based approach.
As future work we are studying the users patterns which al-
lows a person to be discriminated and recognized among a
group performing altogether activities in the same environ-
ment without using intrusive technologies.
33
Figure 5. Sensor layout of the testbed smart apartment
Figure 6. User performing the early morning routine at the the Washington State University smart apartment tesbed
L01ON L01OFF L02ON L02OFF L03ON L03OFF L04ON L04OFF Class
Day 1 1 0 1 1 1 0 0 0 U2
Day 2 1 0 1 1 1 1 0 0 U2
Day 3 1 0 1 1 1 1 1 1 U2
Day 4 1 0 1 1 1 0 0 0 U2
Day 5 1 0 1 1 1 1 1 1 U2
Table 4. BCM matrix example of CASAS Smart Home Project Grooming activity data
34
50
60
70
80
90
100
60 50 20 10 5 1
C
o
r
r
e
c
t
n
e
s
s

(
%
)
Time Period
Early morning routine
Figure 7. Inhabitant identication accuracy using DOMUS data
50
60
70
80
90
100
60 50 20 10 5 1
C
o
r
r
e
c
t
n
e
s
s

(
%
)
Time Period
Bed to toilet
Breakfast
Grooming
Figure 8. Inhabitant identication accuracy per activities using CASAS Smart Home Project data
35
REFERENCES
1. Domus. http://domus.usherbrooke.ca/.
2. Domus. http://domus.usherbrooke.ca/jeux-de-donnees/.
3. Georgia tech aware home research initiative.
http://www.cc.gatech.edu/fce/ahri.
4. Mit nhouse. http://architecture.mit.edu/housen/web.
5. M. Addlesee, A. H. Jones, F. Livesey, and F. S.
Samaria. The orl active oor. IEEE Personal
Communications, 4:3541, 1997.
6. D. Cook and M. Schmitter-Edgecombe. Assessing the
quality of activities in a smart environment. Methods of
Information in Medicine, 48(5), 2009.
7. K. R. Cuntoor, A. Kale, A. N. Rajagopalan,
N. Cuntoor, and V. Krger. Gait-based recognition of
humans using continuous hmms. In in Fifth IEEE
International Conference on Automatic Face and
Gesture Recognition, pages 321326, 2002.
8. R. DEBNATH and H. TAKAHASHI. Kernel selection
for the support vector machine(biocybernetics,
neurocomputing). IEICE transactions on information
and systems, 87(12):29032904, 2004.
9. I. Edwin O. Heierman and D. J. Cook. Improving home
automation by discovering regularly occurring device
usage patterns. In ICDM 03 Proceedings of the Third
IEEE International Conference on Data Mining, page
537, 2003.
10. W.-T. W. J.-H. R. J.-S. Yun, S.-H. Lee. The user
identication system using walking pattern over the
ubioor. pages 10461050, Gyeongju, Korea, 2003.
11. A. Jain, L. Hong, and S. Pankanti. Biometric
identication. Commun. ACM, 43(2):9098, 2000.
12. A. K. Jain, A. Ross, and S. Prabhakar. An introduction
to biometric recognition. IEEE Trans. on Circuits and
Systems for Video Technology, 14(1):420, 2004.
13. R. Kohavi. A study of cross-validation and bootstrap
for accuracy estimation and model selection.
2(12):11371143, 1995.
14. J. Little and J. E. Boyd. Recognizing people by their
gait: The shape of motion. Videre, 1:132, 1998.
15. R. J. A. Little and D. B. Rubin. Statistical analysis with
missing data. John Wiley & Sons, Inc., New York, NY,
USA, 2002.
16. R. J. Orr and G. D. Abowd. The smart oor: a
mechanism for natural user identication and tracking.
In Conference on Human Factors in Computing
Systems, pages 275276, The Hague, The Netherlands,
2000. ACM.
17. A. Pentland. Smart rooms. Scientic American,
274(4):6876, 1996.
18. P. Rashidi and D. Cook. Keeping the resident in the
loop: Adapting the smart home to the user. Systems,
Man and Cybernetics, Part A: Systems and Humans,
IEEE Transactions on, 39(5):949959, Sept. 2009.
19. J. Shawe-Taylor and N. Cristianini. Kernel Methods for
Pattern Analysis. Cambridge University Press, New
York, NY, USA, 2004.
20. V. Vapnik. Statistical LearningTheory.
Wiley-Interscience, September 1998.
21. E. Vierck and K. Hodges. Aging: Lifestyles, Work and
Money. Greenwood Press, 2005.
36