2015 International Conference on Healthcare Informatics
A Smartwatch Software Architecture for
Health Hazard Handling for Elderly People
Rainer Lutze
Dr.-Ing. Rainer Lutze Consulting
Langenzenn, Germany
Rainer.lutze@lustcon.eu
Abstract —A software architecture for smartwatches support-
ing and securing the everyday life for the elderly is presented.
The approach starts from the insight that the usage of smart-
watches for care-giving and handling personal health hazards is a
best practice driven, empirical discipline, requiring a declarative,
easy to maintain software description for the smartwatch devices.
The second rationale for the approach presented is the decompo-
sition of the complex interrelations of “health hazards” into de-
finable modules of manageable size, each addressing the handling
an individual health hazard. The proposed, hierarchical architec-
ture builds on the automatic detection of the activities and events
of daily life (ADLs, EDLs) by the sensors of the smartwatch.
Handling of health hazards will be modeled by UML state ma-
chines, which utilize the detected ADL/EDLs for their state tran-
sitions. The architecture has to take account for a plenitude of
relevant hazards, managed by a multitude of simultaneously op-
erating state machines, via a central synchronizing control me-
chanism.
Keywords — wearables, smartwatches, ambient assisted living
(AAL), assistance systems, health hazards, problem decomposition,
declarative and easy to maintain solution description, automatic
detection of activities and events of daily live (ADLs, EDLs), UML
state machines, 3 layer hierarchical software architecture
I. INTRODUCTION
Smartwatches are about to extend the scope of support for
elderly people, who want to stay in their familiar home in a safe
and self-determined way, as long as this can be justified by
medical reasons. In addition to usual diseases, the health of
elderly people is threatened in multiple ways: e.g. by tumbles,
getting lost during an absence from home, dehydration due to
insufficient liquid ingestion, accidents requiring immediate
external consultation , … . In these situations, suitable smart-
watches attached to the wrist of the wearer can provide effec-
tive assistance in the home and on the go. Smartwatches will do
so by analyzing the pulse rate, movements and location of their
wearer directly via their rich sensorial instrumentation or estab-
lishing a speech connection to a home emergency call center
(HECC) or distant, but available family members. And they can
communicate their findings and advices to the wearer directly
and in a discreet way via their integrated display. This facili-
tates not only the detection of health hazards as for stationary
assistance systems, but also allows for support by providing
orientation information in everyday life.
978-1-4673-9548-9/15 $31.00 © 2015 IEEE
DOI 10.1109/ICHI.2015.50
Klemens Waldhör
FOM University of Applied Sciences
Essen / Nuremberg, Germany
Klemens.Waldhoer@fom.de
The latter is of special importance for elderly suffering
from mild cognitive impairment (MCI) or beginning dementia.
An inconspicuous glance to a smartwatch can deliver the ne-
cessary orientation information in order to compensate the
social implications of the characteristic obliviousness accom-
panying these diseases.
The special potential of smartwatches for supporting elder-
ly is also boosted by the fact that they dissolute the so far ex-
isting social stigmatization of specialized “senior wristlets” or
wireless remote buttons of home emergency call devices [1].
For our solution we have selected the Samsung Gear™ S
smartwatch, which features an easy to read 2’’ OLED display
with 360x480 pixel resolution, and in addition to the usual
movement and pulse rate sensors also a barometer, WLAN and
GPS. The device can operate autonomously from a smartphone
via an integrated 3G mobile radio. The smartwatch will be
tailored to the specific ambient assisted living (AAL) task by
deploying a specifically programmed app to the device, run-
ning under the platform operating system Tizen™. The AAL
app permanently runs also in the background and does not
infer with the standard apps delivered with the watch for dis-
playing the current time, incoming messages and the wearer’s
fitness status, e.g. indicating performed steps, the current pulse
rate.
The present smartwatch characteristics have two important
engineering implications for a smartwatch based AAL assis-
tance system:
• The display size is relatively small, the user will invest not
more than a few seconds to look at it. Thus, the message to
the smartwatch wearer has to be presented in a cognitive
very efficient way and must be comprehensible at once.
• Battery capacity is extremely limited. The AAL app can at
most provide assistance between rising in the morning and
retiring to bed at nighttime; during the night the current
smartwatches mostly need to be recharged and thus cannot
be actively used.
II. PREVIOUS WORK
The principle approach of (AAL) support systems is to detect
the activities and events of daily life (ADLs like sleeping, liq-
uid ingestion, …, EDLs like tumbles, leaving or reentering
home, … ) from the sensorial information acquired (e.g. [6], [7]
for smartwatches, [8] for smartphones).
357
356
From these captured sensor data, the wellness of the wearer of
the smartwatch can be estimated by comparing the actual tim-
ing and duration of ADLs as well as the inactivity between
ADLs against the individual average nominal values learned
from the past [2]. Initially, these nominal values will be ac-
quired from at least a one week training phase of the assistance
system and will be different for the respective days of the week:
workdays, weekends, holidays [3]. Later on, the nominal values
will be adapted by time series analysis with the actual values
measured in subsequent weeks [2].
Specifically, the excess duration of ADLs, excessive inactivity
periods between ADLs and substantial deviations of the circa-
dian rhythm (measured against the nominal values) can be tak-
en as symptoms of decreased wellness of the smartwatch wear-
er. If defined thresholds are passed, and presented questions or
advices, prealerts, to the smartwatch wearer will not be re-
sponded, finally external help will be automatically alerted; the
same is done for critical EDLs (e.g. tumbles) [4], [5].
Fig. 1: Smartwatch Samsung Gear S with app displaying communication and
III. SYSTEM DESIGN CONSIDERATIONS orientation information

A. Software Engineering Requirements
For detected ADLs and EDL, the handling of the potential
health hazards originating from these activities resp. events
involves a lot of empirical, best-practice knowledge, with is
highly susceptible for changes and improvements over time.
This requests for an easy to maintain, declarative way of for-
mal describing and representing of this processing know-how,
from which the smartwatch code finally can be automatically
derived or executed. In addition, the task of dealing with rele-
vant health hazards demands a crisp decomposition of these
complex interrelations into small, definable module of mana-
geable size for each individual hazard and its specific han-
dling, which can be considered separately and computationally
tracked. The simultaneous interference for all these hazards in
reality must be dealt with in an automatic way.
B. Different Target Groups and Their Specific Interests
The AAL app targets younger seniors as well as best age
seniors and those at very high age. Younger elderly people
With respect to the described target group, and the characteris-
typically search protection by the app against sudden accidents
tics of the smartwatch hardware, the app GUI has been de-
or fateful health blows during activities practiced alone – for
signed around the following four principles:
job and leisure, sports or fitness activities. For best age elderly
people, the main interest is support in coping with age specific
diseases (e.g. adult-onset diabetes). Elderly people at very high
age, or their children and/or care-giving relatives, are much
interested in continuous support for the increasing infirmity
and obliviousness.
C. Smartwatch Functionality
Even for this divergent target group, a common core of
support functionality for the app has been identified:
• Communication. Manual calls for external help have
to be placed very easily and at any time.
• Orientation. Information regarding the current time,
day of the week, date, month and year as well as holi-
days, birthdays of the smartwatch wearer and his rela-
tives shall be provided.
• Localization. Absence from home as well as the leav-
ing of a previously vicinity around the home needs to
be handled. When leaving home, it shall be possible to
automatically issue a command to a home automation
system to power down still active loads (e.g. electric
stoves). When the wearer leaves an agreed vicinity
around his home, a situation not infrequent in the pres-
ence of dementia, the wearer shall be tracked and get
increasingly obtrusive advice to return to his home.
• Detection of health hazards. The following hazards
shall be managed based on detected ADLs/EDLs: i) the
excess duration of ADLs, ii) excessive inactivity be-
tween ADLs as well as iii) substantial deviations of the
used circadian rhythm. The app continuously iv) looks
for tumbles and v) monitors the regular fluid intake by
the smartwatch wearer.
D. Interaction Design Principles
1. All and any information presented to the user in a spe-
cific dialogue situation is comprised in a single display
screen: no swiping, scrolling of lists or navigation be-
tween different screens.
2. Input to the app is exclusively done by i) pressing an
affirmative button for a presented issue or by ii) a yes
/ no decision in favor or against a presented question.
For an affirmative or yes/no button displayed, taking no
action by not pressing a button is always interpreted as
not agreeing, objecting to the presented issue or ques-
tion. Presented issues and questions typically follow a
clear two sentence structure: the first (short) sentence
explains the current situation, the second contains the
proper question or proposed action for the user.

357
358
 3. interaction with the app is multi-modal: the app, which
permanently runs in the background, can be put in the
foreground and made visible by gesture in raising and
rotating the wrist, or pressing a button or touching the
screen of the smartwatch. Questions, advices and hints
presented by the app are announced by vibration and a
message type specific melody, alarm sound to the user.
They will be automatically placed in front of any other
standard apps running at that moment on the watch, ex-
cept ongoing telephone calls, which will not be inter-
rupted. The affirmation or yes/no decision may be also
uttered by speech, if this will be supported by the spe-
cific smartwatch device.
4. Users can always prevent automatic calls to be issued
by the app including data transmission of vital, location
and movement data, as long as there is evidence that the
user has taken a deliberate decision. Only in situations,
where the user does not respond to the app, maybe be-
cause he/she is severely hurt or unconscious, automatic
external communication will take place.
Especially principle 4 has been a major issue of acceptance by
all involved users and age groups, who insisted “to stay in the
driver seat when using the AAL app”.
E. Modeling the Handling of Health Hazards
For the handling of an individual health hazard we propose
the notion of an UML state machine (USM) [9]. Each USM
formally describes the different stages of the handling process
for an individual hazard (e.g. the monitoring of regular fluid
ingestion in fig. 2, the monitoring of absence from home in fig.
3). Each specific stage of the handling process is modeled by a
state within the specific USM. An USM state is associated
with a specific state content, which explains the concise situa-
tion of hazard handling to the user and might request input
from him in order to determine the next handling steps. For the
active state of an USM, the associated state content is the (vir-
tual) output of the USM at that time. Because many USMs
will run simultaneously on the smartwatch, it will be the task
of a central scheduler to determine, which output of all run-
ning USMs shall be actually selected and displayed to the user
on the smartwatch at that time.
The distinguished USM state, where no hazard is present,
is denominated the »normal« state of the USM. This is the
initial state of all USMs. Being in this active state, the output
of an USM displayed on the watch is depicted in fig. 1; it will
coherently show the combined time and orientation info to-
gether with the button for manually calling external help.
The state transitions of the USMs are triggered by the fol-
lowing three possible events:
a) the detection of an ADL, EDL,
b) affirmations or yes/no decisions done by the user by press-
ing a button or uttering the affirmation resp. decision by
spoken language,
c) lapse of time: either i) an actual value for the excess dura-
tion of an activity or inactivity between events exceeds a
358
359
predefined threshold, or ii) a short period timer for an ex-
pected user response, for displaying a presented hint or
iii) a grace period timer (before a question, advice to the
user will be repeated) has expired.
Viewed separately, for the USM in figure 2, an initial drinking
advice will be displayed, when the actual value for the inac-
tivity period between two drinking acts exceeds the nominal
value learned from the past. The user either promises to drink
- by pressing the affirmative button -, which starts a grace pe-
riod, before the advice will be repeated again, if no drinking
takes place in the meantime. If the user does not respond to the
advice in due time, a hint will be displayed announcing the
upcoming automatic call to the HECC or family (see also sys-
tem architecture in fig. 4). A recognized drinking ADL will
always bring back the state machine to the »normal« state, at
least after a clarification call has been terminated.
Figure 3 shows the intertwining between spatial and temporal
criteria, which delimits an absence from home and will cause
return advices to be presented. The return advices may either
address the situation, that the user has left an agreed vicinity
around his home, or the situation, that the absence activity
lasts longer than the nominal absence learned from the past
(for the specific day of the week). In both cases, after the ad-
vices have been presented, specific grace periods start, before
the corresponding advices will be repeated. Returning to home
in the meantime will of course cancel any further advice. And
again, if advices are not answered by an affirmative promise to
return to home, an upcoming automatic call to the HECC or
family will be announced and then issued. The hazard man-
agement in the state machine of figure 3 does also tackle the
transmission of a power down command to a home automation
system, as soon as the smartwatch wearer has left this home
continuously for a certain time duration (measured by the loss
of the known home WLAN SSID signal).
F. Synchronizing Health Hazards Handlers (USMs)
The necessary simultaneity of USM execution requires at-
taching additional priorities [ranging from 0 (lowest) to 9
(highest)] to all states of the USMs, which will be used by a
central scheduler for selecting the output to be actually dis-
played. The labeling of priorities to the states of an individual
USM – denominated by the circles in the state depictions of
figures 2, 3 – of course has to be done in the awareness of the
priorities assigned in other USMs.
The central scheduler and the USMs utilize as a central
synchronization structure - a blackboard (BB) [10] - which is
subdivided by the corresponding priority levels. Upon a state
transition to a new (active) state, an USM pushes this state on
the BB, which needs to be handled by the scheduler. If an
USM performs a state transition from a state, it will remove
this state from the BB. The scheduler always selects the output
from an USM to be displayed to the user, which has the state
with the highest priority on the BB. Two disambiguation rules
will be applied, in order to ensure fulfillment of the directive:
(1) Prefer the specific USM, for which the output has
been already displayed so far.
 (2) Prefer the specific USM, for which its active state has
been finally added to the BB, if (1) isn’t applicable.
The scheduler has also to restrict the possible state transi-
tions from active states of those USMs, for which the output is
currently not displayed on the smartwatch. For such active
states of USMs, all transitions depending on a user reaction
(affirmative actions or yes/no decisions of the user, response
timers dealing with a non appearing answer from the user)
will be temporarily disabled. Such transitions would be only
meaningful, if the output of the USM would have been visible
to the user. All other state transitions from the active state of
an USM, of course, remain possible. The transitions will be
enabled again, as soon as the output of the USM will be se-
lected for display to the user.
In figure 2, if the USM turns to the “drinking advice” state
with priority 1, but which isn’t displayed because a tumble –
with higher prioritized handling states – has been detected
before and the output of the corresponding USM is displayed
to the user, the potential state transitions depending on the
user’s response to states “info about upcoming call” (left) and
“grace period drinking advice” (down) would be disabled by
the central scheduler for the time of the tumble handling. The
state transition to the “normal” state of the USM of figure 2 by
a detected »drinking ADL« would be always possible for the
USM, even during handling of the tumble.
If otherwise no drinking will be detected during tumble
handling, after the respective tumble handling USM returned
Fig. 2: Finite State Machine (USM) for handling the health hazards resulting from regular liquid ingestion, drinking
359
360
to its “normal” state, the central scheduler enters the “drinking
advice” state of the USM of fig. 2 - still remaining on the BB -
and displays the corresponding advice as output of this USM
to the user.
Finally, if state transitions of two different USMs are per-
formed in very quick succession, and the scheduler has to dis-
play the output of both USMs following the above directive,
the display of the later output will be postponed until the earli-
er one has been displayed to the user for a sufficient amount of
time in order to perceive the content.
IV. SYSTEM DESCRIPTION
A. System Architecture
The implemented system (figure 4) consists of an AAL app
and the long-term analysis server (LTAS). The LTAS receives
detected ADLs / EDLs from the smartwatches via asynchron-
ous http and will utilize them for future charged information
services targeting the user resp. his family members. Another
task of the LTAS is backup and restoring of the app configura-
tion data on the smartwatch in case of a lost, replaced or defec-
tive end device.
The HECC – alternatively available family members or rela-
tives – receives the incoming 3G manual or automatic help
calls from the smartwatches. The corresponding HECC soft-
ware system will also process and log the incoming SMS, sent
out just immediately before the call, with the identity and cur-
rent localization data of the smartwatch wearer.

Fig. 4: System Architecture: Information flow between sm
martwatch app,
LTAS server and home emergency call center (HEC CC)
B. App Architecture and Implemenntation
The implementation of the smartwatch appp is based on a
three layer, hierarchical architecture on toop of the smart-
watch hardware / OS (figure 5).
• The lower layer contains the ADL andd EDL detection
based on sensor fusion. For simple ED DLs, like leaving
resp. reentering an agreed vicinity arround the home
[4], [5], this detection will be done bby trigonometric
math calculations based on the current G GPS sensor data.
Complex ADL detection, for example for the detection
of fluid ingestion, drinking [6], [7] uutilizes neuronal
network or statistic regression methodds for achieving
the task. The nets resp. statistic param
meters have to be
trained before by several hundreds of supervised sam-
ples, in order to achieve the necessary pprecision (see [7]
for details of this process, which incluudes data condi-
tioning and mining of the sample ddata on standard
PCs).
• The medium layer on top of the ADLL/EDL detection
layer comprises the USMs for recogn nition and han-
dling of health hazards. The USMs arre represented by
their state transition tables including thhe corresponding
actions to be executed when enteringg the states, the
state contents as output of the USM Ms, the attached
priorities of states as well as the indicattion of the active
state of the respective USMs.
• The upper layer contains the central schheduler described
above for synchronizing the simultan neous operations
of the USMs in the medium layer.
The conceptual model of the USMs and ccentral scheduler
can best be implemented by Fowler’s »moddel, passive view
and presenter« pattern, elaborating from the well-known
MVC pattern [11]. The pattern avoids anny direct relation
between the passive view on the smartwatcch screen and the
360
361
model comprising the USMs deescribed by their transition
tables in the medium layer. Thiss allows the presenter, cen-
tral scheduler, to be the only com
mputationally active module
on the smartwatch together with h the ADL, EDL detectors
and the service modules for co ommunication with HECC
and LTAS, the daily computation n of orientation information
and adaption of nominal valuess, in this way reducing the
computational load for the smartwwatch.
Fig. 5: Three Layer App Architecture, concep
ptual view
V. DISCUSSION
The app has been tested with younger
y seniors and seniors
at high age, including persons with progressing dementia, over
a couple of weeks. The functionality y and practical utility of the
app has been judged collectively po ositive by the test persons,
whereas the day in, day out handlingg of the selected smartwatch
raised concerns. Attaching the watch h to the wrist and especially
coupling the watch with the charging g tray required fine motoric
skills typically not present any mo ore for elderly at high age.
Inductive, wireless charging interffaces for the smartwatch,
which do not presuppose a precisee positioning of the device
with respect to the charger, are suppposed to alleviate this han-
dling substantially and probably – and
a hopefully - will become
a standard for such devices in the nex
xt product cycle(s).
With currently five health hazaards actually monitored: i)
daily routine at home (including night sleep, midday rest …),
ii) absence from home (figure 3), iii)
i drinking (figure 2), iv)
tumbles and v) manual calls the sign
nificance of the concept and
easily maintainability of the USMs could
c be proofed. Although,
more work needs to be done in thee short-term recognition of
health hazards in due time, e.g. suudden physical discomforts
causing a collapse without a tumble. This can be achieved with-
in the developed conceptual framewwork by simply adding addi-
tional EDL detectors and hazard specific, additional USMs.
VI. CONCLUSIONS
The developed conceptual framewo ork and layered architecture
is well suited for coping with the handling
h of specific health
hazards for elderly people wearing smartwatches.
s In particular,
because the care-giving practice in
n utilizing those innovative
wearables is growing on a daily bassis. Therefore the modeling
of these experiences gained indeed requires frequent updating,
adjustments and amendments of thee formal description of the
processing in the USMs to be done with
w economic effort.
 VII. REFERENCES
[1] K.Kirkpatrick, “Sensors for Seniors”, ACM Communications, Vol.
57(2014), No. 12, pp.17-19
[2] N.K.Suryadevara and S.C.Mukhopadhyvay, “Determining Wellness
through an Ambient Assisted Living Environment”, IEEE Intelligent
Systems, Vol. 29(2014), No3, pp.30-37
[3] E.Nazerfard, P.Rashidi and D.J.Cook, “Discovering Temporal Features
and Relations of Activity Patterns”, Proc. 2010 IEEE Int. Conf. on Data
Mining (ICDM) Workshops, Sydney/Australia, pp.1069-1075
[4] R.Lutze and K.Waldhör, “Smartwatches as Next Generation Home
Emergency Systems, Proc. 8. German AAL Conference 2015, Frank-
furt/Germany, Paper A1, VDE Press (in German)
[5] Waldhör and R. Lutze, “Effective Support of Care-Giving Relatives by
Smartwatches”, in press, Proc. 8. European Nursing Informatics Conf.
(ENI) 2015, Hall/Austria, (in German)
Fig. 3: Finite State Machine (USM) for handling the health hazards resulting from an absence from home
361
362
[6] R. Baldauf, R. Lutze and K.Waldhör, “Dehydration Prevention and
Effective Support of Elderly by the Use of Smartwatches”, in press,
Proc. IEEE HealthCom Conference Boston/USA, 2015
[7] K.Waldhör, R.Baldauf: “Recognizing Trinking ADLs in Real Time
using Smartwatches and Data Mining”, in press, Proc. Rapid Miner
Wisdom / Europe Conference, Ljubljana, Slovenia, 2015
[8] D.Anquita, A.Ghio, L.Oneto, X.Parra, J.Reyes-Ortiz, “Human Activity
Recognition on Smartphones Using a Multiclass Hardware-Friendly
Support Vector Machine”, Proc. 4Int Conf. on Ambient Assisted Living
and Home Care (IWAAL’12), Springer Publ., 2012, p. 216-223
[9] S.Bennett, J.Skelton, K.Lunn, “Schaum’s Outline of UML – Second
Edition, Mc Graw Hill ubl., 2004
[10] R. Engelmore, “Blackboard Systems”, Addison-Wesley Publ., 1989
[11] M. Fowler, “Patterns of Enterprise Application Architecture”, Addison-
Wesley Publ., 2003