Hands D2.2.1 Elte R Pu 2008-12-16

HANDS, GA no 224216, Del 2.2.
PROJECT DELIVERABLE
Grant Agreement number:
224216
Project acronym:
HANDS
Project title:
Helping Autism-diagnosed teenagers
Navigate and Develop Socially
Funding Scheme:
Collaborative Project
Deliverable description
Deliverable no: 2.2.1
Deliverable name: Report on initial cognitive psychology requirements on
software design and content design & content
Work Package No: 2
Lead beneficiary: ELTE University
Authors: Miklos Gyori, Ildikó Kanizsai-Nagy, Krisztina Stefanik, Katalin
Vígh, Patrícia Őszi, Anna Balázs, Gábor Stefanics
Nature: Report
Dissemination level: Public
Document number: HANDS/D2.2.1/ELTE/R/PU/2008-12-01
Summary:
This document is a detailed discussion of suggestions/requirements concerning

hardware choice, platform choice, and, primarily, software design and software
content from a (broadly meant) cognitive psychological point of view, including
some psychiatric and a great bulk of pedagogical-therapeutical considerations, as
well as heuristic findings from a pilot eye-tracking study.
It contains the following main sections:
0: Introduction. 1: Adolescents and adults with autism spectrum disorder. 2:
Therapeutical intervention in ASD. 3: ICT solutions in therapeutical intervention for
ASD. 4: Suggestions on hardware choice. 5: Suggestions on platform choice. 6:
General remarks on persuasive software design for autism. 7:
Suggestions/requirements on software content. 8: Initial eye-tracking findings on
the user surface design. 9: Summary. 10. References.
Page 1/105
HANDS, GA no 224216, Del 2.2.1
Project co-ordinator name, title and organisation:

Project Co-ordinator: Professor Peter Øhrstrøm
Organisation: Aalborg University
Tel: +45 9940 9015
Fax: +45 9815 9434
E-mail: poe@hum.aau.dk
Project website address: http://www.hands.hum.aau.dk
0 Introduction
0.1 The aims of this document
The major aim of this document is to set out the initial requirements for the
design and content of the HANDS software, from a cognitive psychology perspective.
This simple formulation, however, needs some more explication.
First, the term ‘initial’ is meant to emphasise that the final design and content of
the HANDS software, as the outcome of the present Project, are subjects of further
testing and development in the course of the Project, so the requirements laid out in this
document are meant, and can only serve as, starting heuristics for initial software designing.
Second, the term ‘cognitive psychology’ is used in a very broad sense here. In fact, it
involves much more than is consensually meant by cognitive psychology. Within the
framework of the present Project, this term came to cover expert knowledge on the
diagnosis, behavioural features, and neuro-cognitive characteristics of, and on
evidence-based cognitive-behavioural pedagogical-therapeutic approaches to, autism
and autism spectrum disorders, as well as expertise in research methodologies in
these fields. In this sense, therefore, the aim of the present document is definitely much
broader than representing a narrowly-meant cognitive psychological perspective.
Third, as it comes from the highly interdisciplinary organisation of the present

Project, it is worth emphasising briefly, that the perspective of cognitive psychology,
however broadly it is meant, is just one perspective among the several involved.
Therefore, cognitive psychological perspective in itself is neither able, nor is expected to give
a full specification of the design of the HANDS software. What is written below as
cognitive psychological requirements is meant to be integrated with requirements
from other disciplines involved. This is even more so as cognitive psychology,
whatever widely one fixes the meaning of this term, does not have the conceptual
and methodological toolkit that could enable it to give a full specification of such a
complex software product as the outcome of the present Project is expected to be.
Page 2/105
Fourth, and at last, we wish to emphasise that, in our conviction, cognitive

psychological expertise (in the broad sense as it is meant in this Project) has a
somewhat distinguished role. This is so, because the HANDS Project should
naturally rely (and, in our perceptions, does rely) heavily on up-to-date, evidence-
based knowledge about the nature of autism (autism spectrum disorders): its
behavioural, developmental, cognitive, and, to some extent, neural features. The
nature of autism determines the possible goals and the potential limits of any
intervention, including that aided by the future HANDS softwares. In this document
we take this distinguished role as an important one, and we aim to explore and explain
the potentials and limits of mobile-ICT-based intervention in autism (autism spectrum
disorders), as they follow from the nature of these conditions.
0.2 The approach of this document
When writing this document, the following considerations determined strongly

the approach we took, in terms of depth, details, and explication.
(1) As emphasised also in the Introduction of the deliverable 2.1.1 (Report on test
methodology and research protocols), the HANDS Project is a widely inter-
disciplinary project that involves such diverse disciplines as pedagogy,
psychiatry, psychology, computer science, software design and development.
This fact, together with the major function of the present document – namely,
that it should be read, understand and used heavily primarily by software
designers, but also by other experts in the Project –, makes it necessary to be
intelligible for readers from various disciplines. This fact motivates why this
document goes into relatively deep details when it is relevant, but, at the same
time, why it takes a generally introductory approach when clarifying the
foundations of the Project as marked off by evidence-based knowledge on the
nature of autism spectrum conditions.
(2) In connection with the above point we apply the strategy to give first a
relatively detailed image of the very nature of autism spectrum conditions,
and then of the central principles and practices of therapeutical-pedagogical
intervention methods. In our view, some grasp of these foundations is
indispensable for the success of HANDS Project. This is so, because, as it will
be explained in more detail later, the nature of atypical cognition in these
conditions is far too complex to allow simple algorithmic solutions in
intervention. All up-to-date, effective and evidence-based intervention regimes for
autism spectrum conditions are necessarily based not only simple-to-execute ‘recipes’,
but, instead, a creative and flexible application of detailed knowledge about the nature
of these conditions. As the HANDS software will be a part of such an
Page 3/105
intervention regime, its design, development, and use must be based also on
an intelligent understanding and creative use of knowledge on the cognitive
nature of autism spectrum conditions.
(3) As a part of the methodology behind this document (see also below), we put a
strong emphasis on, and devote a considerable space to, the analysis of some
available existing solutions with analogous aims and analogous means. This is
not a task genuinely tailored to a Cognitive Psychology Work Package (WP2),
but we find the discussion of some selected solutions, from a cognitive
psychology perspective, a potentially useful tool to make the major points of
this document more plausible, especially for software designers and
developers.
(4) Also, in order to ensure a more effective utilisation of this document, and
especially for experts of relatively distant disciplines, we offer concise
summaries and direct recommendations at the end of every each larger
thematic unit. Though this again may give this document a somewhat
repetitive and redundant fashion, we mean it partly as a store for potential
rapid references to the main points of the deliverable.
0.3 Methodology
This document presents considerations and recommendations primarily from

five methodological sources:
Expert knowledge and literature review on the diagnostic, behavioural, neuro-

cognitive characteristics of, and evidence-based cognitive-behavioural pedagogical-
therapeutic approaches to, autism and autism spectrum conditions. Conclusions and
recommendations based entirely or partly on this methodology can be seen as
products of a ‘deductive methodology’, based on up-to-date scientific knowledge in
the above fields.
Intuitive-conceptual analysis of existing ICT solutions with aims and means

analogous to the future HANDS software. This analysis has had dual benefit. On the
one hand, it served and continues to serve as an intuition-pump for creating novel
solutions and for avoiding maladaptive solutions within the HANDS Project. On the
other hand, as noted above, it may potentially serve as a useful tool to make the
major conclusions and recommendations of this document more plausible, especially
for experts in computer technologies, such as software designers and developers.
Page 4/105
Eye-tracking methodology represents the third methodological source of this

document. The application of this methodology served foremost the development of
experimental techniques to be applied later, in the testing of Prototype 1 & 2 (for
details, see deliverable 2.1.1, Report on test methodology and research protocols). However,
results from the pilot study aimed at refining the eye-tracking testing technique
yielded relevant conclusions for software designing, especially designing the user
surface. These findings and conclusions will therefore be presented in this document.
Collecting and analysing target problems and potential solution algorithms from
teachers and therapists working with high functioning teenagers with autism
spectrum conditions. This intuitive methodology represents and important
contribution to the initial requirements on the content of the future HANDS
software.
Collecting and summarising relevant contributions from various channels of

communication (personal communication, e-mailing, forum discussions) with
partners in the project.
0.4 Within-document and cross-document redundancies
As also noted there, this document, at certain points, shows some redundancy
with the deliverable 2.1.1, Report on test methodology and research protocols. These
redundancies – always noted explicitly, if significant – are to a large extent due to the
facts that (1) some aspects of the nature of autism (autism spectrum conditions) are
highly relevant in both documents, and that (2) eye-tracking methodology appear as
an emphatic aspect in both deliverables. The need for both documents being readable
and intelligible independently of each other made these redundancies unavoidable.
Also, one will find some redundant parts within this document, too. Again, if
these are up to a significant extension, they will be noted explicitly. These within-
document redundancies, here, too, serve the intelligibility of the text especially for
those partners in the Consortium whose professional background is distant from the
expertise involved heavily in Work Package 2.
0.5 The structure of this deliverable
Following this introduction, a detailed overview of the nature of autism will be

given, with a specific emphasis on the aspects relevant to pedagogical-therapeutical
intervention via ICT tools, and characteristics of this condition in adolescence, as the
Page 5/105
age in focus for the HANDS Project(Part 1). This part is meant to pinpoint some of
both the potentials and the limitations of the Project. The next part (Part 2) is devoted
to a clarification of the general principles of evidence-based, effective pedagogical-
therapeutical intervention in autism, and especially in adolescence – the general
principles which must form the basis of the development process within this Project,
too. Part 3 is a review of some selected existing ICT solutions, with aims and tools
similar to those of the future HANDS software. Parts 4 and 5 offer brief suggestions
for hardware choice and platform choice, based on considerations delineated in Part
1. Parts 6 and 7 are in the very focus of this document. Part 6 introduces
requirements and suggestions on the design of the HANDS software – in general and
specifically for the planned components of the HANDS software, too. Part 7
formulates suggestions on the content of the software, and serves both with general
principles and specific examples, too. The last significant part (Part 8) summarises
the empirical findings from initial eye-tracking testing, relevant to the design of the
user surface of the HANDS software. Finally, a summary and a list of references
make this document a complete report.
Page 6/105
1 Adolescents and adults with autism spectrum conditions –

as a specific target group for ICT technologies
1.1 Fundamental facts about autism spectrum conditions
Autism, and, more broadly, autism spectrum disorders – including, among

others, Asperger’s syndrome – are human neurodevelopmental disorders. The term
‘developmental’ here implies that these are not acquired disorders: typically, it is not
the case that a previously healthy person becomes ‘autistic.’ Instead, autism
spectrum disorders represent a specific atypical way of human development, an
atypical neural-psychological route to an atypical adulthood. As recent studies strongly
suggest, this atypical pathway of development typically begins before birth: the basis
of autism is with the affected individual already within the uterus. And these
conditions, once arose, last throughout the human lifespan: although their surface
features may change a lot, and in some cases there comes a considerable decrease in
symptoms, autism and autism spectrum conditions remain with the affected
individual (Howlin, 2005).
As these are neurodevelopmental disorders, the atypical pathway to adulthood

means that the development of the brains of these individuals takes unusual routes.
Although the real details of this atypical brain development process have come
somewhat uncovered just recently, it seems clear that the ‘autistic brain’ develops
differently from the neurotypical brain in many respects. How massively atypical the
brain development in autism is, is well exemplified by the highly atypical speed of
brain growth, and, consequentially, the highly atypical brain size at certain ages – see
Figure 1.1
Page 7/105
Figure 1.1. Atypical brain growth in autism, from birth to

adolescence, as compared to typical brain size. Typical brain size at
any age is shown as 100%. On the basis of Akshoomoff et al., 2004.
The size of the brain, although it is clearly an important sign of the atypical
neural development, is not at all the only difference between the typical and the
‘autistic’ brain. The more specific structural differences are manifold. A detailed
overview of these is not relevant for the purpose of this document, so Figure 1.2
below summarises the most essential ones. The discussion below will occasionally
refer to some of these, when relevant.
Page 8/105
Figure 1.2. Atypical structures in the brains of individuals with

autism. (A) Focal structural abnormalities. (On the basis of Rodier
et al., 2003.) (B) An overall micro-structural anomaly (of micro-
columnar organisation) of the cerebral cortex. (On the basis of
Casanova et al., 2003).
Our aim with this relatively detailed examination of the neural basis of
autism was partly to show, why a cure for autism – one that offer complete
recovery from the disorder – seems to lie in the unforeseeable future. To re-
structure the human brain following a massively atypical developmental history
which left behind a massively atypical brain organisation is not within our reach
at the present day – and most probably it remains so in the next coming years. This
is a strong limitation on the potentials of any intervention concerning these
disorders.
Although, as we have just seen, at their roots autism and autism spectrum
conditions are developmental disorders of the brain, and our knowledge on their
neural bases keeps growing rapidly, the current diagnostic definition and the
diagnostic process are based on strictly behavioural features. (This so because our
knowledge and technologies concerning the underlying brain abnormalities – or on
any other possible biological markers – is not yet refined enough for reliable and
valid biologically-based diagnosis.)
The basis of definition and diagnosis of autism and autism spectrum

conditions in both of the widely used diagnostic systems – ICD-10 (WHO, 1990) and
Page 9/105
DSM-IV (APA, 1994) – are marked and qualitative developmental impairments in

three fields of behaviour —‘the autism triad’:
(1) Qualitative developmental impairments in reciprocal social interactions and

socialisation;
(2) Qualitative developmental impairments in reciprocal communication (both
verbal and non-verbal),
(3) Qualitative developmental impairments in flexible organisation of behaviour
and interests (repetitive and stereotypic activities, restricted and stereotypic
interest).
The developmental impairment must be clearly present before the age of 30 months,
at least in one of the three areas.
As these behavioural impairments represent the key difficulties for the social
integration and participation of the individuals affected with autism spectrum
disorders, and will also have key significance in framing the focus and limitations of
the HANDS project, we give a somewhat more detailed description of these core
features:
Impairments of reciprocal social interaction. This is perhaps the most characteristic

aspect of autism – and, as all symptoms in autism, it shows striking variability. The
almost complete lack of personal, affectively coloured attachment to other people is
salient in many typical children with autism in the early years. The child with autism
often does not differentiate between family members and strangers in his behaviour -
though he is able to recognise them. His/her ability to control social interactions by
meta-communicative tools is limited, and he/she does not understand meta-
communication of other people properly. If he/she seeks the company of others at all
then often he/she does so in an undifferentiated manner, often only for some simple
and repetitive source of joy without any personal colour. It is an often-described
observation that many young children with autism don’t treat and regard people
around them as persons, instead they ‘use’ them instrumentally. These features
usually remain to some extent after childhood, but they lessen in most of the cases.
Throughout their lives, people with autism are not or just rarely able to set up real,
personal friendships and partnerships. Many of so-called ‘high functioning’1
individuals with autism undoubtedly desire for it, often even initiate it, but quite
often they fail to succeed finally.
Deficits of reciprocal communication. Both the course of linguistic-communicative

development and its end-state show characteristic impairments in autism. Again,
1 By the term ’high functioning’ we label those individuals with autism who have non-impaired general intelligence,
formally non-impaired language, and have a relatively high level of adaptive and self-supporting skills.
Page 10/105
individual varieties are enormous. The development of speech shows great varieties
both individually and in its temporal pattern. The linguistic performance of
individuals with autism ranges from the total muteness to the almost perfect
language in its formal aspects – that is, pragmatics and communicative function put
aside, as these are always markedly impaired in autism spectrum disorders. Happé
(1994, pp. 36-37) summarises the problems of communication in autism as follows:
• delay or lack of development of speech, without significant compensating

gestures;
• failure to respond to others’ speech;
• stereotyped and repetitive use of language;
• pronoun reversal;
• idiosyncratic use of words, and appearance of neologisms;
• failure to initiate or sustain conversation normally;
• abnormalities of prosody;
• semantic/conceptual difficulties;
• abnormal non-verbal communication (gesture, facial expression).
Some further points can be added to this list:
• over-literalness; a tendency to interpret others’ utterances literally, and to use

language literally;
• over-formality; typical of the language-use in high-functioning individuals.
Repetitive, stereotyped patterns of behaviour and narrow interest. It is generally

conspicuous that the behaviour of people with autism is penetrated by repetitive,
stereotyped patterns of action. These may include simple repetitive motor activities,
as well as elaborate daily routines, and, as we have seen, echolalic speech. The actual
‘content’ of these repetitive behaviours varies strongly across individuals, and
especially along overall level of functioning, and may change from time to time
developmentally. The focus of the atypically narrow field of interest varies to a large
extent across individuals, and can take the form of seeking certain elementary
sensory stimuli on the one extreme (like observing the light of the sun or of lamps
through the flapping fingers), as well as highly complex but very circumscribed
intellectual activities (like devoting life to designing humane slaughterhouses for
cattle), on the other (see Sacks, 1995).
Beyond these core (definitive) features several other atypical patterns of

behaviours and abilities are present in these conditions, though not necessarily in all
affected individuals. The most important ones of these additional features are:
Page 11/105
General mental handicap. Approximately 75 % of the individuals with core autism, and
about 20 % in the total autism spectrum also show mental handicap to varying degree
(e.g. Bailey et al., 1996; Gillberg, 2000, 2003).
Islets of preserved abilities. Besides the low-functioning areas, however, preserved or

even exceptionally developed capabilities can be also observed in this disorder. In
general, these relatively or absolutely well-functioning capacities are called islets of
abilities or “autistic assets”. These are, most often, certain aspects of memory functions,
spatial cognition, arithmetic skills, and so on.
Uneven IQ profile. The above-discussed feature of cognition in autism seems to

demonstrate itself more generally in the uneven IQ profile, which is typical of core
autism and is sometimes of diagnostic importance (see, e.g., Frith, 1989; Happé,
1994). In case of people with autism performance in various sub-tests shows
considerably more extreme within-individual fluctuation than in the neurotypical
population, or in many other developmental impairments.
Savant skills. Even severe mental handicap can be coupled with excellence in some
narrow, well-circumscribed skills. Music, drawing, arithmetical and calendar
calculations, spatial tasks, and rote memory are some of the most frequent examples
for such savant skills (for examples see Sacks, 1995).
Sensory-perceptual deficiencies. In autism, both atypical over-sensitivity and insensitivity

to certain kinds of stimulation can often be observed, especially in young childhood.
Also, sensory sensitivity can fluctuate extensively (see, e.g., Bogdashina, 2003).
Emotional control problems. There are several manifestations of impaired emotional

and mood control in autism: many individuals with autism show sudden changes in
mood, without clear cause in the current environment; many of them show high level
of anxiety; many of them, especially but not only children, show emotional tantrums,
usually as response to sudden changes in their environment or when their obsessive
routines are hindered.
Motor control problems. It has been long known – since the first descriptions of autism
– that these individuals are often characterised by motor coordination problems to a
varying degree. Their level of difficulties with adequate control of various
movements ranges from mild clumsiness till neurologically relevant, deep difficulties
with fine motor control and/or gross motor control. That is, problems with
intentional control of movements may affect fine manipulation as well as overall
movements of the body and limbs.
Page 12/105
Additional medical conditions. Autism is to some extent associated with certain medical
conditions, which tend to co-occur with autism with varying probability. From
among these, epilepsy and general mental handicap may be relevant here.
As mentioned before, it is a key feature of autism and autism spectrum

conditions that symptoms show a considerable variability across, and even within,
individuals. Although definitive symptoms always fit into the autistic triad, the
actual behavioural manifestations of the symptoms show striking difference across
individuals. Practically, there are not two individuals with autism who would show
absolutely the same symptoms. And also, even within one single individual with autism
spectrum condition, symptoms may easily show deep variability:
• according to age (symptoms change as the individual develops);

• according to actual state (symptoms change with mood, overall activity level,
fatigue, etc.);
• according to the actual situation and environment;
• as a result of changes in the circumstances of daily life (e.g., new home, new
school, changes in daily routines, loss of family members, etc.).
An important consequence of the striking variability in symptoms is that any

intervention technique in autism must allow a considerable level of
individualisation, ‘customization’. Tailoring the intervention tools to the needs of
the actual individual with an autism spectrum condition requires expertise in
these conditions as well as a detailed knowledge on the actual individual’s
development, psychological profile, problems, symptoms, and circumstances.
1.2 Cognition in autism: unusual abilities
The facts that (1) autism and the related conditions are defined behaviourally,
and are characterised by quite specific patterns of abilities and disabilities, and that
(2) behind these there lies an atypical development of the brain, make it a necessity
that the understanding of autism is a multi-level, multi-disciplinary enterprise.
Figure 1.3 below shows the explanatory levels and the corresponding disciplines that
play necessarily key roles in our improving understanding of these conditions.
HANDS project, as is based on up-to-date expert knowledge on these conditions,
must work, to some extent, in the matrix of these disciplines, too.
Page 13/105
Figure 1.3. The explanatory levels and corresponding disciplines

that play key roles in the understanding of autism and the related
human conditions.
As our knowledge on the neural background of autism and the available

technologies do not yet allow a direct intervention on the neural level, cognitive
systems ‘mediating’ between brain and behaviour have a specific importance in
understanding autism and designing well-based intervention tools. This is even more
so, as a considerable volume of psychological research on autism spectrum condition
suggest that – in terms of psychological mechanisms – autism is primarily (though
not exclusively) a cognitive disorder. While emotional and motivational processes
are also atypical in autism to a varying extent, the very core of autism, on the
psychological level, seems to be closely related to atypical information processing in
various ways. Both for a scientific understanding of autism and for developing well-based,
effective intervention regimes and tools it is a key question: which cognitive (information-
processing) systems are impaired/atypical within autism spectrum conditions?
As the unusual organisations of various cognitive functions represent a crucial

level of understanding for the HANDS project – as they imply the perspectives as
well as the limitations for any psychological-pedagogical intervention –, below we
summarise the core cognitive features of autism and autism spectrum conditions.
As the neural background of autism is on remarkable complexity, one must

expect that the corresponding cognitive deficits are also multiple, and show complex
patterns. Recent studies are on the way to reveal this complexity, as various cognitive
Page 14/105
functions related to sensory processing, perception, attention, learning, memory, social

cognition, problem solving, high-level action control and motor control have been found
atypical and/or impaired. This is not our aim to give a detailed description of all
these; instead, we shall focus on those three major cognitive
impairments/characteristics which seem to play the most crucial role, and have the
major relevance for HANDS project (software designing).
1.2.1 Impairment of social cognition (‘naïve theory of mind’)
‘Naive theory of mind’ is the human cognitive-behavioural capacity to

understand, explain and predict various agentsʹ behaviours by attributing mental
states to them and interpreting behavioural acts as causal consequences of the
attributed mental states (Gyori, 2006). This intuitive, mostly non-conscious, but
arguably highly complex cognitive ability of humans forms the most important basis
of human social cognition and understanding. Typically developing humans seem to
possess this ability already as early as the age of 15-18 months, and show a
considerable development in this regard throughout childhood, the years 3-5
bringing the most striking development.
This ability plays key role in everyday social interactions and participation, as
this ability allows us to recognise quickly and efficiently the intentions behind
human actions, and other mental states (beliefs, emotions, etc.). In other words, this
ability makes human action meaningful for us. Also, it allows to a considerable
extent to foresee, to predict human actions – also as a crucial benefit in everyday
social life.
As important theoretical (Sperber & Wilson, 1986) and empirical (e.g., Happé,
1993) advancements have shown, naïve theory of mind ability has a key role in
flexible human communication, verbal and non-verbal equally. This is so, to put it
briefly, because the key cognitive factor in human communications seems to be the
display of (communicative) intentions (what one wants to achieve with her
words/actions), and recognising such (communicative) intentions (to understand
what the communicator wants to achieve with her words/actions). Now, intentions
are mental states, so display of intentions and understanding intentions, by
definition, require naïve theory of mind ability.
A huge bulk of empirical studies demonstrated that individuals with autism,

and especially children with autism have an impaired theory of mind ability. In some
cases this impairment means a total lack of the ability to attribute mental states, while
in the majority of individuals with autism it means a seriously limited ability to do
so.
Page 15/105
As it follows from the above description of the use of normally functioning

naïve theory of mind ability – guiding social understanding and flexible
communication – the impairment of naïve theory of mind in autism explain well,
although not totally, the first two fields of definitive symptoms of autism: the
qualitative impairment in reciprocal social interactions, and the qualitative
impairment in reciprocal communication (see, e.g., Frith, 2003).
It is also in line with findings from typical brain basis of sound naïve theory of
mind and from autism, that prefrontal cortical areas form an important basis of this
ability, and that these areas are markedly impaired in autism.
That is, we have good reasons to think that much of the social difficulties
people (including teenagers) with autism spectrum disorders face with in
everyday life are due to the fact that they have an impaired theory of mind ability.
This impairment therefore has a specific importance for the HANDS project,
partly because it puts some important limitations and risks on this enterprise.
1.2.2 Impairment in executive functions
In Ozonoff et al. (1991; p. 1083) words,
“Executive function is defined as the ability to maintain an appropriate problem-

solving set for attainment of a future goal; it includes behaviors such as planning,
impulse control, inhibition of pre-potent but irrelevant responses, set maintenance,
organised search, flexibility of thought and action.”
Other authors provide lists somewhat different from the above – for example
self- or action-monitoring, verbal self-regulation is included by several authors (e.g.,
Miyake et al., 2000; Beveridge et al., 2002). Nevertheless, factor analyses identified
four dimensions of executive functions (see, e.g., Perner & Lang, 2000):
planning;
cognitive flexibility / set shifting;
inhibition;
working memory – more precisely, its central executive component.
So defined, executive functions, strongly based on human prefrontal cortex

again, are necessary for any complex, non-routine action, and also for complex, non-routine
cognitive processes. People with acquired pre-frontal cortex injuries and executive
function impairment, as a consequence, show non-adaptive impulsive acts, rigid,
stereotypic behaviour, inability to create plans for complex actions, and serious
Page 16/105
impairment in executing action sequences in complex but flexible ways. Executive

functions are also crucial in controlling memory processes and attention processes;
they have a role in emotional control as well as in daily living skills, and pursuing
life-long goals consequentially but flexibly.
People with autism show also several from among the just mentioned
symptoms, and the fact that executive function impairment is quite widespread,
most probably universal in autism, has been confirmed in several studies.
As it follows from the description of executive functions and the consequences

of their impairments, the well-documented executive impairment plays a key role in
giving rise to the symptoms of autism in the third are of the autistic triad: “ qualitative
developmental impairments in flexible organisation of behaviour and interests (repetitive and
stereotypic activities, restricted and stereotypic interest).”
Again, we have good reasons to think that a part of the social difficulties, and
much of the self-regulatory and self-management problems teenagers with autism
spectrum disorders face with in everyday life, are due to the fact that they have an
impaired set of executive functions. This impairment, as naïve theory of mind
impairment, too, therefore has a specific importance for the HANDS project, partly
because it puts some important limitations and risks on this enterprise.
1.2.3 Weak central coherence, or detail-focused processing style
This is probably the least well-understood cognitive aspect of autism,

nevertheless as crucial as the previously mentioned two.
The term ‘central coherence’, as well as the hypothesis that this cognitive
characteristic is impaired in autism have been introduced by Frith (1989). In her
view, central coherence is a general characteristic of human information processing:
it is “a tendency to draw together diverse information to construct higher-level
meaning in context” (Frith & Happé, 1994, p. 121). In other words, central coherence
is the human cognitive tendency to integrate partial bits of incoming information into
meaningful patterns in a context-sensitive, context-appropriate manner. The weak
central coherence theory of autism claims primarily that central coherence is
impaired (weak) in autism (ASD), and this deficit explains many behavioural
manifestations of the syndrome (Gyori, 2006).
These behavioural features include, among others (after Frith & Happé, 1994):
Page 17/105
problems with perceptual integration;

restricted repertoire of interests;
obsessive desire for sameness;
insistence on non-functional routines;
islets of preserved abilities, uneven cognitive profile;
excellent rote memory;
pre-occupation with parts of objects;
narrowed and stereotypic interests; etc.
Just like the other two cognitive impairments, the presence of weak central
coherence of detail-focused processing in autism (autism spectrum conditions) has
been demonstrated in dozens of studies. The emerging picture shows this
impairment especially complex, as central coherence show considerable within-
individual variability: it is stronger in some domains while weaker in others, and
these relative strengths and weaknesses vary a lot across individuals.
1.2.4 Some cautionary notes: the space for (technology-based) intervention
It is important to see clearly that all these cognitive functions are highly
complex ones, not yet at all clearly understood in terms of underlying computations
and representations. Therefore, it is not a reasonable goal in the foreseeable future
that ICT could replace or substitute these functions.
As it will be explained in Section 2 to more depth, presently available

psychological-pedagogical interventions and therapies are also unable to supply or
compensate significantly these missing/impaired functions.
Also, as Persuasive Technology is at the core of the HANDS project, it is

important to see that autism is a complex neuro-cognitive condition, and not a mere
matter of persuasion. Putting it in other way, people with autism and autism
spectrum condition show very specific behavioural symptoms and problems not
because they are “unconvinced” to behave in a neurotypical way.
Moreover, as several cognitive functions work atypically in autism spectrum

conditions, Persuasive Technology (and technology in general) must be accordingly
tailored to their specific needs and limitations. One cannot take it granted that
technological solutions efficient in neurotypical people will similarly be efficient in
people with autism spectrum condition. Generally, Persuasive Technology, and,
more generally software designing principles are tailored to what we may call ‘the
typical human agent’. People with autism are not at all ‘the typical human agents’:
Page 18/105
they are not in the way they perceive and understand the world, neither in the way
they think and act.
More on the reasonable perspectives for psychological-pedagogical intervention

and for ICT comes later on.
1.3 Why and how autism leads to social marginalisation and

isolation?
It is worth summarising briefly why and how the core features of, and cognitive
impairments in, autism lead to social marginalisation and isolation.
• The impairment in reciprocal social behaviours directly prevents the affected

individual from participating everyday social interactions smoothly.
Moreover, the oddness of social behaviours often leads to rejection by others
in the social environment, and also anxiety in the affected person. The massive
naïve theory of mind impairment underlying inadequate social interactions
prevents the individual from handling these problems in an insight-driven
way.
• Similarly, the impairment in reciprocal communicative behaviours prevents

the affected individual from being a natural part of usual everyday
communicative transactions. This again leads to social isolation, and, in turn,
anxiety in the individual. Again, massive impairment in naïve theory of mind
does not make it possible to handle these problems via insight.
• The strong tendency for repetitive and stereotypic activities and actions, as
well as the obsessively narrow interest make the affected individual strange
and often even bizarre in the eyes of others in the social environment,
seriously preventing her/him from social participation and inclusion.
Underlying executive functions problems are too fundamental to be overcome
by insight or therapeutic intervention.
• Similarly, such additional symptoms as impulse control problems and

emotional tantrums, or strong insistence on non-functional routines,
difficulties with daily life management and self-care make these individuals,
even if they are in the high-functioning segment of autism spectrum
conditions, generally they are social partners hard to cooperate with, socialize
with, build partnership with – without specific expertise.
Page 19/105
That is, social isolation and marginalisation is multiply determined by the

core features of autism, and is a serious risk at all ages. To overcome to a possible
largest extent, they require evidence-based intervention regimes and cooperation
of involved professionals, family members, and other key people in the affected
person’s environment.
1.4 Specific features and problems in high-functioning teenagers with

autism spectrum disorders
Beyond the general features of autism just summarised, what specific features
arise in that subset of people living with autism which is relevant for the HANDS
project?
This subset is labelled as “high-functioning teenagers” with ASD. The term

‘high-functioning’ refers to those individuals with autism who are characterised
heuristically as it follows:
• They do not have general mental handicap, that is, their overall cognitive
abilities (intelligence) are in the non-impaired range. In terms of IQ, it means
an IQ score not less than 70.
• They have age-appropriate linguistic abilities. That is, they are not
characterised by linguistic impairments, in the formal aspects of language.
Pragmatics and communicative functions are always impaired to some extent
in autism.
• They have at least elementary self-help skills and daily living skills, so that
they do not depend on continuous support and guidance.
As it is a heuristic definition, there is not at all clear-cut borderline between

high-functioning and non-high-functioning individuals with autism. Nevertheless,
high-functioning teenagers with autism spectrum conditions have some specific
features which do not necessarily characterise other age groups and/or not-high-
functioning teenagers with autism. The most important are set out below (Shea &
Mesibov, 2005).
Overall development
• Adolescence is a period of considerable improvement for many individuals

with autism spectrum conditions, in terms of social, communicative, and self-
management skills.
Page 20/105
• For a minority (~ 10-20%), adolescence brings varying level of deterioration,

with such possible changes as growing aggression, loosing skills, and
behaviour becoming more rigid and stereotyped.
• At the same time, many teenagers with autism continue to develop without a
dramatic change, on a steady maturational course.
These are important facts that must be taken into consideration when designing
research methodologies in HANDS project, too.
Institutional, social and emotional challenges
• Adolescence brings a growing expectation of more adaptive behaviours in school

and/or other institutional environment.
• Also, family may expect more independence, more social insight and self-help
skills from the teenager with high functioning autism.
• At the same time, bullying and teasing have a bigger chance, as is among
typically developing teenagers – a phenomenon especially hard to cope with
for individuals with autism.
• A tension between the growing interest in having friends and partners, while not
having sufficient social skills to develop and maintain such relationships.
• Emerging sexuality may cause tensions, emotional problems and also conflicts
arising from the lack of clear insight about relevant social norms and customs.
• Growing self-consciousness and insight about the person’s own condition and
limitations bring emotional problems and instability in some teenagers with
autism spectrum conditions.
1.5 Unusual cognition in autism spectrum conditions: unusual

perspectives, unusual limitations, unusual risks
Now we wish to summarise what kind of perspectives the specific cognitive

background of autism opens for ICT-based intervention; and at the same time, what
kind of limitations it represents and what kind of risks it brings.
Impaired naïve theory of mind, and, more generally, impaired social cognition
makes, among other factor, ICT an especially appropriate medium for many
individuals with autism. Interacting with digital devices does not evoke such anxiety
and stress as interacting with people can occasionally bring up. This leads also to a
potentially very strong reliance on the digital device, making the intervention more
effective. As this impairment often makes the individual deeply helpless in some
social situations, even simple algorithms offered by the digital device may help a lot.
Page 21/105
Limitations and risks: we must emphasise repeatedly that no digital solution for
theory of mind is available, so one should not aim to offer the digital device as a
substitute of the basis of social cognition. A further possible difficulty is that – due to
the impaired ability to understand mental states – the subject may misunderstand the
pedagogical intention behind the device (and will take it as if it were meant as a
companion, a real friendly agent, for example). As a possible consequence, she may
develop inadequate and maladaptive, but strong emotions towards the device;
strong attachment or, on the contrary, strong resistance. Also, theory of mind
impairment and broader problems in social cognition and social perception may
make the toolkit of Persuasive Technology non-efficient or even counter-efficient.
E.g., the social actor may be irrelevant or even disturbing for some subjects with
autism. See Figure 1.4 for a summary.
Figure 1.4. A summary of the perspectives, limitations and risks

provided by the naïve theory of mind impairment in autism for
intervention based on ICT devices.
Impairment in executive functions. As this impairment again causes very deep

difficulties in flexible organisation of one’s own behaviour, even simple algorithms
may offer huge help, in this respect, too. Also, mobile digital devices can offer ‘on-
site’ help in visual modality, a feature especially valuable in the management of
behavioural difficulties in autism.
Page 22/105
Limitations and risks. We emphasise again that there is not computerised

solution that could, in a prosthetic way, substitute the deeply impaired executive
functions. Another serious potential difficulty is that executive problems per se may
prevent the individual with autism from using the digital device flexibly and
adaptively: the tendency for behavioural rigidity may appear in the way the subject
uses the device meant to overcome this rigidity. Yet another serious limitation may
come with the potential lack of generalisation, a frequent executive problem in
autism: the subject may not understand, for example, what is the kind of situations in
which the digital device should be used, and keeps using it just one very specific
situation. Finally, it is a considerable problem here that executive function difficulties
show quite variable patterns in autism. See Figure 1.5 for a summary.

provided by the executive function impairment in autism for
intervention based on ICT devices.
Weak central coherence / detail focussed processing. The major space for
intervention based on digital devices in relation to this impairment is the possibility
to control and focus adaptively the autistic person’s attention. As this cognitive
characteristic is essentially perceptual, and, at the same time, highly abstract, cannot,
at the moment, be substituted by computerised solutions. Nevertheless, even
relatively simple algorithms/solutions may ease the related difficulties of high
functioning individuals with autism in some situations.
Page 23/105
Limitations and risks. As in the case of executive difficulties, the problem to

overcome itself may hamper the success of intervention: e.g., subjects may have
problem to integrate those pieces of information that are displayed on the screen of
the digital device. Similar difficulty may arise in the integration of the displayed
information with the situation. And/or, their attention may get grasped by irrelevant
details either on the display or in the environment. See Figure 1.6 for a summary of
this point.

provided by weak central coherence / detail-focused processing in
autism for intervention based on ICT devices.
As the pattern of cognitive impairments varies to a large extent within autism,

and, more narrowly, within high functioning teenagers with autism, also the
perspectives, limitations and risks for mobile ICT-based intervention vary to a large
extent from individual to individual. These facts, together with the genuine
variability of symptoms of autism mentioned before, make it highly necessary the
future HANDS software be deeply customizable, both in content and in the way of
functioning.
Page 24/105
1.6 Possible target problems for mobile ICT-based Persuasive

Technology in autism spectrum disorders
Finally, we set out a few kinds of returning problems in autism in adolescence

that may serve as targets of mobile ICT-based Persuasive Technology:
• Management of non-adaptive emotions specific to certain situations (e.g.,

situation-specific anxiety, phobia, anger, etc.).
• Management of behavioural problems specific to certain situations (e.g.,
situation-specific irrelevant speech, inability to wait, situation-specific
passivity, etc.).
• Development of certain simple, algorithmic social skills, specific to certain
persons/situations (e.g., salutation in certain situations, asking for help, etc.);
• Management of relatively fixed, simple daily life routines (travelling,
shopping, going to doctor, etc.).
• Development of relatively fixed, simple daily life routines (becoming more
independent in the above activities gradually).
• Development of specific cognitive, socio-cognitive skills systematically
(practising several possible skills in spare times).
1.7 Summary: limitations, reasonable goals and means
To summarise the previous discussion in Section 1, we emphasise the following

points:
• Autism, and, more broadly, autism spectrum disorders – including, among

others, Asperger’s syndrome – are human neurodevelopmental disorders.
• To re-structure the human brain following a massively atypical

developmental history which left behind a massively atypical brain
organisation is not within our reach at the present day – and most probably it
remains so in the next coming years. This is a strong limitation on the
potentials of any intervention concerning these disorders.
• Also, it is important to see clearly that the cognitive functions which are
impaired in autism are highly complex ones, not yet at all clearly understood
in terms of underlying computations and representations. Therefore, it is not a
reasonable goal in the foreseeable future that ICT could replace or substitute
these functions.
Page 25/105
• Social isolation and marginalisation are multiply determined by the core

features of autism, and is a serious risk at all ages. To overcome to a possible
largest extent, they require evidence-based intervention regimes and
cooperation of involved professionals, family members, and other key people
in the affected person’s environment.
• Also, as Persuasive Technology is at the core of the HANDS project, it is

important to see that autism is a complex neuro-cognitive condition, and not a
mere matter of persuasion. Putting it in other way, people with autism and
autism spectrum condition show very specific behavioural symptoms and
problems not because they are ‘unconvinced’ or ‘un-persuaded’ to behave in a
neurotypical way.
• Moreover, as several cognitive functions work atypically in autism spectrum

conditions, Persuasive Technology (and technology in general) must be
accordingly tailored to their specific needs and limitations. One cannot take it
granted that technological solutions efficient in neurotypical people will
similarly be efficient in people with autism spectrum condition.
• All the cognitive characteristic that offer a space, a perspective for mobile ICT-
based intervention in autism also carry serious limitations and risks – these
must be taken very seriously.
• As the pattern of cognitive impairments varies to a large extent within autism,

and, more narrowly, within high functioning teenagers with autism, also the
perspectives, limitations and risks for mobile ICT-based intervention vary to a
large extent from individual to individual. These facts, together with the
genuine variability of symptoms of autism mentioned before, make it highly
necessary the future HANDS software be deeply customizable, both in content
and in the way of functioning.
• This is even more so as, due to the nature of cognitive impairments, HANDS
software cannot offer general solutions, applicable in the same way across
situations, problems, and persons. Instead, it must focus on the management
of very specific problematic situations, routines, skills, and on the
development of more and more independent application of such highly
specific adaptive routines and skills.
Page 26/105
2 Therapeutical interventions in ASD: an overview of core

principles and core solutions
Despite the indisputable fact that autism is a developmental disorder attributed

to biological factors, intervention, for the time being, is scarcely influenced by the
outcomes of neurobiological, or even less by genetic research. Data from cognitive
psychological research (see section 1), however, have a considerably larger impact on
clinical work, particularly on therapy. In this part of the document we summarize the
outcomes of the surveys of intervention and the principles of the most efficient
therapeutic approach. We do not discuss those wider perspectives that are very
important, but not yet relevant for development of HANDS methodology (e.g.
transactional models in integration of children with ASD).
2.1 Different therapeutic approaches and their efficiency
Though autism is one of the most intensely concerned developmental disorder,

relatively few studies cover the efficiency of particular therapeutic procedures and
programs, while applying a precise research methodology (for a review see Howlin, 2002,
2005). What makes it difficult to compare the results of the hundreds of efficiency
studies is that, in the majority of these studies, a lot of methodological problems arise
(e.g. there are no solid baseline and testing assessments, matched control groups;
blinding is usually missing, etc.). Yet, we may draw some relevant inferences:
(1) Physical-sensory therapies as ASD-specific interventions have been

supported by weak results (e.g. sensory-integration – Ayres, 1979), or
have clear scientific refutations (e.g. facilitated communication – Biklen,
1990).
(2) As for medical therapies, data related to pharmaco-therapies (for a review

see Volkmar et al., 2004) are insufficient; but their risks and the lack of
information about their side-effects and long-term effects have to be
mentioned.
(3) The recently „popular” diets and mega-vitamin courses (e.g. Rimland,
1994) have no scientific confirmation yet, thus these are not
recommended (unless, of course, there is vitamin deficiency or food
allergy diagnosed).
Page 27/105
(4) Recently, the most effective of all therapies are those programs which
apply early intensive behavioural approaches (e.g. Bondy & Frost, 1994;
Smith et al, 2000).
Naturally, it is a reasonable requirement of the parents that they wish to find

the most appropriate treatment for their children. However, it is not helpful and
correct to take unrealistic promises, such as ‘full recovery’ (e.g. Kaufman, 1981;
Lovaas 1987). Even if the applied methods are appropriate, the promotional-
rhetorical tweaks are objectionable for ethical and professional reasons. On the one
hand, because autism is a developmental disorder which has influence across the
life-span, and, on the other hand – considering the complex genetic and
neurobiological background of the disorder –, it is not plausible to think that there is
only one ‘beatific’ way to support people with autism.
Out and away, the individualised ‘mixtures’ of elements of evidence-based

autism therapies are the most effective. As a matter of fact, nowadays there is a
recognisable tendency: different interventional approaches strongly converge. It
means that the ‘ideological’ background of a given therapy is less important than the
analysis/interpretation of the given child’s behaviour, and the facilitation of the
interactions between the therapist and the child (Jordan & Powell, 1996; Dunlap,
1999).
2.2 Comprehensive, evidence-based model of intervention for autism
In the followings we give a review of those most important elements which are
crucial in evidence-based therapeutic approaches of autism – see Figure 2.1 below.
(For more specific, HANDS-related examples of the methodological toolkit of this
model see section 7.)
The confirming evidence of this model came from two sources: on one hand, it
suits to the scientific facts of the nature of autism, while, on the other hand, it
contains scientifically proved methods from complex therapeutic programmes. The
basis of the model are the TEACCH program (Treatment and Education of Autistic
and related Communication–handicapped CHildren; see e.g. Mesibov et al., 2004;
Peeters, 1997); the Pyramid approach (Bondy & Frost, 1994), the ABA program
(Applied Behavior Analysis; e.g. Smith et al., 2000), and other high quality, systematic
and complex programmes (e.g. Quill, 2000, Howlin, 1997; Jordan & Powell, 1995).
The first step of the intervention is to create a therapeutic ambience which

adopted to the characteristics of the disorder, and to the child’s/adult’s individual
needs, patterns of the weaknesses and strengths. In the next steps we gradually make
Page 28/105
this structure more flexible and more ‘permeable’– by always monitoring the child’s
current state.
The appropriate therapy always leads to better outcome, although the prognosis is
strongly influenced by (1) the child’s intellectual capacity; (2) the severity of the
autism; (3) the level of language skills; (4) the additional
problems/diseases/disorders; (5) the child’s personality, (6) family background, (7)
other environmental factors, etc. It is essential to choose the goals of the therapy
adequately: the expectations should not be unachievable, nor on ‘insolently’ low
level. That is why the careful diagnostic and therapeutic assessments are crucial for
planning interventions in individual cases of ASD.
tools / methods
visual-help
behavioural approach
structured education
adapted auxiliary therapies
involvement of parents
ambience – emotionally safe, predictable, comprehensible
Figure 2.1. A comprehensive model of the evidence-based therapeutic approaches
2.2.1 Goals
Our most important, comprehensive goal is to help children with autism to

experience themselves as autonomous and competent people. Though the level and
the field of independence strongly depend on individual skills and needs, setting a
table sometimes means as great success as defending a Ph.D. thesis.
Page 29/105
Considering the specific, primary goals of the intervention it is not surprising

that the compensation of the socio-communicative difficulties is the most important aim.
We could not train academic or language skills without functional, comprehensible,
easily applicable communicational tools. Learning is interaction, but not with the
teacher only. The intensive, one-to-one trainings are usually effective, but the focused
group activities are indispensable, too. Improving self-help skills and leisure time skills
are also crucial – in integration with supporting social and communicational skills.
Ideally, treatment of challenging behaviours is not one of the primary goals of

the intervention, but the beneficial consequence of emotionally safe therapeutic
ambiance, appropriate methodology and improving communication, social skills and
daily living skills. In fact, we prevent these challenging behaviours instead of
applying a direct problem-focused approach.
In every case the fundamental therapeutic goals have to be eked out with non-
autism-specific, but individually relevant additional aims. Though the developmental-
disorder-related fields are more emphatic, those aims which are connected to the life-
quality and/or sanity of the family and the child are very important, too.
2.2.2 The therapeutic ambience
It is obvious that we have to create an emotionally safe ambience in any

therapeutic or educational situation. In autism predictability and comprehensibility are
the key aspects of this. Predictability means not only the predictability of the events
or activities, but also the unambiguous and consistent behaviour of the therapists.
Moreover, therapeutic ambiance has to be appropriate for generating several
opportunities to interact – from the artificial on-to-one situations to more natural
settings.
2.2.3 Tools, methods, techniques
Visual support
Visual support is one of the most important and most effective tools in
developing emotionally safe therapeutic ambience, and in realization of
intervention goals. ‘Seeing is knowing’ in autism (Peeters, 1997), that is,
the appropriate visual information (pictures, photographs, pictograms,
written instructions) is comprehensible in an individualised manner, it is
non-transitional, and the timing of the information processing can be
controlled by the individual. It means more effective communication,
more independency, less frustration and less stress in everyday life. (For
more specific examples see section 7.)
Page 30/105
Behavioural approach
The behavioural approach gives exquisite framework for careful
planning and evaluating of the intervention, and also for teaching new
skills. We would like to emphasise, that modern behavioural
modification (see e.g. Clements & Zarkowska, 2001; Clements, 2005)
definitely differs from the methodology of the ‘old-fashioned’
conditioning.
To give a strong motivational basis means more effective
intervention, and using behaviour modification is a crucial element of
the development of emotionally safe ambience. In this therapeutic
framework the therapist gives clear, consequent, comprehensible feed-
back for the child, and reacts consistently and positively for the child’s
appropriate behaviours.
Structured education
In teaching situations – especially in group settings –, giving structure
for time, for the activity, and for the content is indispensable (see e.g.
Jordan & Powell, 1995). The lack of comprehensible structure leads to
loss of emotional safety and to increased frequency of challenging
behaviours.
Auxiliary therapies
Auxiliary therapies are originally non-autism-specific therapies which
focus on different non-ASD-relevant skills (e.g. hippo-therapy, sensory-
motor trainings; music therapy; etc.). Though these do not have direct
influence on the definitive aspects of the disorder, they can be adapted
to autism – especially because usually these offer enjoyable activities,
and give opportunities to develop interactions in different settings, with
different persons. Additionally, if these interventions are needed for
individual reasons, autism-specific adaptation could increase their
efficiency.
Involvement of the parents

The role of the parents/family members in ASD interventions is crucial.
First, because reinforcement of the parent’s competence is a strong
psychological need, and, second, because the specific support of the
children could not be reduced only to the therapeutic settings; and third,
because positive effects on the life-quality of the whole family is
essential. However, usually there are great shortcomings in this respect
for practical reasons. There is no enough professional capacity to
support the family members regularly after a short intensive phase of
co-operation (e.g. after a parent training or a home training). Thus, the
Page 31/105
parents already have ideas about the appropriate interventional

techniques, but they usually do not get enough help from the
professionals to implement these ideas. The expectation from parents to
work as if they were professional therapists is not at all adequate. The
main goal is to support them to work as ‘professional parents’ –
continuous monitoring of their needs and capacities is needed.
Practical considerations
The intervention is more effective (1) when it starts at as early age
as possible; (2) if it is intensive (at least 20 hours/week); (3) if it is long-
term, regular and systematic; (4) if there is high professional/client ratio
(1/1; 2/1) and (5) the professionals are well-trained theoretically and
methodologically, and they are experienced.
2.3 The role of the HANDS toolkit in the therapeutic framework of ASD
The HANDS toolkit is not a new therapeutic method, but it could be the extension
of existing evidence-based interventional techniques. Its primary benefit is
transferring the school-based autism-specific support into different settings -- ‘having
a little teacher in the pocket’. Naturally, there is no chance to substitute entirely the
traditional therapist-based interventions, but – with limitations – the HANDS toolkit
is appropriate to include several individually relevant supportive tools/techniques.
Therefore the HANDS software has to suit to the recent therapeutic approaches,
since there could be several possible benefits of the toolkit (about the risks see section
3):
(1) Its targets suit to the most specific goals, such as improving socio-
communicative skills, independence.
(2) It gives the opportunity to apply visual support.
(3) It definitely could facilitate individualisation (with the teachers’ and
parents’ assistance), not only in audio- and visual skins, but – more
importantly – in the contents and the forms of supports.
(4) The toolkit can easily be emotionally safe, as computer-based
techniques usually offer predictable and comprehensible
‘environments’.
(5) The presentation of the information (e.g. timing, relevance, structure)
could be specific and individualised.
(6) The HANDS toolkit fits well to the behavioural therapeutic
framework.
(7) It gives the opportunity to apply existing, computer-based
training/practicing programs for better achievements.
Page 32/105
3 ICT solutions in therapeutical intervention for ASD
The aim of this part of the document is to give a brief review on the computer-
based interventions which have been used in the education of people with autism
spectrum disorders. It is not a comprehensive overview; rather we focus on analyzing
and systemizing some of the existing ICT tools by highlighting their main
characteristics. We focus on the intervention areas they targeted, and also on the
methods and techniques they use, considering the autism-specific and ICT issues as
well. We give a bit more detailed description of some comprehensive ICT programs
which we found most relevant for HANDS. We think it is important to know and
analyse them – to build on their strengths and fill in their gaps. Another aim of this
discussion is to give a structure for this field and define the possible space and role of
the HANDS toolset within it. Finally, we raise some possibilities how the HANDS
can be unique in this field.
3.1 Specific advantages of using ICT tools in education/intervention for

people with ASD
First, we list the general advances of using computers when working with
people living with ASD. We find important to summarize them – even if some are
trivial –, because we can highlight some specific features of autism by this list,
features which must be kept in mind when developing an ICT tool (Murray and
Lesser, 1999).
As a computer usually works in a consistent and predictable way, it can be

controlled easily by the user. For people with ASD – who usually have marked
difficulties to handle their complex social environment, and this leads to stress and
frustration – the computer often provides safe, comfortable and rewarding
environment where they can confidently move around.
It raises less social demands: there is usually no need to communicate with

other people and adapt to them. Even if communication does appear (e.g. in case of
e-mail, chat-forums, etc.), the rules and structure of the interaction are more defined
than in a face-to-face contact, and also there is a possibility to control the pace of the
interaction (i.e. to slow down, if needed).
The computers can be used without any spoken language, which is a great
advantage in this population, where a considerable part of the individuals are non-
verbal. Furthermore, even for those individuals who have relatively good language
Page 33/105
skills, verbal communication is not the safest and most comfortable. Computer is also
a visually based medium, which fits well to the information processing preferences of
people with autism.
All the activities on the computer can be repeated, and it gives the relaxing
possibility to correct eventual errors.
Using computers is culturally accepted and supported by the community. First,

it means that a person with autism who is using a computer does not mark out from
her/his peers, which is usually a very important issue for the individuals and their
parents as well. Second, computer games or other computer activities can provide a
shared field of interest with typical peers, and thus a good chance to build up peer
relationships. In sum, it can support better integration of the effected people.
It is also a perfect tool to transfer and transport specific support and knowledge
across different environments (e.g. from school to home). This is highly important
issue in the field of autism, as one of the most challenging problems of intervention is
the deep difficulties with generalizing acquired skills across various settings.
From a research or intervention-development point of view, it is an additional

benefit that, during a computer-based intervention, data on usage can be recorded
and processed continuously.
3.2 Specific risks of using ICT tools in the education/intervention for

people with ASD
We also find important to keep on stressing the possible risks of using ICT or
multimedia tools with people living with ASD. These considerations have been also
discussed in Deliverable 2.1.1, Report on test methodology and research protocols, under
Ethical issues (part 1.4), but in view of importance of this topic we highlight the main
points here, as well.
The well-known general dependency on computer games or activities threatens

people with autism similarly to typical peers. However it can lead to more serious
social isolation, as it is an easy way to escape from everyday social interactions and
the abundance of anxiety and frustration they so often cause.
Again, all teenagers browsing freely on the internet can meet with dangerous
contents or sites. It is even more risky in case of individuals with autism, as they are
Page 34/105
less aware of jeopardy, have less capacity to filter information, and also can become
defenceless (e.g. by giving out their personal data).
If a pupil uses a trendy mobile device in the classroom or at public places it can
induce jealousy and therefore can raise the possibility of victimization or even
criminal acts (i.e. robbery).
Possible maladaptive attachment can be developed toward the device or the

social actor, which – instead of supporting the social integration – can lead to more
serious marginalization.
A high-tech and complex device may make the false impression in the parents
that this is a „miracle cure”, and will solve all their problems, which would be a
serious misunderstanding and can cause frustration and disappointment.
As these mobile devices are very fragile and expensive, this can put
unreasonable high level of responsibility on individuals and their parents, which can
lead to extra stress within the family.
3.3 A taxonomy of ICT tools in ASD intervention
As there are existing computer-based intervention techniques for high

functioning individuals with autism, it seems essential to analyse what kind of
problems these solutions seem to handle successfully, by what means, and what kind
of novel problems they may uncover. We present briefly our view on some existing
techniques especially relevant for HANDS, putting the emphasis on how they utilise
the possibilities we see for intervention, how they handle the specific risks, and what
is known about their effectiveness.
We will go through these programs by following the logic presented in Figure

3.1, below.
Page 35/105
Figure 3.1. A taxonomy for ICT tools in ASD intervention
3.4 Specific, skill-focused solutions - a brief overview
3.4.1 General characteristics
These programs are focused on developing one or more (usually interrelated)

specific skills (e.g. communication functions, emotion recognition, social problem
solving, etc.).
Their content is usually closed, thus the user can not modify it. Consequently
the level of individualization is limited; however most of these programs offer
different levels of tasks. These solutions are mostly restricted to the computer setting,
and also generalization of achievements is limited. In these software-based
programmes the user usually is a responder, but in case of products using virtual
environments the client can function as a (quasi-) agent.
3.4.2 Examples
(1) Golan and Baron-Cohen (2006) developed an interactive systematic guide

to emotions (The Cambridge Mindreading [CAM] Face-Voice Battery) to teach adults
with ASD to recognize complex emotions on faces and in voices. In the software, 412
emotions and mental states are introduced by silent films of faces, voice recordings
Page 36/105
and written examples of situations that involve the given emotion. The user can
access the emotion database in three ways: there is an emotion library where the user
can freely browse, play with the faces, voices and scenarios. In the learning centre
lessons and quizzes are available, and in case of the correct answer the system
provides various rewards. Finally, in the game zone enjoyable educational games are
offered. The software can be used in many different levels and within a wide age
range. (See illustration on Figure 3.2 below.)
According to their efficiency testing (Golan and Baron-Cohen, 2006) users

significantly improved in recognizing complex emotions and mental states. However
they also reported poor generalization of the acquired skills.
Figure 3.2. Screenshots from Mind Reading: The

Interactive Guide to Emotions, by S. Baron-Cohen,
O. Golan, S. Wheelwright, & J. J. Hill, 2004,
London: Jessica Kingsley Limited. Copyright
2003 by the University of Cambridge. Golan and
Baron-Cohen (2006), p. 595
(2) Bernard-Opitz et al. (2001) developed a software that presents everyday

social problem situations, and also elicits effective solutions for these. The authors
provided eight problems that targeted several important social skills (e.g. turn-
taking, requesting for help, negotiating) and these are meant to be on various levels
of difficulty. In easy stories the child meets every day conflicts (e.g. not being to able
to reach a desired object or not getting his turn), and has to find some solution for
Page 37/105
that. In the difficult situations the authors present higher-level, more complex social
conflicts (e.g. not having enough money to buy some desired item).
The conflicts are illustrated by animations and are followed also by verbal
comments by children’s voices. After watching the short clip the computer asks
children what they would do in such a situation. The software offers two
appropriate/correct and two inappropriate/incorrect solutions, presented in pictures
or animations on the screen. Children are also instructed by a light bulb on the screen
to tell their own ideas about the solution of the given conflict. If the child could
choose a good solution from the offered ones, or s/he could give an appropriate,
innovative answer, the computer praises the child and shows an animation about the
‘happy end’ of the problem situation. Furthermore, the child could select from
additional reinforcements, like sensory conditions (e.g. spirals or moving lines) or
natural conditions (e.g. a child jumping on a trampoline).
The authors measured the effectiveness of their software, and found that its
use enhanced the production of own, novel solutions of the subjects, and also
influenced the performance on untrained probes (i.e. problem situations the subjects
did not watch during the training session, but only in the test-phase).
In the same laboratory a generalization study was also conducted for the
intervention group. According to the findings, the software-trained subjects could
transfer their knowledge to real life settings only if the two situations (i.e. laboratory
and real life) were highly similar. In case of distinct problem settings the transfer was
poor.
(3) In their study, Hetzroni & Tannous (2004) developed and investigated a
computer-based intervention for enhancing communication functions of children
with autism. They focused on children who used delayed echolalia for functional
communication. They targeted the communication functions of their subjects during
three familiar daily activities: play, food and hygiene in classroom settings. Several
appropriate and inappropriate communication behaviours (e.g. delayed and
immediate echolalia, relevant and irrelevant speech and communication initiations)
were recorded during the baseline in the above-mentioned natural situations. In the
intervention phase a computer program simulated the targeted situation and these
simulations were presented to the children. The program asked a question (e.g.
“What would you like to play?” in play setting or “What would you like to eat?” in
food setting) and the subject could choose from among the activities or food items by
pressing the button presented her/his desire. Once a button was selected, the chosen
activity or object appeared in picture or in an animation.
Their findings showed that their subjects improved in their communicative

functions (i.e. their irrelevant echolalia decreased and their communicative speech
Page 38/105
increased) after they could practice these functions in a controlled, structured and
familiar context through computer simulation. The most important result of this
study was that the development of the target skills manifested also in natural
settings. The marked shortcoming of this research was the minor sample (with
altogether three subjects), and also some unclear features of the efficiency testing.
(4) Mitchell et al (2007) used virtual environment technique to authentically

simulate real-life situations and, by that, improving restaurant skills of teenagers
with ASD. The key learning objectives were skills of finding a place to sit, and asking
appropriate questions. Furthermore, they aimed to investigate generalization of skills
across contexts. The virtual environments were built with Superscape Virtual Reality
Toolkit™, and ran on a computer using Visualiser™ software. The setting they
simulated on the computer was a virtual cafe where the subject could move around,
initiate interactions or activate objects with a joystick and a mouse. The users’ actual
task was to find an appropriate seat to sit down from six available tables with chairs,
and ask appropriate questions if needed. The program consisted of four levels of
complexity of the scene (e.g., at a higher level there were more people in the
situation, or some background noise was introduced). (See illustration in Figure 3.3
below.) The subject also had to reason verbally why s/he delivers the actual activity.
Figure 3.3. VE Level 1 (Empty Cafe) and VE Level 4 (Busy Cafe);

Parsons et al (2007), p. 593
In their study, the auhors showed videos of real cafes to the teenagers and
asked them to tell where and why there they would sit down, and what they would
say to others at the table, and why. In the intervention phase the subjects had
training sessions with the virtual cafe software. Finally, they tested their skills both in
the virtual environment and in video scenes as well. Independent raters coded and
rated the quality of the subjects’ judgements and reasonings as well. The results they
reported confirmed that both the speed of task completion and the level of
explanations improved.
Page 39/105
Note: there are several other specific-skill-focused ICT-based interventions in autism

literature, the above reviewed are only typical examples.
3.5 Comprehensive interactive software solutions: computer-based
These complex software packages usually offer solutions to multiple problems

in a complex way. Their contents can be varied according to some main
characteristics of the individual who uses the software (e.g., age, skill level).
Therefore the content is more varied than in case of ‘skill-specific software products’,
but these solutions are still quite closed. In case of one specific subject of
intervention, more than one user can be involved in using the same software (e.g. the
teacher and the parents of the same child).
These programs are usually developed for parents, so their utmost advantage
is that there is no essential need for a trained professional to use them effectively.
That is, the intervention methods/activities are offered by the computer, and the
intervention is carried out either by the parent (by off-computer activities), or at the
computer by the child (on-computer activities). These programs are able to give
structured feedback on the progress. Usually they work with evidence-based
theoretical and therapeutic approaches as bases of intervention techniques.
3.5.2 Examples
(1) AutismPro™ is an on-line software which gives guidelines to its users how
to teach and treat a child with autism. The software offers comprehensive and
detailed curriculum in eight developmental areas (social, emotional, communication,
academic, language, understanding, self-care and motor skills). It is a multimedia
training offering step-by-step activity protocols to the user to follow with her/his
own child, illustrated by several video demonstrations. With the help of the software
the progress of the child can be precisely followed and, according to it, an
individualized treatment plan can be worked out. Several experts and key educators
– who are familiar with evidence-based methods in autism and also their theoretical
background – helped produce and review the content and the logic behind the
development of AutismPro.
An on-line parent survey was designed and conducted (Howroyd and Peeters,
2007) to measure if this technology can support the families with basic treatment
Page 40/105
guidelines just after the autism diagnosis, and whether it enables the parents to
become active participants in the intervention process. They found that most of the
users reported that AutismPro helped them decide about methods, goals, supports
and teaching strategies, and also helped them to structure their interventions.
(2) TeachTown™ software (Whalen et al, 2006) teaches receptive language,

social understanding, self-help skills, attention, memory, auditory processing and
early academic skills through a comprehensive curriculum. It uses the best practices
of applied behaviour analysis (ABA) within a developmental framework. They apply
intermittent reinforcement schedule – the rewards are interesting video games at
different levels and fields of interests. The software is able to follow the child’s
performance in each task and automatically gives prompts when s/he fails, or fades
help when performance improves.
The authors investigated the effect of the software within a multiple baseline
design on the subjects’ spontaneous language and social behaviour. According to
their results, the children who used the software with their parents presented more
spontaneous comments during the treatment sessions, and less inappropriate
language in the generalization phase (i.e. off-computer games with parents after
treatment sessions). In social behaviours (e.g., in rates of looking or showing positive
affect toward others) they could not register any significant changes.
3.6 Comprehensive interactive software solutions: professional-based
These software products offer solutions to multiple problems in a complex

way. Their content is open, that is the input and output is highly individualized.
Usually more than one person provide data for the system, and more users are able
to work with the program Most of them are underlain by evidence-based
theoretical/therapeutic approaches. These software systems are able to collect and
process data in an on-line fashion.
The intervention running on the computer is designed by professionals and

the computer intervention is a part of the complex therapy, and is embedded into a
comprehensive educational environment.
3.6.2 Example: SymTrend© (Calvanio, Mesibov & O’Callaghan, 2006)

(www.symtrend.com)
Page 41/105
This product has two different – although similar – sections, applicable for
individuals with autism (i.e., for younger or lower-functioning ones) and for users
with Asperger syndrome (i.e., for older or more able/high-functioning ones).
The Autism section is a caregiver support device which can be used on a
desktop computer or on a mobile tool (laptop or palmtop). It basically provides a
useful and easy-to-use structure to observe and record the child’s behaviour, and
also some guidelines to handle behaviour problems. The systematically collected and
preserved data give a useful base for the teachers to plan interventions for the child.
Its main functions are the followings:
Recording symptoms
Here parents can record data on daily living activities, school functioning,
social interactions and also problematic behaviours they intend to change. For this
purpose the system provides Log Screens, where different check-lists appear and the
parent can indicate the relevant behaviours. For an example see Figure 3.4.
Tracking Progress and Impact
Using the Progress Profile the impact of an intervention can be followed on a

colour chart. It is suggested to use for creating and updating the educational plan.
Keeping records
The parents can store all the relevant data in a comprehensible structure here.
This information can be shared with various professionals in various times and can
be the basis of their co-operation.
When-To and How-To Reminders
The system is able to alert the parents to keep in mind important events (e.g.
medication, scheduled activities, etc.), and also remind them the way the given
activity should be delivered, or how a disruptive behaviour should be handled. For
an example, see Figure 3.4.
Page 42/105
Figure 3.4. Sample Log Screen for data recording, and sample guidelines for
handling a temper tantrum. Source of illustrations: www.symtrend.com,
Copyright 2005 SymTrend, Inc. All rights reserved.
The Asperger syndrome section can be used by the client solo or tandem with
a parent/professional. It is also able to record data here, but its main function is to
give individualized reminders for the user in different settings. The reminders are
developed and uploaded by teachers according to the difficulties and skills of the
pupils. In some cases the system automatically presents the scheduled reminder.
However, the pupil can also ask for guidelines if s/he needs. For example, basic rules
of conversation can be presented on the screen, so the pupil can control her/his own
behaviour on-line, during the actual discourse. The pupil has the possibility to rate
his own behaviour in different settings on relevant check-lists. To support the self-
evaluation, the system also provides clear definitions of the given behaviours to be
rated. See the example on Figure 3.5.
Figure 3.5 An example of the Cooperation Rating

Scale; the behavioural requirements are available in
the information screen.
Page 43/105
3.7 Conclusions for HANDS
Reviewing above some relevant ICT tools in the field of ASD intervention
several conclusions for the HANDS toolset can be drawn. Here we list these and we
also built them into our suggestions on software design and content (Sections 6&7).
(1) The toolset should be able to record, process and store user data.
(2) The HANDS intervention should be based on a comprehensive curriculum.
(3) The intervention should be underlain by evidence-based theoretical
approaches.
(4) The content of the software should be highly individualized, however ready-
to-use software products can be effectively and usefully integrated into the
system.
(5) The HANDS toolset should be a part of the broader education programme of
the pupil.
(6) The efficiency of the method should be tested. (On this issue see also our
deliverable on test methodology, D2.1.1.)
(7) Within this, a great emphasis should be put on the generalization of acquired
skills across settings.
Page 44/105
4 Suggestions on hardware choice
The aim of this section is to summarise the consequences of Sections 1-3 above
(neurocognitive bases of autism, principles of therapeutical interventions, and
existing ICT solutions) which are relevant concerning a well-based choice of the
specific mobile ICT set to be used in the HANDS project. The brief discussion here
will loosely follow the logic of Sections 1-3, first summarising considerations from a
neuro-cognitive point of view, then from a pedagogical-therapeutical and a user-
centred approach.
4.1 Neuro-cognitive aspects
The following 5 aspects of the neurocognitive basis and its behavioural

consequences are relevant concerning hardware choice in HANDS project.
Variability of assets and difficulties.

As discussed before, both the specific neurocognitive features and their
behavioural consequences show a remarkable variability in autism spectrum
disorders. The general implication of this fact is that preferable and adequate
hardware features expectedly show also a considerable variability. Therefore a
considerable space should be left for individualised choice or customisation of some hardware
features – such as the means of control and input (touch-screen vs. keyboard), the
number and complexity of available functions, the complexity of the visual
appearance of the user surface, and so on.
Executive function problems.

This bunch of neuro-cognitive difficulties in autism leads to limitations in
flexibility and creativity in behavioural procedures, in generalisation of knowledge
and procedures, in systematic search, in controlling attention flexibly, and so on.
These difficulties make a hardware platform the most adequate if
Page 45/105
• it allows the simplest possible use,

• it does not necessarily offer any non-functional procedure and operation,
• necessitates the possible smallest flexibility and creativity from the part of the
user,
• does not necessarily offer not-strictly-functional hardware items.
A hardware platform seriously violating these heuristics may provoke

repetitive failures, getting stuck in errors, diverting attention to non-functional
details in some individuals with ASD – and these errors may occur even more
strongly in stressful situations (precisely when the HANDS software should be of the
most help.)
Weak central coherence (difficulties in perceptual and cognitive integration).

The facts that (1) most people with autism have various difficulties in the
normally automatic and non-conscious processes of perception which integrate the
parts of incoming information into meaningful patterns, and (2) this difficulty results
occasionally in failure to recognise relevant patterns and meanings, and attentionally
focusing onto irrelevant details, make a hardware platform adequate if
• it can be perceptually as simple as possible;

• it is able to contain as few details as possible, even in functional items;
• it does not necessarily contain any non-functional items (labels, logos, extra
buttons, etc.);
• it is clearly structured, that is, functionally relevant functional parts (buttons,
screen, etc.) are clearly marked visually.
A hardware platform seriously violating these heuristics may lead to occasional

difficulties to find relevant items, to catch attention repeatedly by irrelevant details –
in some individuals with ASD. These errors may become even stronger under
psychological pressure.
Sensory anomalies.
People with autism show several sensory anomalies, related to the perception
of simple, elementary aspects of objects and surfaces and other phenomena – and this
is so in various modalities: visual, tactile (touching), and auditory (hearing). On the
other hand, several of people with ASD are strongly but non-functionally attracted
by certain, individualised physical stimulation: repeated touching of certain surfaces,
Page 46/105
listening to specific sounds again and again, or watching certain visual patterns.
Therefore, an adequate ICT set for individuals with autism is
• physically as simple as possible, both in form and surface (texture);

• has a strictly functional structure and surface (texture).
Again a hardware platform seriously violating these heuristics may lead to

occasional difficulties to find relevant items, may divert attention by irrelevant
structure, and may provoke irresistible interest in specific, non-functional details
(such as obsessive touching of a part of the device, or obsessive observation of some
detail of it) – in some individuals. Again, these maladaptive responses may become
stronger if the individual is in a stressful situation.
Motor problems.
Problems and difficulties with fine motor control – control of refined
manipulation – are relevant here. These are quite variable within autism, but in some
individuals they may prevent the use of a touch screen or a keyboard with rather
small buttons. Therefore it is important to use such devices – at least for individuals
with fine motor problems – that
• allow the use of relatively big keyboards / buttons to control the device;
• or a voice control option may also prove to be useful in some individuals with
especially serious fine motor control difficulties.
4.2 Pedagogical-therapeutical aspects & user aspects
Many of the above-presented considerations naturally emerge from a

pedagogical point of view, too. We do not repeat these here. Instead, we add a few
points that are relevant from a more directly user-centred approach. It must be noted
that, at some points, these are in conflict with the above-presented considerations. It
is always – in case of each pupil (user) – a pedagogical challenge to create the optimal
compromise between the careful psychiatric-psychological-pedagogical considerations, on the
one hand, and the needs and desires of the users.
• If possible, the smartphone/PDA should not be just a pedagogical tool. In an

optimal case, the smartphone/PDA is a natural part of the user’s life, and is
also perceived so by the social environment – primarily, by peers. That is, in
Page 47/105
the optimal case, several non-autism-related, non-pedagogical functions can

also be used by the user.
• A visually attractive, ‘trendy’ set is more motivating for many potential users.
• A high-resolution, large screen is desirable for several reasons,
• as well as fast and reliable functioning.
• A shock-proof (unbreakable), waterproof set may significantly decrease
occasional anxiety about damaging it accidentally.
4.3 Summary
Above we formulated several points which should carefully be taken into

consideration when choosing the appropriate hardware (smartphone/PDA) for a
specific user. Considerations from the psychiatric-psychological-pedagogical point of
view, which put the emphasis on maximum efficiency and minimal risks, and
considerations coming from the user’s needs and desires may come into conflicts at
several points. It is always – in case of each pupil (user) – a pedagogical challenge to
create the optimal compromise between the careful psychiatric-psychological-
pedagogical considerations, on the one hand, and the needs and desires of the users,
on the other. Therefore, hardware choice itself, in any case, requires careful
individual consideration, an optimisation taking pedagogical efficiency, user’s
desires, and potential risks all into account.
Page 48/105
5 Suggestions on platform choice
The aim of this section is to summarise the consequences of Sections 1-3 above
(neurocognitive bases of autism, principles of therapeutical interventions, and
existing ICT solutions) which are relevant concerning a well-based choice of the
software platform to be used in the HANDS project. As in Section 4, the discussion
here will loosely again follow the logic of Sections 1-3, first summarising
considerations from a neurocognitive point of view, then from a pedagogical point of
view and a user-centred approach.
5.1 Neuro-cognitive aspects.
By and large, similar considerations apply to the decision on the software

platform (operation system) to be applied in the HANDS project, as to the hardware
choice. Again, 3 aspects of the neuro-cognitive background and behavioural
characteristics of autism spectrum conditions are specifically relevant here. Of
course, all the cautions made above on the hardware choice are also relevant, as the
software platform must support the specific interfaces (sound control, keyboard vs.
touch screen) that individually might be necessary for some subjects with autism.
Executive function problems.

These difficulties, as described above generally in Section 1, and again briefly at
the hardware requirements, put some constraints on the software platform, too, that
is to be applied in the Project. These are similar to those about hardware platform:
• It should allow the simplest possible use,

• it should not necessarily offer any non-strictly- functional procedure and operation
(i.e., irrelevant to the aims of the HANDS software),
• it should require the possible smallest flexibility from the part of the user,
• it should not offer not-strictly-functional software items, as ‘extras’.
Weak central coherence (difficulties in perceptual and cognitive integration).

Again, these difficulties have been described in a more detailed way (Section 1),
and briefly summarised at 4.1. The constraints they impose on the software platform
Page 49/105
are also analogous to those they impose on the hardware platform:
• It must be perceptually as simple as possible;

• it should contain as few visual details as possible, even in strictly functional items;
• it should avoid containing any non-functional items (labels, logos, extra buttons,
various software components and services);
• it should be very clearly structured, that is, functionally relevant functional parts
should be very clearly marked and visually demarcated from each other.
Variability of strengths, difficulties, limitations.

As discussed and mentioned already, the considerable variability in the
specific problems, limitations and particular strengths of individuals with autism
make it a very strong requirement on the software platform to be applied in the
HANDS project, that it must allow the highest possible level of customisation. This
should be true of such fundamental aspects as
• the visual outlook of the user surface – e.g., for subjects with deep difficulties in
controlling visual attention and in perception, the simplest visual outlook
without any non-functional elements can be the most effective; while for other
subject a vivid, aesthetically attractive user surface may be more effective as a
motivational factor;
• the menu system of the user surface – e.g. for subjects with deeper executive
difficulties it should be possible to create an extremely simple menu system,
with a minimal number of levels and with a minimal complexity of tree
structure; while for executively less impaired subjects a more complex menu
system may indeed offer more useful possibilities;
• the number of offered program components and services – e.g., for many subjects
with more serious executive and behavioural control problems an abundance
of offered programs and services may be puzzling, diverting, des-organising
of attention and of appropriate decisions on what to use and how; while other
subjects without or with less deep executive control problems may more
flexibly exploit such possibilities.
Page 50/105
5.2 Pedagogical-therapeutical aspects & user aspects
Again, many of the above-presented considerations naturally emerge from a

pedagogical point of view, too. These are not re-phrased here, but a few points that
are relevant from a more directly user-centred approach are added.
• The overall use of the platform should be as easy as possible.

• Customisation should allow creating even extremely simple user surface, with
a minimal number of icons, pictograms, other visual elements, and available
functions.
• The possibility to take, store and smoothly play photos and videos, as well as
record and play sounds on a high quality is essential.
• Uploading new contents should not require specific skills or significant
training.
• Icons and pictograms should be easy-to-recognise, and should not rely on
symbolic meaning.
• The functioning of the platform should be as fast as possible, to avoid stressful
delays in critical situations.
The decision on the software platform to be used in the HANDS project is a

consortium-level decision – so in this case it is not an individualised pedagogical task
to make a compromise between psychiatric-psychological-pedagogical
considerations, on the one hand, and the needs and desires of the users, on the other.
Page 51/105
6 General remarks on persuasive software design for autism
The aim of this brief section is to formulate a few general, but important remarks
on software design principles in the HANDS project. The major message of this
section is that while the general principles, components and tools of Persuasive
Design are developed primarily for ‘neurotypical adult agents’ as the target
population, people with autism, including high functioning teenagers with autism
are not ‘neurotypical adult agents’.
To put it in other words, several of those design solutions within Persuasive

Design that have proven to work efficiently and, at the same time, (ethically) safely
in typical adult users, may not work efficiently, and/or may cause specific but
potentially serious ethical and/or psychological problems in high functioning
individuals (teenagers) with autism.
Therefore, the usual principles and toolkit of Persuasive Design should be used
with an extreme care that is based on a clear understanding of the specific
characteristics and limitations of individuals with autism. As these characteristics
and limitations make these individuals atypical agents in many respects, the
principles and the specific toolkit of Persuasive Technology must be adapted to
these, in order to maximise efficiency, and, equally importantly, to minimise ethical-
psychological risks.
On the basis of our discussion in Section 1 of this document, we summarise

briefly those aspects of autism that are relevant for conceptualising high functioning
individuals with autism as atypical agents (users) for Persuasive Design. With some
simplification, we shall contrast three cognitive features of autism to three key
functional aspects of using computers in Persuasive Design. Our perception of
Persuasive Design is based on Fogg (2003).
6.1 Naïve Theory of Mind impairment – atypical understanding of own

and others’ agency. Computers as Social Actors in Persuasive
Design.
People with autism have difficulties of varying depth to understand their own
agency (that is, understanding themselves as agents with thoughts, desires,
emotions, intentions) as well as others as agents. This complex and abstract limitation
is especially relevant in connection to the strategy in Persuasive Technology to apply
computers as social actors for more effective persuasion.
Page 52/105
Limited understanding of agency in autism can have various implications in

relation to the social actor strategy:
• One such expectable implication is that this strategy has not, or has just
limited efficiency in some subjects with autism. Where ‘blindness to agency’ is
so pervasive, the subject may be non-sensitive to the social cues built into the
persuasive software.
• On the other extreme, another expectable outcome is that some individuals

with autism cannot clearly bear in mind the borderline between real social
agents and simulated social agents. An intuitive distinction between the two is
expected naturally from all neurotypical users: the software designer will trust
in the user’s clear understanding of the above borderline. In some cases of
autism the user may gradually come to treat the simulated actor as a real
social actor, and may develop unrealistic trust, reliance, and/or strong and
long-lasting emotions (attachment or even anger). These developments could
clearly lead to very serious problems.
• In between the two extreme variations above we shall most probably find a
continuum of perception of the simulated social actor, and many high
functioning individuals (teenagers) will be able to perceive the simulated
social actor in the way as neurotypical users do. However, the above-outlined,
and not at all marginal reactions must make the software designers quite
cautious about applying the usual principles of Persuasive Design when
designing softwares for users with ASD – in order to make the HANDS software
appropriate to the broadest possible range of individuals with ASD.
6.2 Executive Function Deficits – atypical active agency. Computers as

Tool and Medium in Persuasive Design.
As we outlined in Section 1, all individuals with autism show some level of

difficulties in executive functions, that is, difficulties in organising their own
behaviour and cognition in a flexible and adaptive way to reach one’s own goals.
This means, in other words, that they are atypical as active, acting agents, too. They have
problems with, to varying degree, to set up their own goals, to create and keep in
mind plans to reach those goals, to execute the plan flexibly, to initiate their own
actions, and so on. Here Persuasive Design, in the first approximation, seems to offer
promising vistas, as the potential of computers to provide exploration of cause-and-
effect relationships, vicarious experiences, and occasions to rehearse behaviours
Page 53/105
seem highly useful for individuals with autism. However, various potential
limitations and risks may arise:
• Limited understanding of the goal of using the software, or understanding it,

but loosing it while using the software.
• Limited creativity and flexibility in using the software: instead of using the
software with a creativity and flexibility expected generally from neurotypical
users, rigid, non-adaptive, inefficient strategies may appear in the autistic
user’s behaviour.
• Limited transfer of acquired skills and knowledge form the simulated/virtual

situations to real-life situations.
• Limited understanding of symbols and other representations generally well-

understood by the neurotypical user.
• These limitations again will also be present to a varying degree – and for
many high functioning individuals with ASD these functional elements of
Persuasive Design will work as they work in neurotypical subjects. However,
the above-outlined, and not at all marginal reactions must make the software
developers quite cautions about applying the usual principles of Persuasive
Design when designing softwares for users with ASD – in order to make the
HANDS software appropriate to the broadest possible range of individuals with ASD.
6.3 Weak central coherence – atypical perceptual agency. Computers as

Medium in Persuasive Design.
As it follows from weak central coherence (detail focused processing style), and
other perceptual-attentional characteristics of autism (see again Section 1), people
with autism are atypical perceptual agents, too. That is, the way they perceive the
world – though again shows remarkable differences between individuals – is not the
same as neurotypicals do. The following problems & limitations may emerge in some
individuals:
• Getting lost in too much information offered.
• Perceiving isolated parts instead of coherent patterns.
• Perceiving tiny differences where others perceive similarity.
Page 54/105
• Not perceiving as salient what is meant as salient.
• Inability to follow too rapid information.
• We must emphasise again that for many individuals with high functioning
autism the usual visual and other perceptual elements of Persuasive Design
will work as they work in neurotypical subjects. However, the above-outlined,
and not at all marginal reactions must make the software developers quite
cautions about applying the usual principles of Persuasive Design when
designing softwares for users with ASD – in order to make the HANDS software
appropriate to the broadest possible range of individuals with ASD.
6.4 What should be the general strategy for software design, then?
How should software designers reflect to the above-outlined atypical

limitations and characteristics?
First of all, this is not an all-or-none matter: even if these considerations do not
get implemented in the process of software designing, the arising software product
will still be appropriate for a part of high functioning individuals with ASD. This is
so, as in many cases the above-explained limitations are not beyond a threshold, so
they will not hamper significantly the use of the software.
That is, the more thoroughly the above-outlined characteristics are taken into
consideration in the design process, the broader range of individuals can make use of the
HANDS software with less risks and limitations.
Therefore, the right general strategy appears to be the following for software
designing:
1. Take a generally cautious approach: take the above considerations very

seriously as a default, but allow enrichment of the ‘purist’ default solutions by
allowing the biggest possible space for customisation.
2. That is, the default software solutions should be as simple as possible:

a. in terms of using social actor techniques
b. in terms of complexity of navigation
c. in terms of expected flexibility and creativity of users
d. in terms of applied symbols, pictograms, visual effects
e. in terms of available extra functions.
Page 55/105
3. At the same time, widespread possibility for enrichment of this ‘purist’ default core
should be provided via customisation – as enrichment is desirable, when possible,
both from a therapeutic-pedagogical point of view, and for users’ desires for
aesthetic and individualised appearance and functioning.
4. This actual enrichment (customisation), in the process of pedagogical-

therapeutical application should be very careful and, therefore, gradual, in
order to avoid the appearance of the above-outlined problems and risks. If
possible, software design itself should channel customisation (enrichment) into a
careful and gradual process.
Page 56/105
7 Suggestions / requirements on software design and

content
As we have already emphasised, the HANDS intervention is not a novel kind of

intervention, but an extension of those therapeutic approaches for ASD which are
evidence-based, complex, integrated and use developmental-behavioural
methodology. In other words, the HANDS toolkit is a new medium of the well-based
‘traditional’ treatment of autism, and offers several new possibilities and benefits: it
supports the transfer of knowledge, its application is faster, more flexible, the tool
itself is culturally more acceptable, etc.
In this section (as also in section 2) our goal is to give structured information for
HANDS software developers to understand deeper the main principles of effective
therapy in autism – as these strongly determine the content of the software. First, we
give a brief summary of the therapeutic consequences of cognitive psychological
background of ASD. Second, we interpret the functions of HANDS from an ASD
expert’s perspective, and we specify the content of four functions. Third, we show
some specific examples (use cases) of these functions – to adapt to software
developers’ needs.
Naturally, our suggestions do not cover all of the aspects of comprehensive

ASD interventions, but focus on those techniques which are appropriate to apply in
the HANDS toolkit. The more sophisticated methods are still in the therapist’s
hands…
7.1 Cognitive psychological & therapeutic aspects
The main cognitive psychological aspects of the HANDS-contents could not be

separated strictly from the therapeutic considerations. On the strength of cognitive
psychological background we can collect the main principles and the methodological
toolkit of effective, evidence-based therapy. (For descriptions of the specific cognitive
deficits see section 1.)
Impairment in naïve theory of mind leads to a need for such therapeutic

approaches in which (1) socio-communicative skills and social understanding are in
focus, (2) application of alternative-augmentative communication techniques,
prothetic environment, visual support, cognitive-behavioural approaches are
consequential, and (3) the environment is emotionally safe via predictability and
comprehensibility.
Page 57/105
The deficit in executive functions adds (1) further therapeutic aims: coping with
unstructured time, to improve problem solving, transfer of knowledge, organisation
of behaviour; (2) determines the effective ways/techniques of the support: using
behavioural algorithms, time-schedules, work-organisation, visual feed-back; and (3)
it draws attention to other essential aspects of creating emotionally safe therapeutical
context, such as support of the experience of independency and competency.
Weak central coherence, as a specific ‘cognitive style’ in autism, also has

influence to the intervention: (1) raising the relevant stimuli is an important aspect of
the focus of the therapy, and (2) giving only the significant information in any
modality and teaching new skills in an environment without diverting stimuli are
crucial elements of the methodological toolkit.
7.2 Components of HANDS
In the followings we adjust the crucial methodological elements of ASD

interventions to HANDS-functions, and additionally we give a guideline for
appropriate individualisation. Here we specify four of the six HANDS functions
(HIPD, SSSI, TT & Tin), as these are adequate to be integrated into an autism-specific
therapeutic framework. First, we recall the original, software designer’s definition of
the given HANDS function, and then we ‘translate’ this into an autism expert’s
interpretation. We identify the focus of the therapy, and give a collection of
appropriate and relevant, ‘traditional’ techniques/methods, which have to be
adapted to HANDS toolkit.
It has to be clarified that these functions are not necessarily informative for
pupils with ASD. For a child or an adult with ASD it could be irrelevant or
distractive to know which function is to be applied in a given moment. They need
clear, well-known, comprehensible and individualised information/support in actual
situations – irrespectively of the formal structure of the HANDS-content. (This is an
issue of software design, too.)
We would like to emphasise that the separation of functions and contents is

artificial, there are unavoidable overlaps. Thus, the ‘permeability’ of the different
HANDS-functions is required. (Again, this is a relevant point for software design.).
Please note, there is no crucial difference in applied techniques between the HANDS
functions, the difference lies in the content/target skills.
Page 58/105
7.2.1 The Handy Interactive Persuasive Diary (HIPD)
The software designer’s definition
“An interactive calendar function with usual calendar facilities, but also with
configurable/programmable abilities and “knowledge” about situations, where the
user is more likely to be persuaded to adopt a new behaviour or attitude. It is also
capable of raising the awareness of problematic behaviour. It is interactive and able
to initiate a user session autonomously.” (Project Proposal; Annex I; pp. 12)
The autism expert’s interpretation
In case of HIPD the comprehensive aim of the intervention is facilitating

independency and competence. Here we focus on specific skills and behaviours
connected to predictability and organisation of time; predictability and organisation
of self help & work activities. Additionally, there could be an individualised reward
system (but it is adequate in other HANDS functions, too). The HIPD is appropriate
for completion or replacement of those ‘traditional’ tools which are essential
elements of the ‘prothetic environment’ (such as time-schedules, behavioural
algorithms, work organisation). The toolkit of the HIPD mainly (but not only)
focused to those difficulties which are caused by executive dysfunctioning.
In the followings we give brief descriptions of the ‘traditional’ intervention

techniques which are adaptable to HIPD toolkit. (For more specific examples see
section 7.3.)
(0) Basis: visual support is a comprehensive technique in ASD interventions. It helps

to compensate not only verbal communicational problems (e.g., low level of
receptive language), but executive dysfunctioning as well, and seems useful in social-
skill trainings. The appropriate visual information (pictures, photographs,
pictograms, written instructions) is comprehensible in an individualised manner, is
not transitional, and the timing of information processing can be controlled by the
individual. It is important not only for less verbal subjects with ASD, but also fir
those who have relative good language skills. Understanding the relevance of visual
help is essential not only in developing HIPD, but in thinking over other HANDS-
functions too.
However, we should think over the possible benefits and functions of using
auxiliary stimuli in other modalities – if it is individually appropriate (e.g. using a
sound or vibration as a reminder; using a well known music as a sign of passing
time).
Page 59/105
(1) Using time-schedules, as one of the possible fields of applying visual support, is
not a goal of the therapy, but an important way of developing emotionally safe,
comprehensible and predictable environment. It is meant to answer such very
important questions as ‘What?’; ‘Where?’; ‘How long does it take?’ and ‘What is the
next activity?’. The time-schedule could contain several additional individually
relevant information, such as the name (and/or photograph) of other persons
participating the activity; the settings; alternatives; rewards, etc. Note that the
complexity and abstraction of the visual help has to be individually appropriate.
A time-schedule with photographs. It This time-schedule contains more

shows only a small part of the day. abstract pictures, and shows the whole
day.
Page 60/105
A mobile time-schedule for out-of- A time-schedule can give information not only about a
school activities (going to day but about a week, a month or a year…
swimming pool.)
(2) Activity algorithms have crucial role in supporting the organisation of different
behaviours. Here we discuss only those algorithms which are relevant for the HIPD,
supporting daily living and work skills/behaviours. Again, for people with autism
(as for everybody, but with a lower threshold) unpredictability and
incomprehensibility leads to high level of stress. To avoid this, usually they do not
co-operate without specific help to understand clearly the expectations. Moreover,
having impairment in executive functions means having difficulties in execution of
goal-directed behaviours, therefore people with autism need individualised help in
keeping the sequential order, monitoring the situation, keeping in mind the goal of
the given activity, impulse-control, etc. Activity algorithms could compensate the
flexible inner ‘scripts’ of several types of goal-directed behaviours, and helps
dissolve rigid routinised patterns of behaviour. For examples of ‘visual help’ see the
illustrations below.
Page 61/105
A simple algorithm for using toilet. A bit more complex algorithm for setting the table.
This algorithm helps not to execute a given This is a visual support for doing exercises. It
behaviour, but to participate in a complex shows the timing and form of possible rewards
activity (having a haircut). It gives (here: tokens) as well
comprehensible information and makes the
situation more predictable, therefore less
stressful
(3) The individualised reward system. As we have already emphasised in section 2,

in case of autism, the currently most effective interventional techniques are based on
a cognitive-behavioural approach. It is useful not only in prevention and treatment of
Page 62/105
challenging behaviours, but also in teaching new skills. A strong motivational basis
is a crucial element of these interventions. Having no or just very limited conception
of others’ mental states makes asking a child with autism to do something ‘just for
our sake’ often quite meaningless.
The reinforcements/rewards have to be positive and highly individualized,

could change over time and have different forms: e.g., favourite activity, a desired
object, food, (simple and clear) social reinforcement, or – as especially relevant in
case of a smartphone – interesting computer games.
Within the HIPD (but also in SSSI & PT/TT) a comprehensive on-line reward
system can be developed which is able to record, store and process the results of the
pupil. The input can take different forms: evaluation from the teacher/parents, self-
evaluation of the pupil, the results of tasks done via the smartphone, etc. In case of
computer game reinforcement, the system would only provide access to the game if
the pupil has enough credit gained. The TeachTown software’s reward processing
and providing system is a useful example for this function (Whalen, 2006; reviewed
above in section 3).
7.2.2 The Simple-Safe-Success Instructor (SSSI)
“This is an instructor function, which gives precise and practical advice on how
to solve a given problem. E.g. how to travel by public transportation. This function is
also configurable/programmable, and its level of support to the user can be reduced
once the user becomes better capable of managing the problem. (...) The Simple-Safe-
Success Instructor can be integrated into the HIPD.” (Project Proposal, Annex I., pp
12.)
In our view this functionality differs from HIPD primarily in its content and the
main targeted skills. The SSSI basically supports the improvement of socio-
communicational skills. However the techniques used here are highly similar to
what applied in HIPD.
Here we list some relevant, ‘traditional’, autism-specific intervention techniques

which can be applied within this HANDS functionality. We also illustrate these
methods with examples within this text and we also embed some of them into the
usecases (section 7.3).
Page 63/105
(1) Social-communication routines
These are written and unwritten social rules which are widely accepted and
expected in the society. These are closed, well-defined simple rules which apply to
everybody, therefore their individualization is limited.
For example, in case of greetings there are general rules about the time, the
formula, the situation of the greeting which can be available in cards or in a small
book for the pupil in the relevant settings.
For an ICT-based form of routine for greeting see Usecase No 2/ELTE&AF (in
this document, section 7.3).
(2) Social behaviour alternatives
This is a kind of ‘repository of examples’ about the appropriate behaviours in

specific situations where the given pupil has problems.
For example, in a waiting situation a list of accepted alternative behaviours can

be offered to the pupil, such as reading, playing on PlayStation (or even with her/his
smartphone), watching TV, etc.
(3) Positive social-communication scenarios
These are brief written and/or visualised scripts that give direct instructions in
situations which are regularly difficult for the pupil.
For example, if a pupil usually can hardly indicate that s/he has finished her/his
task, s/he can have a short written scenario with some clear steps what to do in these
situations. Like, for example:
• If I have finished the task Æ I stand up Æ I go to my teacher Æ I say: I am ready!
Using these scenarios maladaptive and disruptive behaviours can be prevented

(e.g. instead of shouting, doing something irrelevant or destroying the work s/he had
done, s/he follows the positive behavioural scenario).
Page 64/105
Visually supported positive behavioural scenario for going

to swimming pool. [iskola=school; várj=wait;
uszoda=swimming pool]
(4) Social rules
These are short instructions referring to one specific situation and one specific
behaviour. They are presented in positive form and are comprehensible for the pupil.
They are usually written and/or are supported by pictures. They raise the pupil
attention for the appropriate behaviour expected by others in the given setting.
Predictable reward is usually connected to these rules.
For example, in a classroom setting a pupil regularly catches up the lesson. He

can have a written rule saying: “You talk when the teacher asks you to!”
Page 65/105
Chris! If you work in silence you get picture!
Chris! If you finish all your tasks you get picture!
Examples of visualised social rules with predictable rewards.
(5) Diary & “Me-book”
The diary is a systematic collection of pupil’s important experiences and life

events to improve her/his self-concept, self-knowledge, episodic (personal) memory,
communication and social skills. Notes are usually made every day at home and in
the classroom as well. (Thus it also can serve the effective communication between
parents and teachers.) Written texts, photos, drawings can take place in it, also.
Examples of Diary at different levels of functioning
Page 66/105
The “Me-book” is again a tool for supporting the development of the pupil’s social
knowledge, self-image and language skills. Its formulation is highly individualized, but
usually contains lots of photos. The main topics are the followings:
(1) the pupil’s personal data, appearance, personal characteristics and their
changes over time, main activities;
(2) her/his social network, e.g. family members, school-mates, friends, their
features and connecting social routines and rules;
(3) the settings and events of the pupil’s life.
A family-tree from a “Me-book” of a 10 year old pupil with

ASD (with kind permission of the family).
(6) Social Stories
They are individualized short stories focusing on a given problem situation of the
pupil. The main principles are developed by Carol Gray (e.g., Gray, 2000). Their main goal
is to describe the social nature of the problem situation for the pupil, and to make it
comprehensible for her/him. Besides providing the relevant social information, a Social
Story also offers effective behaviour alternatives as well. Social Stories are highly relevant
for being applied on a mobile device.
(7) „Emergency button” or „Panic button”
This function should be specifically developed for HANDS toolset. It could be used
in unexpected situations where the pupil – due to the stress – is not able to use the familiar
functions of her/his mobile device appropriately. If the pupil presses the emergency/panic
button the system asks a series of yes-no questions (e.g., on where s/he is; on what the time
is; on who is around; etc.), and offers a list of possible answers to these questions to make
this function easier to use. Processing the answers and choices of the pupil, the system is
able to generate and to provide some guidelines what to do or what part of the HANDS
toolset should be used.
7.2. 3. The Personal Trainer/Travelling Trainer (PT/TT)
“A training function which is basically a simulator of problematic situations with

concrete and practical advice input, given with the necessary credibility. The Travelling
Trainer can be used whenever the user has the time and is motivated, e.g. while travelling
on the bus. The starting point is either the Simple-Safe-Success Instructor or the Handy
Interactive Persuasive Diary.” (Project Proposal, Annex I., pp 12.)
In our view the PT/TT basically functions as a training area. Here the pupil can safely
practice his skills having the possibility to make errors and also to correct them in a safe
context. It also could serve as a free-time activity organizer. It has three – more-or-less –
connected parts. The materials within PT/TT seem as strongly related the content of HIPD
and SSSI.
(1) It can contain ready-to-use training software products which can be chosen by the
teacher according to the pupil’s age, skills, interests and difficulties. These can be
changed/modified over time. Some of these software products were briefly reviewed
in section 3, but there are many more in the market.
(2) New, individualized training materials can also be the part of PT/TT, ones that were
developed personally according to the given pupil’s needs. A possible kind of this
personal content can be the so-called “Social Lexicon”. It contains a collection of clear
definitions of relevant social phrases or phenomena. It can be refreshed regularly,
according to the actual pupil’s changing needs.
(3) This component could also be used for a bit different function: to provide age-
appropriate and effective leisure activities. In free time it could offer different
interesting computer games or access to the internet. The MindReading software’s
‘Game Zone’ serves as a useful precedent (Golan and Baron-Cohen, 2006, for our
review, see section 3).
Page 68/105
7.2.3 The Individualiser (TIn)
“In order to allow users to derive maximum benefit from the HANDS toolset, it is
very important that it, to a very large extent is customisable in terms of both aesthetic
aspects of the interface and in terms of the presented functionality.(…) Furthermore, the
teacher of the young person with autism is able to customise the functionality too – even
remote controlled from the school when the pupil is on the move.” (Project Proposal,
Annex I., pp 12.)
Beyond the aesthetic customisation, here we focus on presenting support in an

individualised manner. The TIn is an odd-one-out among the HANDS functions discussed in
this document, as this could be a function available only for teachers. We suggest two main
roles of the TIn: on the one hand, this could offer assistance for teachers via offering
considerations for individualisation. On the other hand, the TIn may contain a ‘store of
specific examples’ – from HIPD, SSSI & PT/TT. We envisage this as a collection of possible
target problems and supportive/interventional tools for different level of functioning of
the subjects. These could help the teachers in developing their own, individually created
toolkits, pupil by pupil. Below we offer some key aspects to consider at individualisation.
In our view, these should form the basis of the selection of prothetic tools for a given
child/adult (see Table 7.1 below). During testing prototype 1 the consortium has the
opportunity to fill up this ‘store of examples’, as every test-site will use the HANDS toolkit
creatively for different children with different problems.
Page 69/105
THE MAIN ASPECTS OF INDIVIDUALISED SELECTION OF TOOLS/METHODS
• Motivation and interests • Fine-motor skills
• Sensory abnormalities (if any) • Eye-hand coordination
• Spread of attention • Usual reactions to frustration
• Style and speed of information processing • The level of symbol-comprehension
• Characteristics of arousal regulation • The level of language-comprehension

• The level of specification in behavioural • The level of heightening relevant elements of
algorithms complex stimuli
• Known strategies to cope with time (if any) • Understanding cause and effect relations
Table 7.1. The key aspects to consider when individualising HANDS tools in the
pedagogical-therapeutic process
Page 70/105
7.3 Use cases
SECTION: Handy Interactive Persuasive Diary – HIDP
7.3.1 Usage Narrative: No 1/ELTE&AF
B.B. is a 13 old years boy. He is able to travel independently, but he needs help to do
any kind of shopping. He would like to buy his monthly ticket by himself.
7.3.1.1 Use Case Content:
General objective: shopping
Related Objectives: organisational skills, daily life skills, following instructions, social and
communication skills
7.3.1.2 WHAT?
1. Primary actor:
Child
2. Secondary actor(s):
The teacher is setting up the tasks analysis.
The teacher is setting up the algorithm for B.B.
3. Level:
User goal level
7.3.1.3 WHY?
4. Scope:
Public transportation
5. Goal(s) and motivation:
Buying a monthly ticket by himself.
Page 71/105
6. Stakeholder(s) and interests:
Teachers and parents.
7.3.1.4 HOW?
7. Precondition(s):
Teacher is logged in.
8. Persuasive tool(s):
Tailoring, suggestion
9. Main Success Scenario:
The best scenario for him would be to follow the steps without any help.
Tin
1. Teacher and B.B. create a list with the steps of buying a monthly ticket (including
social routines and use of money).
2. B.B. goes to the cash-desk and accesses the list.
3. Options available on the phone: (1) If there is a queue or not. (2) What to do if the
cash-desk is closed./
4. There is a help button: what can B.B. do if he is anxious?
Open Issues:
Page 72/105
SECTION: SSSI or PT
Z.Z. is a 15 years old boy. He has a problem about social routines (how to greet people,
peers vs. adults, strangers vs. well-known people, etc.)
General objective: social skill: greeting
Related Objectives: social and communicational skills
7.3.2.2 WHAT?
1. Primary actor:
Child
The teacher/parents.
3. Level:
User goal level
7.3.2.3 WHY?
4. Scope:
At school, at home, out of school
Greeting people without the help of the parents/teachers.
Teachers and parents.
Page 73/105
7.3.2.4 HOW?
7. Precondition(s):
Teacher is logged in
Tailoring, suggestion.
The best scenario for him would be to use the social routines easily and without the
support of an adult.
Tin
1. Teacher and Z.Z. collect the main rules and types of greeting with associated
photos/videos.
2. Teacher is loading up photos and written worlds/sentences or videos with the
appropriate forms of greeting.
3. Z.Z. can set the age, the sex and the relationship on the PDA according to the actual
person he’s going to greet.
4. There is a help button if it is hard to choose the correct age, sex or relation to that
person.
Open Issues:
Page 74/105
SECTION: PT
Richard is a 10 years old boy. He does not look into the mirror and does not look at
photos of himself. He shows a tantrum if somebody takes a photo on him. He wants to
damage the camera. He is not willing to talk about people’s appearance (i.e., any external
features).
General objective: To get him understanding the fact that all people – including him, too -
have external features; to teach him what external features he has and how these changed
during his life, to date?
Related Objectives: understanding human body, understanding growing older
7.3.3.2 WHAT?
1. Primary actor:
Child
The parents – they provide the photos of Richard from his birth to the recent time.
The teacher – sets up a photo album for the child.
3. Level:
User goal level
7.3.3.3 WHY?
4. Scope:
Home, the school.

• To make the child understand that everybody has external features.
Page 75/105
• To make the child understand that these external features change, and it is
very interesting the follow these changes.
• To make the child understand that people are partly similar to each other
according to their external features, but, partly, they differ from each other,
too. It’s very interesting to observe these facts.
• Richard is interested in computer animations very much. It would be
possible to make advantage of this. A program dealing with people’s
appearance could be developed: for instance one, that allow one to put
human bodies together, to make changes on these bodies, including one’s
own body (image). He could collect scores by that exercises. The photos on
him could be the most valuable rewards.
Teacher and parents – to teach the child what it means: all have external features.
7.3.3.4 HOW?
7. Precondition(s):
Teacher and parents make a photo album, following Richard’s development from early
ages until recent time.
Selecting, monitoring, tailoring, matching.
The best scenario would be for Richard to play computer games involving the human
body, and with photos of his own body from his whole life.
Tin
1. Teacher and parents complete a photo album about Richard.

2. Teacher/computer expert make some computer games/programs about human
body, with Richard’s own body among them.
3. Richard plays with these games and collect scores. When he uses his own photos he
earns extra scores.
4. Hopefully he accepts his own body and its changes.
Page 76/105
Open Issues
Page 77/105
SECTION: SSSI
Edit is a 12 years old girl. She wants to hug everybody, it happens many times a day. She
wants to hug everybody at home, in the school, and everywhere else. This disturbs very
much everybody who knows Edit, and it very often shocks people who don’t know her.
General objective: to teach who is allowed to hug and when?
Related Objectives: to teach who are familiar people and who are strangers?
7.3.4.2 WHAT?
1. Primary actor:
Child
The parents – they share information on the family habits in connection with
hugging, and make a list about familiar people who are allowed to hug and who are
not allowed to hug.
The teacher – they set up a photo album for the child.
3. Level:
User goal level
7.3.4.3 WHY?
4. Scope:
Home, school

• To make the child understand which people are acquaintances, and which
ones are strangers.
Page 78/105
• To make the child understand who those familiar people are who are
allowed to hug, and who those acquaintances are who are not allowed to
hug.
• To make the child understand when she is allowed to hug those who can be
hugged.
Teacher and parents – teach the child what it means: acquaintances and stranger and
teach the child what the phenomenon of hugging means.
7.3.4.4 HOW?
7. Precondition(s):
Teacher and parents make two photo albums about acquaintances – one about those
who are allowed to hug, and one about those who are not.
Selecting, monitoring
The best scenario would be for Edit to use the photo album when she meets people,
and decide autonomously who can be hugged.
Tin
1. Teacher consults with parents about the acquaintances of the family.

2. Teacher, Edit and her parents create the albums.
3. Teacher puts signs on those people’s photos who are allowed to hug for Edit
4. Teacher/computer expert make a function which shows to Edit when it is allowed to
hug (e.g. arriving and departure, birthday, or every time when they meet). She
touches the photo and the rules of hugging appear.
5. Before meeting one she looks for the right photo.
6. If the photo of the person has a sign with ‘no’, Edit does not hug him/her.
7. If the photo of the person has a sign with ‘yes’, Edit touches his/her photo and
checks the possibilities when she is allowed to hug him/her.
8. If this occasion is suitable, she is allowed to hug him/her.
Open Issues:
Page 79/105
XY has a lot of problem in the break-times at the school. He is not able to choose a free-
time activity, just keeps running around. He likes reading, listening music, building with
Lego, drawing, etc. but he can’t consider these activities by himself. He is able to choose an
activity only if he has a choosing board with 4-6 options displayed.
General objective: choosing
Related Objectives: social skills, independence
7.3.5.2 WHAT?
1. Primary actor:
Child
Teachers.
3. Level:
User goal level
7.3.5.3 WHY?
4. Scope:
School, at break
To be able to choose appropriate free time activities.
Page 80/105
Teachers and parents – they teach relevant free time activities and choosing from
among them.
7.3.5.4 HOW?
7. Precondition(s):
Teacher or parent is logged in.
Suggestion.
The best scenario would be for XY to choose a free-time activity by himself.
The following steps might be sequential.
Tin
1. Teacher consults with XY, they collect the list of potential free-time activities.
2. Teacher and XY create a list of free-time activities.
3. Teacher adds pictures of the free-time activities.
4. When the break-time begins XY accesses the list.
5. XY choose an activity and he has to tick it.
6. A sound or other sign warns the pupil in the break to use this function (i.e., the list
of his potential free time activities).
Open Issues:
Page 81/105
Milan is a 9 years old boy. Every morning he goes to school sobbing. He wants his
father to sit next to the class door and wait for him for hours. Milan is unable to
understand that his father has a job and he should go to his workplace, but he would come
back for him in time. His father has to repeat it many times that he comes back for him in
the afternoon. However, Milan cannot be calmed down by this verbal explanation.
General objective: to get Milan understanding the order of events, and increasing his
sense of safety.
Related Objectives: using and understanding time table, reliance on parents’ return
7.3.6.2 WHAT?
1. Primary actor:
Child
The father – he shares information on the family’s daily life with the teacher.
The teacher – sets up the schedule for the child.
3. Level:
User goal level
7.3.6.3 WHY?
4. Scope:
The school.
Page 82/105

• To make the child understand that all of his family members do something
at given times.
• To make the child understand that his father will come back for him.
Teacher and father– to give the sense of security to the child
7.3.6.4 HOW?
7. Precondition(s):
Teacher and father make a timetable of the day of the family members.
Self-monitoring.
The best scenario for Milan would be to monitor the time-table when he feels insecure
and nervous, and calm himself down.
Tin
9. Teacher consults with Milan and his father about the daily programmes of the
family members.
10. Teacher, Milan and his father create a time-table about the programmes.
11. Teacher put signs to those activities that Milan and his father do together.
12. Before going to the class, Milan and his father review the time-table together.
13. When Milan feels insecure and nervous he looks through the time-table and checks
his father programs.
Open Issues:
Page 83/105
8 Initial eye-tracking findings on the user surface design
In this section we describe briefly the findings from our pilot eye-tracking study. The
aim of this study had been twofold: on the one hand, to develop the eye-tracking testing
methodology for later efficiency testing of Prototype 1 and Prototype 2 of the HANDS
software; on the other hand, to gain initial data on some basic features of the adaptive
visual design of the user surface.
We must emphasise strongly, and therefore we shall do so repeatedly in this

section, that this has indeed been a pilot study: therefore, all conclusions should be
taken as quite heuristic in their validity.
Below we report of these findings. In order to make this part of this deliverable
compact and intelligible, we shortly describe not only the findings from the pilot eye-
tracking study, but also the methodology we used. Therefore – as eye-tracking
methodology is described in details in deliverable 2.1.1, Report on test methodology and
research protocols – the methodological parts of this section are highly redundant with
Section 6 of deliverable 2.1.1.
As explained there in more details, the goal of eye movement measurement and
analysis is to gain insight into the viewer’s attentive behaviour. Eye movements reveal a
good deal about the strategies used to inspect a scene (see, e.g., van Gompel, 2007). These
strategies – that is, actual pathways along which our eyes scan a scene – are strongly
influenced by not only strictly perceptual processes, but also by non-perceptual higher
cognitive functions (see, e.g., Boraston & Blakemore, 2007; Duchowski, 2007). This fact
makes the application of eye-tracking technique highly necessary in a project designing
softwares to help individuals with autism effectively – since these subjects show several
specific features in complex cognition, as described in Section 1 of this document. Eye-
tracking technique is able to open a window on how effectively designed a user surface is,
in terms of minimising (1) the attentional effort needed to pick-up relevant information,
and (2) the non-adaptive, diverting elements of the user surface – so maximising
efficiency. Several of the cognitive functions functioning atypically in autism spectrum
disorders do influence visual attention patterns, and therefore may affect the efficiency of
the HANDS software significantly.
8.1 Equipment and settings
Eye-tracking measurements within the framework of the HANDS project have been
done and are to be done in the newly established Atypical Cognition and Communication
Page 84/105
Lab of the Department of Cognitive Psychology, ELTE University, Budapest, Hungary.

This Lab has been equipped – as briefly mentioned in section 1.6 of this document – with
an ‘Eyegaze Analysis System’ binocular, 120 Hz, desktop-mounted, infrared eye-tracker
system, controlled by a specifically modified personal computer, and equipped with a
NYAN 2.0 software for stimulus control and data analysis. The equipment (hardware) is a
product of the LC Technologies, while the analysis software was supplied by Interactive
Minds (Dresden).
The Lab was divided into two parts with a panel that separates the desk of the
experimenter from the desk with the eye-tracker, where the actual measurements are
done, and where the subject takes place (see Figure 8.1 below). Another panel was
installed to shield infra-red radiation coming from the heating system. One experimenter
welcomes, instructs and supervise the subject, controls and administers the measurements.
The experimenter withdraws to her/his own desk as the subject have been informed, have
signed the consent, and the eye-tracker has been calibrated, but he/she remains ready to
intervene and/or help, if necessary. All measurements are taken in a semi-dark
environment, in order to minimise noise from external infrared radiation.
Figure 8.1. The arrangement of the eye-tracking lab.

(For more explanation see Deliverable 2.1.1.)
At the desk with the tracker, the subject sits comfortably in a chair, at about 60 cm
optimal viewing distance from the screen of the tracker. The display size of the stimulus
smartphones were calculated according to this optimal viewing distance
Page 85/105
8.2 Subjects
Ten subjects took part in this pilot study, on a purely voluntary basis. All of them
have been informed previously about the nature, circumstances and conditions of the
study via a leaflet, and gave – or their parents gave – a written consent before actually
starting the measurements. They all have been offered the possibility to interrupt the
measurements at any point if they wish, and to finish participation. None of them did so.
The overall procedure received consent from the chairman of the Ethical Board of the
HANDS project, Prof. Søren Holm.
Neurotypical subjects
Six university pupils (3 females and 3 males), all of them between the ages of 20 and
30, without known neurocognitive disorder or uncorrected eyesight problems – as self-
reported – took part in the study.
One further neurotypical subject, a 9 year 10 month old fourth-year primary school
pupil participated. A written consent was gathered from him and also from his father
prior to actually starting the measurements. The parent stayed in a room next to the Lab
while the subject was worked with.
Subjects with autism spectrum disorder
Three subjects (all of them male) with autism spectrum disorder diagnosis took part
in this pilot study. All of them were recruited via the Autism Foundation, Budapest, a
partner institution within the HANDS project.
Two of them were pupils from the School of the Autism Foundation, both of them
high functioning boys with autism, between the ages of 9 and 11 years. Their parents
stayed with them in the Lab during the measurements, and gave written consent before
the measurements actually began.
The third subject with autism spectrum disorder was a 25.5 years old young adult
who arrived to the Lab alone.
We experienced no difficulty in co-operating with the subjects, the overall rapport

was quite positive in all cases.
Page 86/105
Note: we must emphasise that small and unbalanced sample sizes do not allow drawing
more than heuristic conclusions.
8.3 Design of the pilot study
To simulate – to some extent – the process as a subject (1) first gets acquainted with the
smartphone/palmtop itself in a relaxed context; then, (2) the subject gets acquainted with
the HANDS software in a relaxed context; and, finally, (3) the subject uses the software in
situations that involve stress to a varying extent, we designed three task conditions, and
these were applied throughout the eye-tracking testing (see in deliverable 2.1.1 in details):
Condition 1: familiarisation.
Condition 2: relaxed usage.
Condition 3: pressurised usage.
In the pilot studies we have run, six display styles were created, on the basis of the
prototype received from the software designer team (Morten Aagaard, Aalborg
University). These were the followings (see also Figures 8.2.A-F below):
Display style 1: text only

Display style 2: text + social actor
Display style 3: icons + rich information
Display style 4: icons + social actor
Display style 5: text + icons
Display style 6: text + icons + social actor
Task themes used in the pilot study were the followings:
Task theme 1: how to behave in breaks in the school?

Task theme 2: leaving home in the morning
Task theme 3: office routine at a post office
Task theme 4: breakfast alone
Task theme 5: daily shopping alone
Task theme 6: travelling alone
Page 87/105
8.4 Stimuli
That is, stimuli were little step-by-step behavioural algorithms meant to help an
individual perform the above-listed everyday tasks, shown on screens of smartphones,
presented on the eye-tracker’s screen. The task of the subjects was always to find the
actual step where the hypothetical user of the phone is at the moment. See Table 8.1 below
for the actual stimulus algorithms, and the corresponding display styles.
Page 88/105
A) text only B) text + social actor C) icons + rich info
D) icons + social actor E) text + icons F) text + icons

+ social actor
Figure 8.2.A-F. Display styles used in the pilot eye-tracking study
Page 89/105
task display Conditions 1 & 2 Condition 3

theme style (familiar + relaxed) (pressurised)
travelling text + 1 Go to the bus stop! 1 Look for the sign of the metro station!
icons + * 2 Stop there! 2 Look for the moving staircase!
social actor 3 Look at the buses arriving! 3 Get foot on the moving staircase!
4 If it’s yours coming, go to a door! 4 Hold on until you get down!
5 Wait until they get off! * 5 Find out in which direction to go!
6 Get on the bus! 6 Stand 1 metre from the edge!
7 Go inside! 7 Wait for the metro!
breaks text 1 Wait a minute sitting in your desk! 1 Wait until the teacher goes out!
2 Find your book in your bag! 2 Find out if the weather is good!
3 Take it with you to the court! 3 If the weather is good, go to the court!
4 Find an empty bench! 4 If the weather is bad, stay in the classroom!
* 5 Sit down and read from the book until the 5 If you are out, keep walking until the bell rings!
bell rings!
6 If the bell rings, go to the classroom! * 6 If you are inside keep playing on the
handy!
7 Put your book into your bag! 7 When the bell rings, go to your desk and sit
down!
leaving text + 1 Set the heating to 18 degrees! 1 Turn the taps off in the flat!
social actor 2 Turn all the taps off! 2 If the heating is on, set it to 18 degrees!
* 3 Put your shoes and coat on! 3 Get completely dressed, put your shoes on!
4 Take your bag and keys! 4 Turn all the lights off!
5 Turn all the lights off! 5 Turn the alarm on and go out!
6 Go out and lock the door! * 6 Lock the door and the bars!
7 Go to the bus stop! 7 Go to the lift!
breakfast icons + * 1 Wash your hands! 1 Get washed!
social actor 2 Go to the kitchen! * 2 Put on your clothes made ready!
3 Put a plate on the table! 3 Wash your hands!
4 Put your sandwich on it, from the fridge! 4 Lay the table for yourself!
5 Warm your tea up in the microwave oven! 5 Take out your food from the fridge!
6 Eat the sandwich and drink the tea! 6 Put it on the plate, sit down and eat it!
7 Wipe your hands in a napkin! 7 Wash your hands and clean your teeth in the
bathroom!
shopping text + 1 Take your list out! 1 Look at the list, still at home!
icons 2 Put everything, one by one, into your basket! 2 Decide which shop you have to go to!
3 Check if you have got everything! 3 Go to the shop, take a basket!
4 Go to the checkout! * 4 Put into the basket the things that are on
the list!
5 Wait until it’s your turn! 5 Go to the checkout!
6 Put everything on the conveyor belt! 6 Stand in the shortest row!
7 * Give enough money to the cashier! 7 If it’s your turn, pay!
post icons + rich 1 Find out where you can transfer money! 1 Find out what you have to arrange today!
office information 2 If there is a row, stand to the end! 2 Choose the appropriate window!
* 3 If there isn’t a row, go to the window! * 3 Stand to the end of the row!
4 If it’s your turn, say ’hello’! 4 If it’s your turn, say what you have to do!
5 Give the cheque and the money to the cashier! 5 Pay if the cashier asks!
6 Take the change in your wallet! 6 Put the money away and say ’goodbye’!
7 Say ’goodbye’ and go home! 7 Go home on the shortest way!
Table 8.1 Task themes, corresponding display styles, and the actual task contents – the latter in the
English translation. The ‘*’ and bold setting mark the item where the hypothetical actor actually is
in the algorithm.
Page 90/105
8.5 Procedure
Table 8.2 below shows the overall procedure of our pilot eye-tracking measurements.
exact /
time
rank activity content max. time
control
(minutes)
1 WARM-UP, ADAPTATION, CONSENT semi-structured open 5

2 calibration fixed, computer-based open 3
3 instruction: general fixed, computer-based open 5
4 instruction: general familiarisation 1 fixed, computer-based open 1
5 test: general familiarisation 1, ‘empty screen’ fixed, computer-based maximised 3
6 RESTING ACTIVITY semi-structured fixed 3

8 instruction: general familiarisation 2 fixed, computer-based open 1
9 test: general familiarisation 2, ‘social actor’ fixed, computer-based maximised 3

12 instruction: familiarisation, task theme 1 / display style 1 fixed, computer-based open 1
13 test: familiarisation, task theme 1 / display style 1 fixed, computer-based maximised 3
14 filling task 1.1 fixed, computer-based fixed 1
15 instruction: relaxed task, task theme 1 / display style 1 fixed, computer-based open 1
16 test: relaxed task, task theme 1 / display style 1 fixed, computer-based maximised 2
17 filling task 1.2 fixed, computer-based fixed 1
18 instruction: pressurised task, task theme 1 / display style 1 fixed, computer-based open 1
19 test: pressurised task, task theme 1 / display style 1 fixed, computer-based maximised 0,5
21-29 cycle 11-19 repeated for task theme 2 / display style 2; (3 tests) fixed, computer-based maximised 13,5
70 CLOSING semi-structured maximised 10
TOTAL POSSIBLE TESTING TIME (minutes): 136

Table 8.2. A somewhat simplified summary of the total testing procedure. Bold
typesetting marks the measurements (eye-gaze recordings).
Page 91/105
8.6 Analyses and findings
We ran 3 kinds of analyses on the fixation data recorded by the equipment:
1. Statistical analyses for rough comparisons of the distribution of fixations in the ASD
and the neurotypical groups;
2. descriptive ‘heat-map’ analyses to present qualitatively fixation distributions in the
ASD group; and
3. descriptive ‘heat-map’ analyses to present qualitatively the differences between
fixation distributions in the ASD group and those in the neurotypical group.
8.6.1 Statistical analyses
For the statistical analyses, we divided the displayed images of the smartphone into three
areas (see Figure 8.3 below):
1. Relevant area: that part of the screen which displays the elements of the behavioural
algorithm.
2. Semi-relevant area: other parts of the screen.
3. Irrelevant area: non-screen surface of the smartphone and the surrounding area.
We ran Repeated Measures Analyses of Variance to reveal any statistically significant

difference between ASD and neurotypical subjects
a. in terms of the overall duration of fixations;
b. and also in terms of the number of fixations.
None of these analyses have brought significant results, that is, no significant
difference have been found between the ASD and neurotypical subjects in terms of the
fixations targeted on relevant vs. semi-relevant vs. irrelevant areas.
Page 92/105
Figure 8.3. Defined areas of displayed smartphones for statistical analyses

of eye-gaze data.
8.6.2 Descriptive ‘heat-map’ analyses of fixation distributions in the ASD group
Table 8.3 below show the so-called ‘heat-map’ analysis findings for the most relevant
3 display styles, in all the three task conditions. This representation of the scanning
strategies highlights those areas in the presented displays which received relatively the
most attention from subjects, in this case subjects with autism spectrum disorders.
Although the low number of subjects allows only heuristic conclusions to be dawn,
nevertheless, these heat-map displays suggest that these subjects with autism showed
quite focused attention, which was directed foremost to the relevant step in the algorithm
– as can be seen nicely in relaxed and pressurised conditions.
Page 93/105
empty screen, empty screen + social actor,

familiarisation familiarisation
display style 1, display style 1, display style 1,

familiarisation condition relaxed condition pressurised condition

Page 94/105

Table 8.3. ‘Heat-map’ analyses of fixations of subjects with ASD for the most relevant
display styles, in all the three presentation conditions. Green areas mark the regions
receiving the most attention; yellow areas mark regions receiving less, but still some
significant attention from ASD subjects.
8.6.3 Descriptive ‘heat-map’ analyses of differences between fixation distributions in

the ASD group and the neurotypical group
Table 8.4 below presents heat-map analyses of the differences in fixations between the
two groups. These require very careful interpretation. Green-yellow areas mark those regions
which received significant attention from ASD subjects, but not from neurotypical subjects.
Red-orange areas mark the regions receiving significant attention from neurotypical
subjects, but not from ASD subjects. That is, areas which are not at all marked received roughly
the same amount of attention from the two groups – either a lot, or not at all.
By and large, these heat-maps also show the relatively well-focused attention of ASD
subjects as was previously shown above. One additional phenomenon which may have
importance, though, is the relative attention appearing on the social actor in the last
condition. This may signify that when subjects with ASD get tired or more stressed, they
tend to pay more attention to non-strictly-functional elements of the display, their
attention becomes less focused. This possibility requires further investigation in future
testing of Prototypes 1 and 2.
Page 95/105
empty screen, familiarisation empty screen + social actor, familiarisation
display style 1, relaxed condition display style 1, pressurised condition
Page 96/105

Table 8.4. ‘Heat-map’ analyses of differences in fixations between subjects with ASD and
neurotypical subjects for all the display styles, in the two relevant presentation conditions.
Green-yellow areas mark the regions receiving significant attention from ASD subjects but
not from neurotypical subjects; red-orange areas mark the regions receiving significant
attention from neurotypical subjects but not from ASD subjects.
8.7 Summary and requirements for the user surface design
As emphasised before, the findings of this pilot eye-tracker study are to be treated as
heuristic, primarily for the low sample size, and also for the fact that the methodology
Page 97/105
used is itself a pilot one. Nevertheless a few requirements for the visual design of the user
surface can be formulated.
As our subjects with ASD were able to locate the relevant information relatively
quickly on the smartphone screen when only the necessary information was shown, like in
display style 1 (text only; see above on Figure 8.2.A), it seems highly recommended that such a
simplest and functional display style should be the default visual outlook in the HANDS software.
This suggestion can be further motivated by the fact that enrichment of this visual
surface (like in display styles 2-6 above) did not lead – as it seemingly could not – to more
focused visual guidance. In the case of display style 6 above, however, some non-
functional visual attention appeared on the social actor, and this may indicate that such
visual elements may indeed divert attention to some extent. That is, using the simplest,
functional display style as default in the HANDS software represents the safest strategy, as non-
effective enrichment of the visual surface should be clearly avoided. That is, (1) such a
display style seems enough to guide visual attention sufficiently, and (2) unnecessarily
enriched visual surface may divert attention non-adaptively.
As, however, the enriched elements, with the above-mentioned apparent exception,
did not lead to significantly diverted attention, the possibility for a careful, step-by-step
enrichment of the visual surface – customisation – can be provided. Such an enrichment may
make the use of the software more pleasant and, in some cases, potentially more effective.
This enrichment must be, however, indeed very careful, and should be always done on a
gradual, step-by-step basis, for two reasons:
• As attentional processes are atypical and quite various cross-individually
within ASD, various individuals may show quite different ‘threshold’ where
the visual surface becomes non-adaptively rich and diverting for them.
• Attentional problems arise more strikingly under stress and/or fatigue, so a
visual surface style/enrichment which may seem adaptive under relaxed
conditions may become non-adaptive and diverting in stressful, tiring
situations – and part of the situations when the HANDS software is to be used
will certainly be such a situation.
Page 98/105
9 Summary
1. Autism, and, more broadly, autism spectrum disorders – including, among others,
Asperger’s syndrome – are human neurodevelopmental disorders. As these are
neurodevelopmental disorders, the atypical pathway to adulthood means that the
development of the brains of these individuals takes unusual routes (see more details in
Section 1).
As Persuasive Technology is at the core of the HANDS project, it is important to see

that autism is a complex neuro-cognitive condition, and not a mere matter of persuasion.
Putting it in other way, people with autism and autism spectrum condition show very
specific behavioural symptoms and problems not because they are “unconvinced” to
behave in a neurotypical way.
Therefore, one should avoid having a ‘naïve normalisation’ in mind as the goal of the
therapeutic-pedagogical intervention. Instead, a clear and detailed understanding of the
symptoms, difficulties, strengths, and the underlying psychological mechanisms is needed
in order to design appropriate therapeutical-pedagogical tools and regimes (see more
details in sections 1 & 2).
2. HANDS intervention is not a novel kind of intervention, but an extension of those

therapeutic approaches for ASD which are evidence-based, complex, integrated and use
developmental-behavioural methodology. In other words, the HANDS toolkit is a new
medium of the well-based ‘traditional’ treatment of autism. Its primary benefit is
transferring the school-based autism-specific support into different settings -- ‘having a
little teacher in the pocket’. The target areas of HANDS-based intervention are the
following:
• social skills
• communication skills
• daily living skills
• improving independence and competence in the above fields.
It must be emphasised that HANDS-based intervention cannot substitute for the
systematic therapist-based intervention, it can be rather an extended component of that.
3. It follows from the above points that creating appropriate software design and contents
requires an understanding of at least the fundamental principles of the evidence-based
systematic intervention techniques (described in more detail in Section 2).
Page 99/105
As autism is a highly heterogeneous disorder in terms of symptoms, difficulties,

strengths, and the underlying psychological mechanisms, the following general points can
be implemented only by keeping an eye on individual differences:
• Using comprehensible visual support

• Giving only the relevant information
• Giving the relevant information in a not-too-complex and not-too-abstract way
(visually, linguistically)
• Using reward systems
• Utilising available evidence-based computerised solutions (see section 3)
In section 7 we present specific suggestions about software components and contents

based on the above general principles.
5. In order to avoid serious risks, ethical and psychological problems that may arise from
an ill-designed or ill-applied software, and also to combine such a cautious approach with
the user’s natural needs for having an individualised, attractive, pleasant, and ‘trendy’
tool, we suggest the following general strategy for software design:
1. Take a generally cautious approach: take the above considerations very

seriously as a default, but allow enrichment of the ‘purist’ default solutions
by allowing the biggest possible space for customisation.
2. That is, the default software solutions should be as simple as possible:

a. in terms of using social actor techniques
b. in terms of complexity of navigation
c. in terms of expected flexibility and creativity of users
d. in terms of applied symbols, pictograms, visual effects
e. in terms of available extra functions.
3. At the same time, widespread possibility for enrichment of this ‘purist’ default
core should be provided via customisation – as enrichment is desirable, when
possible, both from a therapeutic-pedagogical point of view, and for users’
desires for aesthetic and individualised appearance and functioning.
4. This actual enrichment (customisation), in the process of pedagogical-

therapeutical application should be very careful and, therefore, gradual, in
order to avoid the appearance of the above-outlined problems and risks. If
possible, software design itself should channel customisation (enrichment) into a
careful and gradual process.
Page 100/105
A detailed argumentation for this strategy is presented in section 6.
Our pilot eye-tracking findings are in line with the above-formulated strategy – see
section 8 for details.
***
Page 101/105
10 References
Akshoomoff, N., Lord, C., Lincoln, A. J., Courchesne, R. Y., Carper, R. A., Townsend, J., &
Courchesne, E. (2004). Outcome classification of preschoolers with autism spectrum
disorders using MRI brain measures. Journal of the American Academy of Child and
Adolescent Psychiatry, 43, 349-357.
APA, American Psychiatric Association (1994): Diagnostic and statistical manual of mental
disorders, IVth edition. Washington, DC: American Psychiatric Association.
Ayres, A.J. (1979). Sensory integration and the child. Los Angeles: Western Psychological
Services.
Bailey, A., Phillips, W., & Rutter, M. (1996): Autism: Towards an Integration of Clinical,
Neuropsychological, and Neurobiological Perspectives. Journal of Child Psychology and
Psychiatry, Vol. 37, No.1., 89-126.
Baron-Cohen, S., Golan, O., Chapman, E., Granader, Y. (2007) Transported into a world
of emotion. The Psychologist 20(2):76-77
Bernard-Opitz V. Sriram N. Nakhoda-Sapuan S. (2001). Enhancing social problem solving
in children with autism and normal children through computer-assisted instruction.
Journal of Autism and Developmental Disorders, 31(4), pp. 377-384.
Beveridge, M., Jarrold, C., & Pettit, E. (2002): An experimental approach to executive
fingerprinting in young children. Infant and Child Development, 11, 107-123.
Biklen, D. (1990). Communication unbound: autism and praxis. Harvard Educational Review,
60, 291-315.
Bogdashina, O. (2003): Sensory and Perceptual Issues in Autism and Asperger Syndrome:
Different Sensory Experiences, Different Perceptual Worlds. London & New York: Jessica
Kingsley Publishing.
Bolte S. et al. (2002). The development and evaluation of a computer-based program to test
and to teach the recognition of facial affect. International Journal of Circumpolar Health,
61(Suppl. 2), pp. 61-68.
Bondy, A., Frost, L. (1994). The Picture-Exchange Communication System. Focus on Autistic
Behavior, 9, 1-19.
Boraston, Z. & Blakemore, S-J. (2007): The application of eye-tracking technology in the
study of autism. Journal of Physiology, 581, 893-898.
Calvanio, R., Mesibov, G., OʹCallaghan , C. (2006) Computer Assisted Improvement of
Social and Executive Functioning in Asperger’s Disorder. Presentation on ASAʹs 37th
National Conference on Autism Spectrum Disorders (July 13-15, 2006)
Casanova, M.F., Buxhoeveden, D.P., Gomez, J. (2003): Disruption in the inhibitory
architecture of the cell minicolumn: implications for autism. The Neuroscientist, 9 (6):
496-507
Clements, J. & Zarkowska, E. (2001). Behavioural Concerns and Autistic Spectrum Disorders:
Explanations and Strategies for Change. London: Jessica Kingsley Publishers.
Page 102/105
Clements, J. (2005) People with autism behaving badly. Jessica Kingsley Publishers
Duchowski, A. (2007): Eye Tracking Methodology: Theory and Practice. Springer.
Dunlap, G. (1999). Consensus, engagement, and family involvement for young children
with autism. Journal of the Association for Persons with Severe Handicaps, 24, 222-225.
Fogg, B. J. (2003): Persuasive Technology. Using Computers to Change What We Think and Do.
San Francisco, CA: Morgan Kaufmann Publishers.
Frith, U. & Happé, F. G. E. (1994): Autism: beyond ‘theory of mind’. Cognition, 50, 115-132.
Frith, U. (1989): Autism: Explaining the Enigma. Oxford: Basil Blackwell.
Frith, U. (2003): Autism. Explaining the Enigma. 2nd edition. Oxford: Blackwell.
Gillberg, C. (2000): An overview of the biology of autism. Conference paper. Autism-Europe
Congress, Glasgow, Scotland.
Gillberg, C. (2003): Epidemiology of autism. Conference paper. The Social Brain Conference,
Göteborg, Sweden.
Golan, O., Baron-Cohen, S. (2006) Systemizing empathy: Teaching adults with Asperger
syndrome or high-functioning autism to recognize complex emotions using
interactive multimedia. Development and Psychopathology 18:589-615
Gray, C. (2000) Writing Social Stories With Carol Gray, Accompanying Workbook to Video,
Future Horizons
Gyori, M. (2006): Autism and cognitive architecture. Budapest: Akadémiai.
Happé, F. (1993): Communicative competence and theory of mind in autism: A test of
relevance theory. Cognition, 48, 101-119.
Happé, F. (1994): Autism. An introduction to psychological theory. London: University College
of London Press.
Hetzroni, O. E., & Tannous, J. (2004). Effects of a computer-based intervention program on
the communicative functions of children with autism. Journal of Autism and
Developmental Disorders, 34, pp. 95–113.
Howlin, P. (1997). Autism. Preparing for Adulthood. London: Routledge.
Howlin, P. (2002). Treatment of Autism. (Conference paper) „…a world of difference…” NAS
40th Anniversary International Conference, London, UK.
Howlin, P. (2005): Outcome in Autism Spectrum Disorders. In: Volkmar, F.R., Paul, R.,
Klin, A., Cohen, D., eds., Handbook of Autism and Pervasive Developmental Disorders.
Third Edition. Vol. 1. Hoboken, NJ: Wiley. 201-220.
Howlin P. (2005) The effectiveness of interventions for children with autism J Neural
Transm. Suppl. 69, 101-119
Howroyd, C., Peeters T (2007 ) Parent participation in early intervention with software-
assisted guidance from AutismPro. The Good Autism Practice Journal (GAP), Volume 8,
Issue 2, pp. 31-36
Jordan, R. & Powell, S. (1995). Understanding and teaching children with Autism. Chichester:
Wiley & Sons.
Page 103/105
Jordan, R. & Powell, S. (1996). “Therapist drift”: Identifying a new phenomenon in

evaluating therapeutic approaches. In: Therapeutic Intervention in Autism- collected
papers from the conference organised by Autism Research Unit, Durham.
Kaufman, B. N. (1981). A Miracle to Believe In. New York: Doubleday.
Lovaas, O. I. (1987). Behavioral treatment and normal educational and intellectual
functioning in young autistic children. Journal of Consulting and Clinical Psychology 55,
3-9.
Mesibov, G.B., Shea, V., Schopler, E. (2004) The TEACCH Approach to Autism Spectrum
Disorders (Issues in Clinical Child Psychology) Springer-Verlag New York, LLC
Mitchell, P., Parsons, S. and Leonard, A. (2007). Using virtual environments for teaching
social understanding to 6 adolescents with autistic spectrum disorders. Journal of
Autism and Developmental Disorders, 37(3), pp. 589-600.
Miyake, A., Friedman, N.P., Emerson, M.J., Witzki, A.H., Howerter, A., Wager, T.D. (2000):
The unity and diversity of executive functions and their contributions to complex
‘frontal lobe’ tasks: A latent variable analysis. Cognitive Psychology, 41, 49-100.
Murray, D., Lesser M. (1999) Autism and computing [Online] Available from
www.autismandcomputing.org.uk
Ozonoff, S., Pennington, B. F., & Rogers, S. J. (1991a): Executive function deficits in high-
functioning autistic individuals: Relationship to theory of mind. Journal of Child
Psychology and Psychiatry, 32, 1081-1105.
Peeters, T. (1997) Autism: From Theoretical Understanding to Educational Intervention.
Singular Pub Group
Perner, J., & Lang, B. (2000): Theory of mind and executive function: is there a
developmental relationship? In: Baron-Cohen et al., 2000a.
Quill, K. A. (ed.) (2000). Do-watch-listen-say: Social and communication intervention for children
with autism. Baltimore, MD: Paul H Brookes Publishing Company.
Rimland, B. (1994). Information pack on vitamins allergies and nutritional treatments for
autism. Autism Research Review International, Information Pack P24.
Rodier, P., Hyman, S., Stodgell, C., Bennetto, L., Ingram, J., LaPoint, S., Arndt, T., Stanton,
M., Herbert, J., Bryson, S., Miller, M. (2003): Where in the Brain is Autism?
Conference paper. The Social Brain Conference, Göteborg, Sweden.
Sacks, O. (1995): An Anthropologist on Mars. In: An Anthropologist on Mars. London:
Picador, pp. 233-282.
Shea, V. & Mesibov, G. (2005): Adolescents and Adults with Autism. In: Volkmar, F.R.,
Paul, R., Klin, A., Cohen, D., eds., Handbook of Autism and Pervasive Developmental
Disorders. Third Edition. Vol. 1. Hoboken, NJ: Wiley. 288-311.
Smith, T., Buch, G. A., & Gamby, T. E. (2000). Parent directed, intensive early intervention
for children with pervasive developmental disorder. Research in Developmental
Disabilities 21, 297-309.
Sperber, D. & Wilson, D. (1986): Relevance. Communication and Cognition. Cambridge, MA:
Harvard University Press.
Page 104/105
van Gompel, R. (2007) Eye Movements: A Window on Mind and Brain. Elsevier.
Volkmar F. R., Lord C, Bailey A, Schultz R. T., Klin A. (2004). Autism and pervasive
developmental disorders. Journal of Child Psychology and Psychiatry 45(1), 135-70.
Whalen, C., Liden, L., Ingersoll, B., Dallaire, E., and Liden, S. (2006). Behavioral
Improvements Associated with Computer-Assisted Instruction for Children with
Developmental Disabilities. Journal of Speech and Language Pathology and Applied
Behavior Analysis
WHO [World Health Organisation] (1990): International Classification of Diseases. 10th
revision. Section V.: Mental and behavioural disorders (including disorders of psychological
development.) Diagnostic criteria for research. Geneva: WHO.
***
Page 105/105

Hands D2.2.1 Elte R Pu 2008-12-16

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Hands D2.2.1 Elte R Pu 2008-12-16

Încărcat de

Drepturi de autor:

Formate disponibile

HANDS, GA no 224216, Del 2.2.

This document is a detailed discussion of suggestions/requirements concerning

Project co-ordinator name, title and organisation:

0.1 The aims of this document

Third, as it comes from the highly interdisciplinary organisation of the present

Fourth, and at last, we wish to emphasise that, in our conviction, cognitive

0.2 The approach of this document

When writing this document, the following considerations determined strongly

This document presents considerations and recommendations primarily from

Expert knowledge and literature review on the diagnostic, behavioural, neuro-

Intuitive-conceptual analysis of existing ICT solutions with aims and means

Eye-tracking methodology represents the third methodological source of this

Collecting and summarising relevant contributions from various channels of

0.4 Within-document and cross-document redundancies

0.5 The structure of this deliverable

Following this introduction, a detailed overview of the nature of autism will be

1 Adolescents and adults with autism spectrum conditions –

1.1 Fundamental facts about autism spectrum conditions

Autism, and, more broadly, autism spectrum disorders – including, among

As these are neurodevelopmental disorders, the atypical pathway to adulthood

Figure 1.1. Atypical brain growth in autism, from birth to

Figure 1.2. Atypical structures in the brains of individuals with

The basis of definition and diagnosis of autism and autism spectrum

DSM-IV (APA, 1994) – are marked and qualitative developmental impairments in

(1) Qualitative developmental impairments in reciprocal social interactions and

Impairments of reciprocal social interaction. This is perhaps the most characteristic

Deficits of reciprocal communication. Both the course of linguistic-communicative

• delay or lack of development of speech, without significant compensating

Some further points can be added to this list:

• over-literalness; a tendency to interpret others’ utterances literally, and to use

Repetitive, stereotyped patterns of behaviour and narrow interest. It is generally

Beyond these core (definitive) features several other atypical patterns of

Islets of preserved abilities. Besides the low-functioning areas, however, preserved or

Uneven IQ profile. The above-discussed feature of cognition in autism seems to

Sensory-perceptual deficiencies. In autism, both atypical over-sensitivity and insensitivity

Emotional control problems. There are several manifestations of impaired emotional

As mentioned before, it is a key feature of autism and autism spectrum

• according to age (symptoms change as the individual develops);

An important consequence of the striking variability in symptoms is that any

1.2 Cognition in autism: unusual abilities

Figure 1.3. The explanatory levels and corresponding disciplines

As our knowledge on the neural background of autism and the available

As the unusual organisations of various cognitive functions represent a crucial

As the neural background of autism is on remarkable complexity, one must

functions related to sensory processing, perception, attention, learning, memory, social

1.2.1 Impairment of social cognition (‘naïve theory of mind’)

‘Naive theory of mind’ is the human cognitive-behavioural capacity to

A huge bulk of empirical studies demonstrated that individuals with autism,

As it follows from the above description of the use of normally functioning

1.2.2 Impairment in executive functions

In Ozonoff et al. (1991; p. 1083) words,

“Executive function is defined as the ability to maintain an appropriate problem-

So defined, executive functions, strongly based on human prefrontal cortex

impairment in executing action sequences in complex but flexible ways. Executive

As it follows from the description of executive functions and the consequences

1.2.3 Weak central coherence, or detail-focused processing style

This is probably the least well-understood cognitive aspect of autism,

 problems with perceptual integration;

1.2.4 Some cautionary notes: the space for (technology-based) intervention

As it will be explained in Section 2 to more depth, presently available

Also, as Persuasive Technology is at the core of the HANDS project, it is

Moreover, as several cognitive functions work atypically in autism spectrum

More on the reasonable perspectives for psychological-pedagogical intervention

problems with perceptual integration;

Involvement of the parents