22/04/13

Current User Interfaces vs

Emerging User Interfaces
Kasper Hornbk

22/04/13

22/04/13

Mainstream GUIs
WIMP = windows, icons, menus, pointers
Direct manipulaJon (Shneiderman, 1983)
Make visible all objects and acJons of interest
Replace complex syntax with direct manipulaJon
of objects
Allow reversible and step-wise acJons, whose
results are immediately visible

WYSIWYG = what you see is what you get

Desktop hardware (e.g., mouse, 21 screen)

22/04/13

Exercise: emerging UIT you know?

Examples of emerging UIs

22/04/13

URP (CHI99)

Reactable

22/04/13

Tracking

22/04/13

Esben Warming Pedersen

22/04/13

Esben Warming Pedersen

22/04/13

Till Ballendat, Nicolai Marquardt, and Saul Greenberg. 2010. Proxemic interacJon: designing for
a proximity and orientaJon-aware environment. In ITS '10. 121-130.

22/04/13

VisualizaJon

hfp://vis.stanford.edu/papers/d3

sixdegrees.hu
10

22/04/13

sixdegrees.hu

Andrews (CHI 2010)

11

22/04/13

WallViz Display

WallViz Display

12

22/04/13

Paper Phone (CHI 2011)

Relief (UIST 2011)

13

22/04/13

Why emerging UIT?

Many reasons to look at emerging UIT
Break limitaJons of GUIs
New technology oer new possibiliJes
Visions of future techs are rarely GUI/WIMP
Know issues in developing for post-WIMP
interfaces (e.g., context-aware compuJng)
Many interesJng CS problems (e.g., tracking,
rendering)

14

22/04/13

Models of emerging UIs

Reality-based interfaces (Jacobs et al. 2008)

Models, contd
Gentner & Nielsen (1996)
Mac

Anti-Mac

Metaphors

Reality

Direct Manipulation

Delegation

Anti-Mac principles are summarized in Table 1.)

Metaphors
The first Macintosh principle
states that the interface should be
based on metaphors with the
See and Point
Describe and Command
familiar noncomputer world
Consistency
Diversity
around us. In the Macintosh interface, computer files are representWYSIWYG
Represent Meaning
ed as documents in paper folders
User Control
Shared Control
that are placed on a desktop. Files
are deleted by dragging them to
Feedback and Dialog
System Handles Details
the trash can. Many recent interForgiveness
Model User Actions
faces have tried to overcome the
limitations of the desktop
Aesthetic Integrity
Graphic Variet
metaphor by extending it to some
other room or building metaphor
Modelessness
Richer Cues
(e.g., Bob or Magic Cap, Figure 1)
Table 1. The Mac and Anti-Mac design principles
or to a village metaphor (e.g.,
eWorld). These 3D designs try to
(mostly office work, though entertainment and
emulate virtual reality on a flat screen but often seem
multimedia applications have been added later in
to introduce a level of clunky indirectness in achievways that sometimes break slightly with the staning common user goals. They are navigationally cumdard interface).
bersome, asking users to go to the other end of
It controlled relatively weak computational
town to pick up their email from the Post Office, and
resources (originally a non-networked computer
interactionally cumbersome, overloading users with
with 128KB RAM, a 400KB storage device, and a
additional windows and other interface elements
dot-matrix printer).
necessitated by the metaphor but not by the users
It was supported by highly impoverished communi- task.
cation channels between the user and the computAlthough the use of metaphor may ease learning
er (initially a small black-and-white screen with
for the computer novice, it can also cripple the interpoor audio output, no audio input, and no other
face with irrelevant limitations and blind the designsensors than the keyboard and a one-button
er to new paradigms more appropriate for a
mouse).
computer-based application. The designers of the
It was a standalone machine that at most was conPhelps farm tractor in 1901 based their interface on a
nected to a printer.
metaphor with the interface for the familiar horse:

15

22/04/13

Models, contd
documents, but not
Mac
Anti-Mac
overwhelmingly many
Users are the rest of us (have no previous
Users are The Post-Nintendo Generation
of them. The various
computer experience)
(grown up with computers)
Macintosh user interface principles work
Work, play, groupware, embedded, and
Office automation productivity applications
ubiquitous
well together and
help the chosen user
Humongous computer (multi-gigabyte RAM,
Weak
computer
(128K
RAM,
68000
CPU)
category achieve its
Cray-on-a-chip RISC processors)
main goals. Since the
Rich communication (computer can see you,
principles are mutuImpoverished communication bandwidth
knows where you are, large high-res screen,
(small screen, keyboard/mouse input)
ally reinforcing, it
new I/O devices)
may not be easy to
Stand-alone
system
that
is
stable
unless
the
Connected system subjected to constant
break just a few as
user decides to make a change
change
changing users and
Manipulation of icons
Language
tasks lead to new
requirements. For
Weak object-orientation (small number of
Strong object-orientation (large number of
example, as argued
large objects with very few attributes)
small objects with rich attribute sets)
earlier, it becomes
Personal information retrieval as unifying
impossible to make
Finder (visible file system) is unifying home
principle with atomic information units as
base, and files are the basic interaction object
all objects visible in
basic interaction object
the interface as sysSurf
your
hard-drive
Information comes to you
tem usage changes to
deal with millions of
The Power to Be Your Best
You won't always have to work that hard
information objects.
But having invisible Table 2. A comparison of the Mac and Anti-Mac interfaces
objects not only
breaks the design principle of see-and-point, it makes characteristics of the original Macintosh and Antidirect manipulation impossible, risks weakening the Mac user interfaces.
chosen metaphors, and breaks perceived stability if
Bruce Tognazzinis Starfire film [19] was an attempt
some objects are made visible anyway at certain times. to show how a high-end workstation might look in the
The Anti-Mac principles outlined here are opti- year 2004. The interface visualized in the film has sevmized for the category of users and data that we eral similarities to our Anti-Mac design: The system
believe will be dominant in the future: people with understands objects well enough to integrate them
extensive computer experience who want to manipu- seamlessly (it can construct a 3D texture map of a perlate huge numbers of complex information objects son from a 2D video image); the screen is very large,
while being connected to a network shared by with fairly expressive interface elements; the computer
immense numbers of other users and computers. has some independent initiative (when the heroine
These new user interface principles also reinforce searches for an article reprint, the agent automatically
each other, just as the Mac principles did. The richer decides to search further and display subsequent artiinternal representation of objects naturally leads to a cles with links to the original article); and the computmore expressive external representation and to er monitors the user (it does not attempt speech
increased possibilities for using language to refer to recognition while she is talking with a colleague).
objects in sophisticated ways. Expert users will be
more capable of expressing themselves to the comESPITE its Anti-Mac aspects, Starfire is still a very recputer using a language-based interface and will feel
ognizable computer with many similarities to curmore comfortable with shared control of the interrent user interfaces and research systems. Indeed,
face because they will be capable of understanding
there are some Anti-Mac features in evidence in
what is happening, and the expressive interface will current commercial products. For example, products
lead to better explanations.
like On Location and SearchIt show autonomy in taking the initiative to index the users file system when
Conclusions
needed and allow users to retrieve email objects by conFirst, to avoid misunderstandings, let us reiterate that tent and other rich attributes even though each email
we are not criticizing the Macintosh interface because message may be part of a large text file when viewed
we think it is bad, but because we view it as a starting through the standard system interface.
point for considering the differing needs future user
Any realistic person who reads the preceding outinterfaces must meet. Table 2 compares some of the line of the Anti-Mac interface will realize it is mostly

Models, contd

Post-WIMP intefaces (van Dam, 1997)

3d interfaces

Natural user interfaces (Wigdor & Wixon,

2011)
Touch and gesture for expert users

Invisible computer (Weiser, 1991)

Invisible servant, making us calm
COMMUNICATIONS OF THE ACM

August 1996/Vol. 39, No. 8

81

16

22/04/13

Exercise: examples of models in use?

Visions
Papers
Put that there, Bolt (1980)
Computer of the 21st century, Weiser (1999)

Videos
Starre (1992)
Nokia vision video (2009, www.youtube.com/
watch?v=AyIgImgDOCU)
Oce vision videos (e.g., www.youtube.com/
watch?v=8Ff7SzP4gfg)

Sketching

17

22/04/13

Henry Fuchs/van Dam

A Vision of TelecollaboraJon: A
Normal Oce

18

22/04/13

19

22/04/13

20

22/04/13

Touch (1965)

Touch (1952)

21

22/04/13

22

22/04/13

And beyond
More other issues
speech, wearable, physiological sensing, shape-
changing physical interfaces, roboJc interfaces,
etc.
Programming/sonware architecture for post-
WIMP interfaces

References/Read more
Andries van Dam: Post-WIMP User Interfaces. In: CommunicaJons of the ACM, 40(2)
(February 1997), pp. 63-67
Shneiderman, Ben. "Direct manipulaJon: a step beyond programming languages,"
IEEE Computer 16(8) (August 1983), 57-69.
Mark Weiser. 1999. The computer for the 21st century. SIGMOBILE Mob. Comput.
Commun. Rev. 3, 3 (July 1999), 3-11.
Richard A. Bolt. 1980. \Put-that-there\: Voice and gesture at the graphics interface.
SIGGRAPH Comput. Graph. 14, 3 (July 1980), 262-270. DOI=10.1145/965105.807503
hfp://doi.acm.org/10.1145/965105.807503

23