Extending Interactive Intelligent Workspace Architectures

with Enterprise Services

Rudi Vernik 1, 2 Terence Blackburn 1 Damien Bright 1
1
School of Computer and Information Science
University of South Australia
Mawson Lakes Blvd
Mawson Lakes South Australia 5059
2
Defence Science and Technology Organisation
PO Box 1500
Edinburgh South Australia 5111

rudi.vernik@dsto.defence.gov.au
terence.blackburn@unisa.edu.au
damien.bright@unisa.edu.au

Abstract
This paper examines the use of ubiquitous and enterprise computing infrastructure for future
workplace environments. In particular, we focus on the integration of Interactive Intelligent
Workspaces (IIWs) into the wider information enterprise. IIWs employ new forms of operating
environments to coordinate and manage interactions between people, multiple display surfaces,
personal information devices, and workspace applications. We argue that if these media rich future
workspaces are to become an integral part of a computing enterprise, consideration needs to be
given to how these elements interface with and use enterprise services and artefacts. In this paper
we propose an architectural reference model called LiveSpaces to guide research and development
activities for enterprise-enabled ubiquitous workspaces. We also discuss our experiences in
developing an initial LiveSpaces implementation. The main contribution of this work to date has
been to build a richer infrastructure to support research into the broader architectural issues related
to the development and use of ubiquitous computing workspaces within enterprise-wide settings.

1. Introduction
One of the driving factors in the trend towards ubiquitous, or pervasive, computing has been the
desire to move computers into the backdrop of natural human interactions [1]. In these future
environments, keyboards and mice, the main human–computer interfaces for the last couple of
decades, will be augmented with more human-centric approaches. New interfaces in these
ubiquitous workspaces will be more intuitive, context aware, and adaptive. They will comprise
natural, human-centric approaches, such as the use of natural language interaction, through the use
of speech, pointing instruments, and gestures, for interacting with multiple displays, new types of
media such as augmented reality, and a range of personal information devices. Computing
applications will migrate from being hosted on workstations to being hosted within physical
environments such as offices, rooms, buildings, and vehicles. In Section 2, we provide an overview
of various projects that are working on making this vision a reality through the application of
ubiquitous computing approaches for the augmentation of physical workplaces. This work is
described as being interactive or intelligent depending on the main focus of the research effort. We
refer to this overall body of work as Interactive Intelligent Workplaces (or IIWs).

This paper extends the work that we [2] and others have done in examining the underlying
architectures and software infrastructure to accommodate these types of future environments and
associated workplace applications. For example, research is well underway in various laboratories
[3,4,5] to more fully understand the underlying infrastructure requirements for IIWs. As with
individual workstations, which require an operating system to coordinate and run applications,
applications within IIWs require an underlying operating environment. Much of the research into
infrastructure for IIWs has focused on support for the coordination and interaction of humans and
devices. Less attention has been given to how these future workplaces might be integrated and used
within the broader enterprise. In this paper, we begin to explore how IIW infrastructure might be
extended with enterprise services and integrated into enterprise-wide solutions.

We argue that if IIWs are to become an integral component of the broader enterprise, consideration
needs to be given to how these elements interface with and use of enterprise services and artefacts.
Some of the considerations include: the use workflows for coordinating and synchronizing
processes and activities; distributed computing support; support for information and knowledge
management; and support for the management and use of enterprise policies and rules in areas such
as security and accountability. Recent work in the area of Enterprise Application Integration, such
as the DSTC work on the Open Distributed Systems Infrastructure (ODSI) [6], has begun to address
how various enterprise services can be assembled into distributed and evolvable enterprise
solutions. We draw on this work to extend the conceptual and architectural basis for future
enterprise-enabled workspaces.

The types of services that support enterprise-wide requirements can also be important for
supporting the activities within a particular workplace. For example, in this paper we describe how
IIWs might be used to support Intense Collaboration Activities such as time critical contingency
planning. Work by Gloria Mark [7] and others focus on the use of project or “war” rooms for
undertaking intensive design activities in areas such as the design of NASA space missions, and for
software design. The appropriate combination of electronic workplace support and new work
processes has shown significant reductions in the time required to undertake these activities.
Simple services such as a publish-subscribe messaging system proved particularly effective at
facilitating communication and collaboration within the environment. Our research begins to look at
the use of other services that may enhance the effectiveness of these types of intense collaboration
both within a workplace and between workplaces. For example, although workflow support would
appear useful in synchronizing activities between workplaces, we have also begun to experiment
with the use of workflow for orchestrating activities within a workplace. In this paper we introduce
the concept of a workplace orchestration service which, in addition to supporting the coordination
and delivery of information to workplace participants and devices, also augments group cognitive
activities that focus on specific goals. Other workplace services such as user-centric interaction
services, speech transcription services, and advanced media services are also considered in terms of
an overall workplace reference architecture.

The paper is structured as follows. After a brief overview of current work in the area of IIWs, we
propose an architectural reference model for enterprise-enabled ubiquitous workplaces. This model
forms the basis of our initial LiveSpaces environment which has been developed at the University
of South Australia e-World Lab. The model draws from background research in the areas of
ubiquitous and enterprise computing. ODSI concepts provide an underlying integration framework
based on the concept of an enterprise bus and the use of peer interfaces to enterprise services such
as workflow engines, metadata services, etc. The intra-workplace architecture is also service based
and identifies several new types of services that would be important in supporting intense
collaborative activities.

We then present our initial LiveSpaces implementation. This environment uses ODSI infrastructure
to support the enterprise enabling aspects and Stanford’s Interactive Room Operating System
(iROS) [8] to support event coordination and data integration within the physical workplace.
LiveSpaces has been used to conduct a series of design collaborations and briefings. We report on
our preliminary findings, current and future work. To date, the main contribution of this work has
been to build a richer infrastructure to support research into the broader architectural issues related
to the development and use of ubiquitous computing workspaces within enterprise-wide settings.

2. Interactive Intelligent Workplaces

A community of ubiquitous computing researchers has been focusing their attention on the
technology augmentation of physical collaborative workspaces such as meeting rooms. Examples
of research initiatives in this area include Stanford’s Interactive Workspaces [3], GMD’s i-Land[5],
MIT’s Intelligent Room[4], and Active Spaces at the University of Illinois Urbana Champaign
(UIUC)[9].

Each of the projects has a particular research focus and has developed a core infrastructure to
support their specific objectives. For example, the Interactive Workspaces project focus on the
coordination of interactions between people, multiple (large screen) displays, mobile personal
devices such as PDAs and laptops, and workspace interaction devices such as buttons, sliders, and
wands [10]. An infrastructure called the interactive Room Operating System (iROS) [11] has been
developed to support ubiquitous control of applications across a workplace, the movement of data
across applications and devices, and dynamic application coordination. The underlying architectural
metaphor for iROS is a blackboard with two levels of data abstraction implemented using an
extended tuple space approach. iROS also implements a persistent context memory that allows any
process to store and retrieve data that will be relevant across applications and sessions [12]. iROS
includes several other components including iCrafter [13], a system that provides support for
service advertisement and invocation and for the generation of services user interfaces, and
PatchPanel [14], a mechanism for integrating heterogeneous software and hardware components
into a ubiquitous workspace. The Interactive Workspaces project focuses on the use of flexible
lightweight software infrastructure to provide the basis for rapid integration and coordination of
many common business applications (e.g. Microsoft Office) with ubiquitous computing approaches
and devices within rich interactive spaces.

The iLand project being undertaken by the Ambiente Group at GMD-IPSI in Darmstadt, Germany
[15,16] is somewhat similar in focus to the Interactive Workspaces Project in that it focuses heavily
on human interaction aspects. The project has developed a range of Roomware components,
including the Interac Table, CommChairs, and DynaWall (a large screen interactive display). The
work focuses on the development of custom applications rather than integrating “off the shelf”
applications as is the case for the Interactive Workspaces project. Applications for the iLand
environment are developed using the BEACH infrastructure [17] and conceptual model. The
BEACH system, developed in Smalltalk, supports object sharing to allow multiple users to
simultaneously interact with a particular information object and to support interactions with
multiple display surfaces as if they were one large canvas . The Beach architecture uses a layered
approach with four main abstractions: the core, model, generic, and task levels. A Beach
conceptual model defines the various classes of objects used including data, application, user
interface and interaction. A design methodology is provided to help guide the development of
custom ubiquitous computing applications.

MIT’s Intelligent Room project uses an agent-based architecture called MetaGlue [18] to provide
computational glue for large groups of collaborative software agents and device controllers. A
primary focus of the project is in the area of context awareness to allow environments to be aware
of, and respond automatically to, users’ needs (designated a type of intelligence). The work also
focuses on natural modes of human-computer interactions including the use of speech and gestures.
The Active Spaces project at the UIUC takes primarily an infrastructure focus. The approach is
based on the use of a meta-operating system for ubiquitous computing rooms called GaiaOS [19], a
CORBA-based system that provides object exchange, a publish-subscribe type event service, and
other core services such as context, presence, a space repository (using CORBA Trader), and a
context file system. The concept of a Unified Object Bus (UOB) is a key underlying abstraction.
Gaia OS uses an extended version of the model-view-controller approach (MVC) called MPACC
(Model, Presentation, Adapter, Controller, Coordinator) to guide application development. The
Active Spaces work takes more of an enterprise computing perspective than the other projects
discussed in that the focus is on extending the concepts of traditional operating systems to support
heterogeneous distributed ubiquitous spaces, where a space could range from a single mobile user
through to collaborative environments such as meeting rooms and classrooms.

Much of work on IIWs to date has focused on individual workspaces. Projects such as Stanford’s
Interactive Workspaces and MIT’s Intelligent Room are beginning to consider extending their
infrastructures to enable integration between multiple workspaces and the outside world. For
example, MIT is working on Hyperglue [20], an extension to their agent-based infrastructure.
Stanford has also recently begun a project called Event Spaces [21] which looks at extensions to
their iROS infrastructure. Our work on LiveSpaces looks at the enabling aspects of IIWs. We
consider issues of integration, communication, and coordination within and between workspaces
but also look towards future concepts such as the introduction of services such as transcription,
orchestration (a service that supports procedural and cognitive aspects of team interactions), and
simulation within learning environments which can adapt and evolve to meet individual and
enterprise needs.

3. LiveSpaces Reference Model

This section describes an architectural reference model that we have developed to help set the
context for and to guide our research activities. Figure 1 provides the setting for our research. We
are interested in how technology augmented physical spaces such as IIWs can be used as an integral
component of an overall enterprise system. One of our domains of interest is support for large-scale
distributed planning situations such as the deployment of peace-keeping forces, the rapid evacuation
of citizens from areas of political unrest, and the planning for disaster relief situations such as
cyclones. A typical scenario might unfold as follows. A cyclone devastates one of our Pacific
neighbours. The Australian government is called on to provide disaster relief through the provision
of medical support, food and water, and engineering support to re-establish services. Planning is
time-critical. Planning teams are assembled quickly to address issues such as the identification and
tasking of required personnel, provision of transport, and the sourcing of supplies. They collocate in
available spaces which can include command centres, meeting rooms, lunch rooms, and even tents
for large-scale operations. Some specialist teams may also be geographically remote from the main
planning location. Each team must develop plans for their specific area of interest from information
that they bring with them or sources accessed from supporting information systems. Coordination
within and between teams is particularly important. Teams need to synchronise periodically so that
information developed by one team can serve as inputs to other teams. Awareness is critical to
success. Teams need to have awareness of approaches being proposed by other teams,
synchronisation points, refreshment breaks, and progress. Teams work almost around the clock to
achieve desired goals.

In supporting these types of intense collaborative activities, key aspects to be considered include the
synchronisation of distributed workspaces and mechanisms that allow individual workspaces to
draw from and contribute to various enterprise services and resources. A core architectural
abstraction has been defined to provide support for these aspects: the Enterprise Bus. Our concept
of an Enterprise Bus is influenced by the approach developed by DSTC in relation to ODSI. The
Enterprise Bus forms the basis for both system and enterprise integration. System integration
involves providing mechanisms for aspects such as communication and deals with considerations
such as the interoperability between heterogeneous components and dynamic systems evolution.
The Unified Object Bus of Active Spaces and the Information Bus [22] address these types of
systems aspects. Enterprise integration is a much broader concept. Enterprises involve
communities of cooperating people and processes, helped along the way by technology [23].
Therefore, enterprise integration involves an understanding of the roles and attributes of people, the
processes they are performing and the policies and rules that guide them. An Enterprise Bus must
therefore employ mechanisms that support heterogeneous systems integration and coordination as
well as providing mechanisms that support evolution through changes in organisational, process,
and governance aspects.

Workspace
Figure 1. Context for Enterprise Enabled Ubiquitous Workspaces.

Figure 2 shows the reference architecture for a particular workspace (or LiveSpace instance). The
architecture has at its core a Workspace Infrastructure which integrates with the broader enterprise
by way of the Enterprise Bus. The Workspace Infrastructure provides for the coordination and
integration of applications, services and devices within a workspace, much like an operating system
might provide for personal computers. It provides support for event management, information
management, and services invocation across the workspace. The reference architecture identifies
two sets of key services: knowledge services, and workspace support services. Knowledge services
provide support for those aspects that help make a workspace “intelligent”. These could include
instrumentation services that capture and provide information on usage and interaction within a
workspace, context services that have knowledge of the participants and activities, and learning
services that maintain symbolic representations to facilitate and automate workspace functions.
Workspace Support Services provide capabilities that directly support collaborative activities. For
example, we are currently working on services that can be invoked to automatically transcribe
speech; media services that allow information to be presented using various technologies such as
personal devices, large interactive displays and augmented reality; interaction services that
coordinate various modes of interaction including speech, gestures, touch, and gaze; and
orchestration services that, in addition to supporting the coordination and delivery of information to
workplace participants and devices, facilitates group cognitive activities in relation to specific goals.
 Knowledge
Figure 2. LiveSpaces Reference Architecture

So how might LiveSpaces support the type of activities outlined in the planning scenario above?

Services
Initially, participants of a pre-planning phase might use a LiveSpace to analyse the situation and
develop an intent (or set of goals). They might do this by using media services to present maps,
imagery, and models on various displays to aid situational understanding. Collaborative authoring
applications together with transcription services might aid in capturing and documenting intent.
Knowledge services might provide expert advice based on previous similar operations. Drawing on
enterprise services through the enterprise bus, participants might define and update the parameters
for the planning teams, including the composition and organisational structure of the teams, roles,
processes and schedules (including aspects such as synchronisation points). Orchestration services
might aid the pre-planning team by guiding them through the procedural and cognitive aspects of
their task.

Contex
Several planning teams might then be formed to address specific aspects of the operation such as
transport, personnel, medicine, and engineering. Some might use existing LiveSpaces augmented
rooms within the headquarters complex. Other physical spaces might need to be rapidly augmented
with required technologies and infrastructure. Some specialist teams or individuals might join in
from geographically remote locations. Teams would begin their intense collaborations being
supported by various workspace services and applications to help them creatively develop solutions,
capture decisions, and produce planning products. Underlying enterprise services would help
provide and manage required information artefacts, help synchronise the activities of the various
teams, and provide team awareness.

4. LiveSpaces Implementation
We have begun to experiment with these concepts by way of our initial LiveSpaces implementation
at the University of South Australia’s e-World Lab (shown in Figure 3). One of the LiveSpaces
project aims is to use devices and infrastructure that allow the rapid technology augmentation of
physical spaces. This is particularly important for the types of activities outlined in our scenario in

Learnin
Section 3. We use a range of “off the shelf” technologies such as interactive Smart Board displays
[24], projectors, laptops, and PDAs. Wireless infrastructure plays an important part in providing
connectivity between the various devices. We have also been experimenting with technologies for
providing passive tracking of devices and people within workspaces using a camera array, fiducial
marker cards, and augmented reality approaches. We use technologies such as X10 [25] for
controlling lights and other electrical devices such as the coffee machine.

Camera Arr

Figure 3. Current e-World Lab LiveSpace Implementation

The LiveSpaces software infrastructure that we are currently using is shown in Figure 4. The
Enterprise Bus is implemented using elements of DSTC’s ODSI infrastructure. A core component
is Elvin [26,27], a publish/subscribe system that uses content-based routing. Elvin is a very
efficient and generic system that provides an effective mechanism for communication and
coordination between workspaces. Elvin also provides a basis for accessing and coordinating
underlying enterprise services through the use of Peers. Peers are generic software elements that
act together (through Elvin) as application servers to discover and manage the use of services.
Service wrappers, written in Python [28], provide the interface between peers and services. This
flexible arrangement allows us to experiment with the introduction of a wide range of enterprise
services including workflow, speech services, and other web information services. We have used
integrated workflow services as part of our initial investigations into procedural orchestration of
LiveSpace activities such as project reviews. Since our Elvin Server is federated across the internet,
events emanating from the e-World LiveSpace can be monitored and used at other distributed
locations such as DSTO Edinburgh and DSTC Brisbane. This capability will provide the basis for
further investigations into the synchronisation and coordination of distributed workspaces.

We use Stanford’s iROS as the basis of our workspace infrastructure for integrating and
coordinating devices and information within a LiveSpace. As discussed in Section 2, the key
architectural abstraction for iROS is a blackboard implemented using an extended tuple space
approach. The Event Heap [29] (based on the use of tuple spaces [30]) provides fast distribution of
simple event tuples. Any process can post tuples. Process can include those that handle inputs
from various input devices such as wireless switches, speech recognisers, keyboards, and motion
sensors. Tuples include fields for their source and timestamp, along with explicit data associated
with the event. Any process can subscribe to a pattern of field values and receive callbacks when
an appropriate tuple is posted. iROS provides interfaces for posting and receiving of events. Tuples
have a time-to-live and so after a specified period can be deleted. The Event Heap is generally
used for short-lived items, with time-to-live extending at most through an interaction session. The
iROS Dataheap provides the basis of storing and sharing of long-lived items for a session or
between sessions such as documents. We also use iROS iCrafter for generating interfaces to
services such as projector controllers or interfaces to X10-controlled devices such as lights. A
number of iROS applications are used in the LiveSpace to support interactive multibrowsing of web
content, and the control of mouse interaction by multiple users/devices across multiple display
surfaces (PointRight). We have also been developing our own workspace application such as
Plexus, a room desktop that provides a universal session interface that allows LiveSpace users to
plan and initiate collaborative activities, control devices within the room, and to control the display
and movement of information within a workspace.

Figure 4. LiveSpaces e-World Infrastructure

By extending the Workspace Infrastructure through ODSI, we have developed a platform for
experimenting with many of the enabling aspects outlined in Section 2. The use of a workflow
service has allowed us to coordinate many of the procedural aspects of collaborative sessions such
as research review meetings. The introduction of speech services has allowed us to use speech
synthesis for guiding activities and for using speech recognition to control the process. The
Enterprise Bus not only gives us the ability to coordinate activities beyond the walls of a particular
workspace, but provides a mechanism for communicating awareness to other spaces, and for
communication between individuals (using chat-type capabilities). Enabling our workspace with
these types of capabilities has allowed us to consider new types of meta applications. For example,
e-Ghosts is an application which automates the presentation and demonstration of the e-World
LiveSpace environment and its related projects. Workflows coordinate integrated presentations
across various displays and devices, augmented by speech synthesis and changes to ambient
conditions such as lighting. A range of materials feature in the presentations including images,
Powerpoint slides, models, videos, and live demonstrations.
5. Conclusion
In this paper we have examined the use of ubiquitous and enterprise computing infrastructure for
future workspace environments. One of the driving factors in the trend towards ubiquitous, or
pervasive, computing has been the desire to move computers into the backdrop of natural human
interactions. Several projects are underway worldwide to more fully understand the architecture and
infrastructure requirements of technology-augmented physical spaces such as meeting rooms. The
work typically focuses on new approaches for human interaction and automation (often described as
intelligence). A significant amount of work has been done to define the workspace operating
environments that will be required to support ubiquitous computing applications. These
applications will interface with multiple displays and devices using a variety of human interaction
modalities such as touch, speech, gestures, and gaze. In this paper we have considered how these
future ubiquitous workspaces might be used an integral component of the broader enterprise. Some
of the considerations include the interfaces between workspace and enterprise infrastructure, the use
of workflow services for coordinating and synchronising activities within and between workspaces,
support for information and knowledge management; and support for the management and use of
enterprise policies and rules. We have discussed these aspects in relation to a reference architecture
called LiveSpaces that we have developed to better characterise related research activities and to
help guide our own research initiatives. We have also discussed the results of an initial LiveSpaces
implementation that we have developed and used at the University of South Australia eWorld Lab.

Our work to date has involved developing the underlying infrastructure, services, and applications
for LiveSpaces. We are currently working on defining and developing session interfaces,
ubiquitous visualisation/media services, interaction services, and transcription services. The
commencement of the DSTC South Australia node will draw on and extend LiveSpaces concepts
and infrastructure in three key domain areas: defence, health, and large-scale software development.
The Defence domain project will focus on Augmented Synchronised Planning Spaces, focusing
specifically on the types our issues discussed in this paper when planning for operations such as
disaster relief, peacekeeping, and the rapid evacuation of national from areas of political unrest.

The main contribution of this work to date has been to build a richer infrastructure to support
research into the broader architectural issues related to the development and use of ubiquitous
computing workspaces within enterprise-wide settings.

6. Acknowledgements
The work to date has involved many talented individuals. We would like to acknowledge Vivien
Nguyen. and Michael Vernik for their work in introducing the iROS infrastructure and applications
into LiveSpaces, Stephen Johnson for his work on integrating ODSI and workflow, and Ben Fry
integrating speech interfaces. Other important contributors to the work include Hannah Slay for her
work on augmented reality and passive tracking, Eugene Kharabash for his support in managing the
laboratory, and Bruce Thomas for his ongoing support and inputs. We would like to thank
environment DSTO for supporting the research, DSTC for their support with ODSI, and Stanford
University for their ongoing support.

7. References
[1] Weiser, M. (1991) The computer of the 21st Century. Scientific American, Sept. 1991, 94-104.

[2] Blackburn, T., Vernik, R. and Bright, D. (2002). Enterprise-Enabled Ubiquitous Workspaces. in
Proceedings of the ACS CONFERENCE-Asia Pacific Forum on Pervasive Computing, Adelaide Nov 2002.

[3] Web site for Stanford Interactive Workspaces project, http://www.graphics.stanford.edu/projects/iwork/

last visited June, 2003
[4] MIT Intelligent Room Project, http://www.ai.mit.edu/projects/aire/projects.shtml#e21, accessed 30 June
2003.

[5] Ambiente Project, http://www.darmstadt.gmd.de/ambiente/i-land.html, last accessed 30 June 2003

[6] BOND, A. (2001) ODSI: Enterprise Service Co-ordination.

www.dstc.edu.au/Research/Projects/Integrator/documents/DOA-abstract.htm visited June, 2002

[7] Mark, G. (2002) Extreme Collaboration. Communications of the ACM,. 45(4).

[8] B. Johanson, A. Fox, and T. Winograd (2002) The Interactive Workspaces Project: Experiences with
Ubiquitous Computing Rooms, IEEE Pervasive Computing Special Issue on Overviews of Real-World
Ubiquitous Computing Environments, April-June 2002

[9] Active Spaces Project, http://www.cs.uiuc.edu/news/archives/resactsp.html, last accessed 30 June 2003

[10] Ballagas, R., et al. (2003) iStuff: A Physical User Interface Toolkit for Ubiquitous Computing
Environments. in CHI 2003: Human Factors in Computing Systems. Fort Lauderdale, FL, USA: Association
for Computing Machinery.